ERFSL uses LLMs to auto-tune reward functions 5x faster

A team of researchers from multiple institutions has introduced ERFSL (Efficient Reward Function Searcher via Language Models), a system that leverages large language models (LLMs) to automate the tedious process of designing reward functions for reinforcement learning in custom environments. The approach addresses multi-objective optimization, where agents must balance competing goals like speed, safety, and energy efficiency.

ERFSL works in three steps: first, an LLM (tested with GPT-4o mini) generates reward function components based on explicit user requirements. Then a reward critic reviews and corrects the code—requiring only one feedback iteration per requirement. Finally, a weight optimizer iteratively adjusts the importance of each reward component using textual logs from training. In simulation benchmarks, even when a weight was off by a factor of 500, the system needed just 5.2 iterations on average to satisfy user requirements. This makes reward design accessible without deep reinforcement learning expertise, accelerating the development of AI controllers for robotics, autonomous vehicles, and industrial systems.