Progress Reward Model for Reinforcement Learning via Large Language Models

Key points

• Traditional reinforcement learning (RL) algorithms face significant limitations in handling long-term tasks with sparse rewards.1
• Recent advancements have leveraged large language models (LLMs) to enhance RL by utilizing their world knowledge for task planning and reward generation.1
• For low-level reward generation, inspired by potential-based reward shaping, we use the progress function to construct a Progress Reward Model (PRM), providing theoretically grounded optimality and convergence guarantees, thereby enabling effective policy optimization.1
• Experimental results on robotics control tasks demonstrate that our approach outperforms both LLM-based planning and reward methods, achieving state-of-the-art performance.1

Sources (1)

• [1]Progress Reward Model for Reinforcement Learning via Large Language Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 03:21 PM

  In this paper, we propose the Progress Reward Model for RL (PRM4RL), a novel framework that integrates task planning and dense reward to enhance RL.

  Traditional reinforcement learning (RL) algorithms face significant limitations in handling long-term tasks with sparse rewards.

Extractive summary: sentences quoted from the sources.