Provably Efficient Risk-Sensitive Offline Reinforcement Learning via Optimized Certainty Equivalents

Key points

• Learning risk-sensitive policies from fixed datasets is important when additional interaction with the environment is costly.1
• In this work, we investigate efficient offline risk-sensitive RL using the optimized certainty equivalent (OCE), a general class of risk measures.1
• Under the tabular setting, we develop pessimistic value-iteration algorithms for both formulations and establish tight sublinear suboptimality bounds, explicitly capturing how risk sensitivity, encoded by utility functions, affects learning difficulty.1
• To handle large state spaces, we further develop pessimistic least-squares value-iteration algorithms that incorporate the general OCE risk criterion into linear function approximation.1

Sources (1)

• [1]Provably Efficient Risk-Sensitive Offline Reinforcement Learning via Optimized Certainty Equivalents
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 01:41 PM

  Risk-sensitive reinforcement learning (RL) addresses decision-making under uncertainty and adverse outcomes.

  Learning risk-sensitive policies from fixed datasets is important when additional interaction with the environment is costly.

Extractive summary: sentences quoted from the sources.