Conformal Sequential Decision Making With Cost and Sublinear Policy Regret Guarantees

Key points

• Automated decision makers, such as AI agents that control robots or network devices, must often satisfy hard cost constraints while maintaining competitive performance.1
• We formulate this setup as an episodic sequential decision problem in which the state sequence is arbitrary, and the reward and cost may depend on past actions within the same episode.1
• We prove that OCEA satisfies per-episode cost constraints for every state sequence and for every realization of its internal randomness, while achieving a sublinear expected policy regret with respect to the best action sequence in hindsight from a comparator class satisfying the same per-episode cost constraints.1
• We apply OCEA to risk-averse control and online compression with distortion guarantees.1

Sources (1)

• [1]Conformal Sequential Decision Making With Cost and Sublinear Policy Regret Guarantees
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 03:46 PM

  We propose the online conformal episodic agent (OCEA), a policy that selects one candidate action per episode using a regret-controlled algorithm, and switches to a zero-cost action when the remaining budget cannot absorb the maximum cost of one more round.

  Automated decision makers, such as AI agents that control robots or network devices, must often satisfy hard cost constraints while maintaining competitive performance.

Extractive summary: sentences quoted from the sources.