OnTrack: Real-Time Monitoring and Intervention in LLM Agent Trajectories via Streaming Structure-Aware Optimal Transport
To overcome this, we propose OnTrack, a streaming monitoring mechanism that compares an agent's steps and dependencies against recorded successful runs to alert users or block the agent in about a millisecond per step.
ProofPaper ↗
Key points
- Agents are deployed in applications from trip planners and stock trading to IT incident triage.
- In most cases, LLM agents work autonomously with minimal rule-based safeguarding, leading to cost and safety issues from irreversible actions.
- We study this problem in three regimes of decreasing access: full reference access (historical runs and tool schemas), intermediate access (only tool schemas), and no prior knowledge (only step logs as generated).
- Finally, we evaluate OnTrack using SWE-bench trajectories.
Sources (1)
- [1]OnTrack: Real-Time Monitoring and Intervention in LLM Agent Trajectories via Streaming Structure-Aware Optimal TransportarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 05:32 PM
To overcome this, we propose OnTrack, a streaming monitoring mechanism that compares an agent's steps and dependencies against recorded successful runs to alert users or block the agent in about a millisecond per step.
Agents are deployed in applications from trip planners and stock trading to IT incident triage.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 8, 2026The State of AI Report 2026
- Oct 8, 2026ConwayResearch/Underdog-Saluki-27B-1.0
- Oct 8, 2026Opera: A Verbal Critic Framework for Long-horizon Coding Agents
- Oct 7, 2026Agent Lightning v1.0: A 3,500-Line Lightweight Agentic RL Framework for Training Agents with Real Harnesses
- Oct 7, 2026Code Understanding is a Bottleneck for Coding Agents
- Oct 7, 2026Coding-Agent Benchmarks Should Match Their Users' Task Flows