Sensitivity or Criterion? Measuring and Moving When Language Models Act
We introduce FlipAct, a diagnostic of tool use in physical states.
ProofPaper ↗
Key points
- Before an agent acts, it must decide whether action is warranted.
- Signal-detection theory separates sensitivity from criterion, but criteria estimated from authored binary labels depend on each benchmark's example mix.
- A rule over annotated tool properties computes each answer and a graded evidence score, fixing the Yes/No boundary at zero across models.
- The slope of the fitted Yes-rate curve measures sensitivity; its crossing at one half measures criterion.
Sources (1)
- [1]Sensitivity or Criterion? Measuring and Moving When Language Models ActarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 11, 07:36 AM
We introduce FlipAct, a diagnostic of tool use in physical states.
Before an agent acts, it must decide whether action is warranted.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 11, 2026A Sovereign, Indic-First, Reasoning Model
- Oct 11, 2026Apertus 1.5 Release Notes
- Oct 10, 2026UltraBERT: Beyond the Limit of BERT Pre-Training
- Oct 9, 2026AgentTune: A Unified Training Lifecycle for Tool-Using LLM Agents
- Oct 9, 2026Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation
- Oct 9, 2026Continually Post-training Small Models via Synthetic Self-distillation