RASSG: Repair-Aware Semantic Survival Gating for Streaming Tool Execution

Key points

• Streaming speech assistants can launch tools before users finish speaking, overlapping execution with incoming speech to reduce user waiting time.1
• RASSG launches a tool when three consecutive speech recognition results arrive with identical nonempty arguments, then combines calibrated argument invalidity risk with execution costs to decide cancellation at the last feasible opportunity before effects become irreversible.1
• Evaluation on ReCAST, FDB-v3, and Audio2Tool uses a controlled execution clock with fixed tool runtimes and cancellation parameters.1
• Ablations with identical initial launches support active cancellation, showing the value of controlling ongoing calls before their effects become irreversible.1

Sources (1)

• [1]RASSG: Repair-Aware Semantic Survival Gating for Streaming Tool Execution
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 07:17 AM

  We propose Repair-Aware Semantic Survival Gating (RASSG) to jointly control tool launch and cancellation during streaming speech.

  Streaming speech assistants can launch tools before users finish speaking, overlapping execution with incoming speech to reduce user waiting time.

Extractive summary: sentences quoted from the sources.