SurgStreamer: proactive, request-driven monitoring of surgical video streams with structured state and a reusable reasoning layer

Key points

• Surgical teams could benefit from assistants that watch the video for requested events, such as "tell me when the clip applier appears during Calot triangle dissection", and answer questions about what has already happened.1
• It separates procedure-specific perception, which maps the 1-FPS stream to a structured state over tools, actions and phases, from procedure-agnostic reasoning, which evaluates a formal monitoring condition over this state and an event memory, either with exact rules or with a small controller that addresses entities through pointers rather than names.1
• The tested configurations of the streaming vision-language model OneStreamer-4B, including ones fine-tuned on SurgStreamer's state or on video with an adapted vision encoder, scored at most 0.540 (Holm-adjusted p <= 0.012).1
• On a new procedure, pituitary surgery, controller weights trained on cholecystectomy matched an in-domain controller on oracle state (0.947 vs 0.938).1

Sources (1)

• [1]SurgStreamer: proactive, request-driven monitoring of surgical video streams with structured state and a reusable reasoning layer
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 01:52 PM

  We formalise this task as request-driven event detection with exact semantics and causal alert decisions, and present SurgStreamer.

  Surgical teams could benefit from assistants that watch the video for requested events, such as "tell me when the clip applier appears during Calot triangle dissection", and answer questions about what has already happened.

Extractive summary: sentences quoted from the sources.