REPORT-OPS: Hybrid Evidence-Guided Reporting with Human-Supervised Refinement for HPC

Key points

• Operational reporting for high-performance computing (HPC) facilities requires combining information from monitoring services, schedulers, incident and maintenance records, storage measurements, periodic reports, and operator investigations.1
• Producing a reliable report requires more than retrieving relevant records: the evidence must refer to the correct system, metric definition, time period, and source of authority.1
• We evaluate REPORT-OPS on 68 sanitized reporting queries constructed from 50 base scenarios in a frozen HPC operational corpus, using three agent models and two LLM judges.1
• Across five rounds of human-supervised refinement, localized policy updates correct 91.17% of targeted failures and improve performance on related held-out cases by 9.33 percentage points, with no observed regression breaks under release gating.1

Sources (1)

• [1]REPORT-OPS: Hybrid Evidence-Guided Reporting with Human-Supervised Refinement for HPC
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 13, 09:40 PM

  We present REPORT-OPS, a hybrid reporting agent in which an LLM interprets requests, plans evidence collection, evaluates possible explanations, and drafts reports, while author-seeded operational policies and automated checks determine whether the available evidence supports the requested conclusion.

  Operational reporting for high-performance computing (HPC) facilities requires combining information from monitoring services, schedulers, incident and maintenance records, storage measurements, periodic reports, and operator investigations.

Extractive summary: sentences quoted from the sources.