Recording-Aware Evaluation of Multimodal OT Intrusion Detection

Key points

• Machine-learning-based intrusion detection systems (IDSs) for operational technology (OT) are commonly evaluated on benchmark datasets in which normal and attack data are collected in separate recordings.1
• In such settings, high performance may reflect recording-associated differences as well as attack behavior.1
• On the ProvICS multimodal dataset, cyber modalities strongly distinguish the benign and attack recordings before the first labeled attack.1
• An external analysis on HAI shows smaller and reference-dependent recording sensitivity.1

Sources (1)

• [1]Recording-Aware Evaluation of Multimodal OT Intrusion Detection
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 06:59 PM

  We introduce a recording-aware evaluation procedure that compares attack windows with background from the same recording, uses prior data for model fitting and calibration, tests temporal alignment against randomized event schedules, and reports event coverage together with alarm burden.

  Machine-learning-based intrusion detection systems (IDSs) for operational technology (OT) are commonly evaluated on benchmark datasets in which normal and attack data are collected in separate recordings.

Extractive summary: sentences quoted from the sources.