AnomalyJev: Towards Evaluating Image Anomaly Explanations via Claim-Level Verification
Recent multimodal LLM (MLLM)-based anomaly detection (AD) methods can generate textual explanations that help interpret their predictions.
ProofPaper ↗
Key points
- However, reliably and efficiently evaluating these explanations remains challenging.
- We instead propose to anchor evaluation in the available ground truth by extracting verifiable claims from each explanation and verifying them against the corresponding annotations.
- We instantiate this idea as AnomalyJev, a lightweight framework that extracts verifiable claims from contextualised representations using ground-truth-defined queries and pointer based heads, without token generation.
- For evaluation, we construct a human-annotated set of 640 explanations with over 1,600 expert ratings across in-domain and cross-dataset settings.
Sources (1)
- [1]AnomalyJev: Towards Evaluating Image Anomaly Explanations via Claim-Level VerificationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 03:29 AM
Recent multimodal LLM (MLLM)-based anomaly detection (AD) methods can generate textual explanations that help interpret their predictions.
However, reliably and efficiently evaluating these explanations remains challenging.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026MPS in PyTorch 2.15
- Oct 19, 2026Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM
- Oct 19, 2026Keep the Lineages Alive: Diversity-Preserving SMC for Diffusion LLM Decoding
- Oct 19, 2026VerifierBench: Benchmarking the Generation of Executable Verifiers for LLM Agents
- Oct 18, 2026Editing as Language: Teaching LLM Agents a Grammar for Native Context Editing
- Oct 18, 2026System X: An LLM Decision Model That Replies from Its Own KV Cache