ExeBench-FHD: Addressing Fidelity Issues in Decompiler Evaluation
We present ExeBench-FHD, a backward-compatible revision of ExeBench that adds 16 new fields: call-site assembly, data sections, test wrappers, linked-binary disassembly and five warning flags.
ProofPaper ↗
Key points
- ExeBench is one of the most widely used benchmarks for evaluating LLM-based decompilers.
- We analyze ExeBench and identify nine issues that affect how faithfully a test measures a decompiler.
- Three of them prevent LLM decompilers from recovering C code from ExeBench's function.
- Six more affect test validity: non-void functions without a return, placeholder types, static or inline functions removed by the optimizer, functions named main, trivial bodies and truncated definitions.
Sources (1)
- [1]ExeBench-FHD: Addressing Fidelity Issues in Decompiler EvaluationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 01:49 PM
We present ExeBench-FHD, a backward-compatible revision of ExeBench that adds 16 new fields: call-site assembly, data sections, test wrappers, linked-binary disassembly and five warning flags.
ExeBench is one of the most widely used benchmarks for evaluating LLM-based decompilers.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026Voxtral 3
- Oct 20, 2026[AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback
- Oct 20, 2026Cache Engineering for Agents
- Oct 19, 2026Introducing Clairic, a new System One class of extraction models
- Oct 19, 2026Launch HN: Laminar (YC S24) – RL-trained model that finds errors in agent traces
- Oct 19, 2026AdaExploreBench: Benchmarking LLM Agents for Proactive Exploration and Test-time Adaptation in Evolving Environments