Humans vs. Machines: A Comparison With AI Agents in Software Reverse Engineering

Key points

• Do they adopt the same strategy, and can agents scale to real-world programs?1
• We compare human and LLM-based software reverse engineering (SRE): nine models tackle two compact static-analysis challenges previously solved by human analysts, scored with a common understanding rubric, and we record their complete interactions with IDA Pro.1
• Because greedy fetching aggregates large amounts of code in the agent's context, we measure context-window saturation across 4,000 real-world binaries: large windows accommodate many programs, but larger applications and lower-level representations quickly exhaust them.1
• Autonomous reverse engineering is becoming competitive, with a large speed advantage, but scaling it will require better navigation, selective retrieval, and persistent analysis state, not merely larger context windows.1

Sources (1)

• [1]Humans vs. Machines: A Comparison With AI Agents in Software Reverse Engineering
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 02:32 PM

  Can an autonomous AI agent reverse engineer a binary as well as a skilled human analyst?

  Do they adopt the same strategy, and can agents scale to real-world programs?

Extractive summary: sentences quoted from the sources.