The Butterfly Effect in LLM Social Simulations: From Architecture Sensitivity to Robustness Audits

Key points

• A butterfly effect occurs when a local perturbation propagates through a system and produces divergence in its collective outcome.1
• In LLM social simulations, this creates a particular challenge for scientific inference: sensitivity to implementation choices can enter agent interactions and alter the collective outcomes on which scientific claims are based.1
• In the echo-chamber simulation, persona representation changes interaction targeting and is accompanied by divergence in collective interaction structure, extending representation sensitivity from agent behavior to network-level outcomes.1
• We introduce TRAILS (Taxonomy for Robustness Audits In LLM Simulations) and the Social Simulation Audit Card as a framework for systematically auditing these dependencies and calibrating the scope of scientific claims to the robustness evidence that supports them.1

Sources (1)

• [1]The Butterfly Effect in LLM Social Simulations: From Architecture Sensitivity to Robustness Audits
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 11:13 PM

  We find that controlled perturbations can substantially alter collective outcomes, with sensitivity varying across both perturbations and models.

  A butterfly effect occurs when a local perturbation propagates through a system and produces divergence in its collective outcome.

Extractive summary: sentences quoted from the sources.