Beyond Behavioral Accuracy: Evaluating Individual-Level Fidelity in LLM Investor Simulation
Large language models (LLMs) are increasingly used as user simulators, but can they faithfully reproduce an individual's behavior and decision logic over time?
ProofPaper ↗
Key points
- We investigate this question in a longitudinal paper-trading study with 80 participants.
- Using rolling next-day predictions, we evaluate trading activity, buy-sell structure, asset selection, and portfolio outcomes, complemented by controlled input ablations and participant-rated explanation alignment.
- No evaluated LLM reliably outperforms a simple recent-activity persistence baseline for predicting trade occurrence, while buy-sell structure and asset selection remain difficult to reproduce.
- These findings reveal a gap between coarse behavioral agreement and individual-level fidelity, motivating separate evaluation of behavioral accuracy, explanation alignment, and evidence dependence in personalized user simulation.
Sources (1)
- [1]Beyond Behavioral Accuracy: Evaluating Individual-Level Fidelity in LLM Investor SimulationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 02:54 PM
Large language models (LLMs) are increasingly used as user simulators, but can they faithfully reproduce an individual's behavior and decision logic over time?
We investigate this question in a longitudinal paper-trading study with 80 participants.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026GAIDA: Generative AI for Human-in-the-Loop Document Annotation
- Oct 19, 2026From Black Box to Controller: Inference-Time Steering of LLM via Feature Amplification
- Oct 19, 2026AdaExploreBench: Benchmarking LLM Agents for Proactive Exploration and Test-time Adaptation in Evolving Environments
- Oct 18, 2026Outcome-Anchored Credit Reallocation for Agentic Knowledge Graph Question Answering
- Oct 17, 2026MisleadQA: Diagnosing LLM Failures under Misleading Context
- Oct 17, 2026Refuse the Harm, Complete the Task via Graph-of-Rules Planning for LLM-Enabled Robots