Outcome-Anchored Credit Reallocation for Agentic Knowledge Graph Question Answering
Agentic knowledge graph question answering (KGQA) requires large language models (LLMs) to solve complex questions through multi-turn interactions with a knowledge graph (KG).
ProofPaper ↗
Key points
- Outcome-supervised reinforcement learning derives verifiable outcome rewards from final answers but uniformly assigns each rollout's outcome-level advantage to all policy-generated tokens, creating a trajectory-to-operation mismatch since KG operations within a rollout can have varying effects on the search process and final answer.
- To address this, we propose Outcome-Anchored Credit Reallocation (OACR), which employs a training-only KG-state outcome model to estimate outcome-reward changes between adjacent KG search states and converts them into operation credit, from which token-wise weights are derived.
- These multiplicative weights reallocate token-level update magnitudes while preserving the sign of the outcome-level advantage at every generated token.
- Across three benchmarks, OACR achieves stronger overall performance than existing KGQA methods while using fewer inference-time LLM calls and tokens.
Sources (1)
- [1]Outcome-Anchored Credit Reallocation for Agentic Knowledge Graph Question AnsweringarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 18, 04:36 PM
Agentic knowledge graph question answering (KGQA) requires large language models (LLMs) to solve complex questions through multi-turn interactions with a knowledge graph (KG).
Outcome-supervised reinforcement learning derives verifiable outcome rewards from final answers but uniformly assigns each rollout's outcome-level advantage to all policy-generated tokens, creating a trajectory-to-operation mismatch since KG operations within a rollout can have varying effects on the search process and final answer.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 18, 2026TIRE: Top-Down Formula Instantiation with Reliable Execution for Physics Problem Synthesis
- Oct 17, 2026MisleadQA: Diagnosing LLM Failures under Misleading Context
- Oct 17, 2026Refuse the Harm, Complete the Task via Graph-of-Rules Planning for LLM-Enabled Robots
- Oct 16, 2026Perceive, Solve, Reason: Progressive Capability Formation in Time Series-LLMs
- Oct 16, 2026Uncertainty-Aware Budget Allocation for Efficient LLM Tree Search
- Oct 16, 2026Language Models Learn to Improve Language Models