Robust Explanatory Analysis for Structured Optimization Networks (REASON) in Learning

Key points

• Inspired by the System 1/System 2 paradigm, we formulate this principle as a two-loop mechanism serving a common inference goal.1
• A fast, System 1-like loop resolves the residual addressable by the available evidence; the remaining epistemic residual triggers a medium speed, System 2-like loop that actively acquires additional evidence and measures its informational value through residual reduction.1
• Over nested solution spaces, the medium-loop optimum is provably no worse than the fast-loop optimum.1
• Experiments show that the resulting two-loop model consistently improves upon the fast loop at approximately half the computation of the medium loop, including under unseen corruptions, and, when provided a cleaner observation, outperforms staged classifiers and learning-to-defer methods at equal computational cost.1

Sources (2)

• [1]Robust Explanatory Analysis for Structured Optimization Networks (REASON) in Learning
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 14, 01:24 AM

  We seek a hierarchical resolution of residual error under a composite learning objective as a principled mechanism for identifying and exploiting additional information, particularly under distribution shift and perturbation.

  Inspired by the System 1/System 2 paradigm, we formulate this principle as a two-loop mechanism serving a common inference goal.

• [2]Robust Explanatory Analysis for Structured Optimization Networks (REASON) in Learning
  Hugging Face Daily Papers · Oct 14, 12:00 AM · same content

Extractive summary: sentences quoted from the sources.