Debiased Prior-Data Fitted Networks for Amortized Bayesian Semi-Parametric Inference

Key points

• In this work, we develop theoretical foundations for debiased PFNs to perform inference on finite-dimensional targets in general semi-parametric settings.1
• First, we show that, despite being Bayes-optimal with respect to the target, an idealized PFN can exhibit a posterior concentration rate that implicitly depends on the concentration rates of (latent) nuisance functions and can become slower as the prior becomes more diverse.1
• Then, to speed up the target posterior concentration rates, we introduce two generic debiasing strategies based on efficient influence functions: (i) a label correction and (ii) a prior correction.1
• Finally, we show that the debiased PFNs (i) & (ii) possess concentration rates that asymptotically match a convergence rate given by traditional frequentist bias-corrected estimators.1

Sources (1)

• [1]Debiased Prior-Data Fitted Networks for Amortized Bayesian Semi-Parametric Inference
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 02:52 PM

  Prior-data fitted networks (PFNs) have recently gained popularity as an end-to-end approach to amortized Bayesian inference.

  In this work, we develop theoretical foundations for debiased PFNs to perform inference on finite-dimensional targets in general semi-parametric settings.

Extractive summary: sentences quoted from the sources.