Generative Models as Adaptive Trust Regions for Discrete Bayesian Optimisation

Key points

• Local trust-region BO methods such as TuRBO address high dimensionality by restricting search to adaptively sized neighbourhoods.1
• However, a direct discrete analogue based on edit distance can be poorly suited to structured discrete objects: neighbourhoods grow combinatorially, and metric proximity does not capture the dependencies that determine whether a design is plausible or useful.1
• Motivated by the growing availability of pretrained generative models, we introduce GenTuRBO, which replaces the geometric trust region with an adaptively scaled proposal distribution defined by a pretrained generator.1
• The generator provides a learned prior over discrete transformations, encoding coordinated changes without explicitly enumerating edit-distance neighbourhoods.1

Sources (1)

• [1]Generative Models as Adaptive Trust Regions for Discrete Bayesian Optimisation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 03:44 PM

  Bayesian optimisation (BO) is a powerful framework for sample-efficient optimisation of expensive black-box functions, but its application to high-dimensional discrete scientific design remains challenging.

  Local trust-region BO methods such as TuRBO address high dimensionality by restricting search to adaptively sized neighbourhoods.

Extractive summary: sentences quoted from the sources.