RubricOpt: Self-Revision of Semantic Rubrics for Preference Optimization

Key points

• A model's failures can reveal gaps in the behavioral criteria guiding its training.1
• A shared semantic representation connects task demands, preference-data contrasts, and rubric clauses, while a deterministic gate checks evidence references and fixed semantic anchors.1
• Our analysis relates the local effect on rubric-free success to semantic alignment and a residual beyond the measured factors.1
• Experiments on capability and open-ended generation benchmarks show that rubric conditioning can improve outcomes over ordinary DPO, with gains and regressions varying across models and datasets.1

Sources (1)

• [1]RubricOpt: Self-Revision of Semantic Rubrics for Preference Optimization
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 21, 09:46 AM

  We introduce RubricOpt, a framework that lets a policy revise these criteria within preference optimization.

  A model's failures can reveal gaps in the behavioral criteria guiding its training.

Extractive summary: sentences quoted from the sources.