Self-improving data synthesis can scale embedding models across domains and tasks
Here we introduce SYNTRA, a novel retrieval data synthesis method whose self-improvement loop iteratively refines reusable synthesis instructions using feedback on trial outputs.
ProofPaper ↗
Key points
- Adapting embedding models to new tasks requires training data that reflect task-specific information needs and relevance criteria.
- Experiments across 24 retrieval datasets spanning 15 domains demonstrate the effectiveness of the synthesized data.
- Self-improvement enhances retrieval on all six evaluated tasks, outperforming human calibration on four.
- Overall, with minimal human intervention, SYNTRA enables scalable adaptation of embedding models across domains and tasks.
Sources (1)
- [1]Self-improving data synthesis can scale embedding models across domains and tasksarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 10, 08:50 AM
Here we introduce SYNTRA, a novel retrieval data synthesis method whose self-improvement loop iteratively refines reusable synthesis instructions using feedback on trial outputs.
Adapting embedding models to new tasks requires training data that reflect task-specific information needs and relevance criteria.
Extractive summary: sentences quoted from the sources.