Best of Both Worlds: Fine-Tuned Small Models Complement LLMs for Cost and Performance Benefits
BERT-based models require thousands of labeled samples to train.
ProofPaper ↗
Key points
- We compare these parameters of a trained RoBERTa-based model, and an (off-the-shelf) GPT-3.5 model for text classification.
- To identify the best prompting approach for GPT-3.5, we explore batch prompting.
- We find the best batch size is 20 with human few-shot curation.
- We find the best approach is an ensemble of a RoBERTa-based and GPT-3.5 model (F1: 97.5%) incurring only 4.4% of inference cost compared to a standalone GPT-3.5.
Sources (1)
- [1]Best of Both Worlds: Fine-Tuned Small Models Complement LLMs for Cost and Performance BenefitsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 16, 09:34 PM
BERT-based models require thousands of labeled samples to train.
We compare these parameters of a trained RoBERTa-based model, and an (off-the-shelf) GPT-3.5 model for text classification.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 16, 2026[Research] 2,788 tokens/s on a single RTX 5090 with a diffusion LLM — Introducing ORL (fully open source)
- Oct 16, 2026Prefix-Tuning for Arbitrary Output Sequences on Pretrained Transformers
- Oct 15, 2026openai/codex rust-v0.163.0: 0.163.0
- Oct 15, 2026Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint
- Oct 15, 2026REKI: Registering Knowledge in VLAs
- Oct 15, 2026Anthropic tells investors quarterly revenue up nearly 50% to $16.8bn ahead of IPO
