Collaborative Learning and Reasoning over Model Slices
To address this limitation, we develop collaborative learning, encompassing the construction of heterogeneous collaborators and the learning of effective cross-model collaboration.
ProofPaper ↗
Key points
- Large language models (LLMs) excel across diverse tasks, benefiting from knowledge priors, in-context reasoning, and test-time computation.
- Pyramid supervision combines protocol distillation with step-level supervision to refine reasoning and interaction decisions throughout collaboration.
- We evaluate our approach across four task categories: multi-hop question answering, mathematical reasoning, code generation, and open-ended constrained generation.
- These findings show that learned collaboration turns model heterogeneity into complementary reasoning capabilities.
Sources (1)
- [1]Collaborative Learning and Reasoning over Model SlicesarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 03:17 AM
To address this limitation, we develop collaborative learning, encompassing the construction of heterogeneous collaborators and the learning of effective cross-model collaboration.
Large language models (LLMs) excel across diverse tasks, benefiting from knowledge priors, in-context reasoning, and test-time computation.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 9, 2026PreSTO: Predictive Subtree Prefetching for Fast LLM Power Sampling
- Oct 9, 2026Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation
- Oct 8, 2026Understanding Jev, the new model everyone is talking about
- Oct 8, 2026One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts
- Oct 8, 2026SpatialOPSD: Self-Distilling Spatial Intelligence from Verified Coding Agent Traces
- Oct 8, 2026Distilling Routed 3D Privilege for Spatial Reasoning in Vision-Language Models