LOOP: Recovery-Aligned Iterative Layer Pruning via On-Policy Distillation

Key points

• Layer-level pruning is attractive for large language model (LLM) deployment as it reduces the model size while preserving a dense architecture that can be efficiently executed on standard hardware.1
• Current methods typically follow a one-shot paradigm, which first removes selected layers to reach a target model size and then performs additional training to recover model capabilities.1
• In each shot, LOOP removes a single layer and performs short on-policy distillation, mitigating the abrupt degradation in reasoning capabilities associated with one-shot pruning.1
• To guide layer selection in subsequent shots, LOOP reuses on-policy trajectories and optimization signals from recovery to jointly assess each remaining layer's reasoning contribution and recovery sensitivity, thereby making layer selection recovery-aware.1

Sources (1)

• [1]LOOP: Recovery-Aligned Iterative Layer Pruning via On-Policy Distillation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 19, 02:30 AM

  To address these limitations, we propose LOOP, a Layer-level Recovery-Oriented On-Policy Pruning framework that interleaves pruning and recovery across multiple shots.

  Layer-level pruning is attractive for large language model (LLM) deployment as it reduces the model size while preserving a dense architecture that can be efficiently executed on standard hardware.

Extractive summary: sentences quoted from the sources.