SliMOO: Interpretable Multi-Objective Evolutionary Search for LLM Depth Pruning
We propose SliMOO, an interpretable multi-objective evolutionary search framework for LLM depth pruning.
ProofPaper ↗
Key points
- As large language models (LLMs) grow more powerful, their scale makes them costly to serve.
- Depth pruning offers a practical route to efficiency by directly reducing memory and latency without hardware-specific support.
- However, we observe that transformer layers exhibit markedly different task-dependent layer contributions, whereas existing depth-pruning methods ignore this heterogeneity and thus tend to optimize for a narrow domain or benchmark rather than preserve broad capability.
- Across the Llama-3.1 and Qwen3 model families, SliMOO consistently achieves stronger multi-task trade-offs than rule-based, greedy, and scalarized baselines.
Sources (1)
- [1]SliMOO: Interpretable Multi-Objective Evolutionary Search for LLM Depth PruningarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 08:05 AM
We propose SliMOO, an interpretable multi-objective evolutionary search framework for LLM depth pruning.
As large language models (LLMs) grow more powerful, their scale makes them costly to serve.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026Do Diffusion LLMs Scale? Testing Mercury 3 + Decide
- Oct 19, 2026Introducing Clairic, a new System One class of extraction models
- Oct 19, 2026MoveBench: A Rubric-Based Benchmark for Evaluating Large Language Models in Sports and Health Coaching
- Oct 19, 2026PFedQFT: Personalized Federated Quantum-Inspired Fine-Tuning for Medical Large Language Models
- Oct 19, 2026TokenGate: Adapting LLMs for Fast and Memory-Efficient Prefill with Learned Token Pruning
- Oct 19, 2026TorchNeuron: running HF models on AWS Trainium is easier than ever