ResearchResearch paperLarge Language Models1 source · Oct 17, 2026

First-Order LoRA Training with Phone NPUs

On-device fine-tuning lets a large language model (LLM) learn from its user's private data without uploading it.

Low-rank adaptation (LoRA) reduces the trainable parameters, but each first-order training step still multiplies by the frozen base-model weights during the forward pass, activation recomputation, and input-gradient computation.
A phone's neural processing unit (NPU) already holds these weights for inference; using it for training also requires computing input gradients, which its inference interface does not support.
We present a system that runs all three frozen-weight products on the NPU while keeping the training graph, adapters, and optimizer on the CPU.
Its input-gradient kernels reuse the resident weight tiles and the forward dequantizer, avoiding a transposed weight copy and ensuring forward and backward use the same effective weights.

[1]First-Order LoRA Training with Phone NPUs
arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 06:49 PM
On-device fine-tuning lets a large language model (LLM) learn from its user's private data without uploading it.
Low-rank adaptation (LoRA) reduces the trainable parameters, but each first-order training step still multiplies by the frozen base-model weights during the forward pass, activation recomputation, and input-gradient computation.

Extractive summary: sentences quoted from the sources.

Before this