LCT: Losslessly Compressed Tensors with GPU friendly execution for Training

Key points

• Lossless tensor compression could reduce memory usage during training, but training requires tensors to be repeatedly re-encoded.1
• LCT exploits the natural redundancy in floating point data to reduce storage for weights, activations and optimizer states.1
• Highly optimized GPU kernels implement Huffman coding with precomputed codebooks for low compression and decompression overhead, essential for training.1
• In Qwen3-4B fine-tuning, LCT saves up to $23.8%$ of peak memory, enabling a $3.5\times$ larger context length in training on the same GPU.1

Sources (1)

• [1]LCT: Losslessly Compressed Tensors with GPU friendly execution for Training
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 15, 04:34 PM

  We introduce Losslessly Compressed Tensors (LCT), a GPU execution framework designed for this repeated encode-decode regime.

  Lossless tensor compression could reduce memory usage during training, but training requires tensors to be repeatedly re-encoded.

Extractive summary: sentences quoted from the sources.