ResearchResearch paperLarge Language Models · Retrieval, RAG & Search · Interpretability1 source · Oct 7, 2026

Lossy Compressive Text Autoencoders

Our work explores learning a compressed latent representation of text, at the intersection of data compression and representation learning.

We propose an autoencoder architecture that performs residual downscaling and upscaling of hidden representations along the time axis, with a residual low-dimension discrete bottleneck.
For different levels of compression, we evaluate the similarity between the original and reconstructed text both at the surface-level (BLEU) and at the semantic-level (LLM-based judge).
Additionally, we evaluate our models on downstream question-answering and semantic text similarity benchmarks.
Our approach results in compressed representations which are on par with lossless text compression algorithms at 2.24 bits per byte on web text data, while having good reconstruction and downstream task performance.

[1]Lossy Compressive Text Autoencoders
arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 06:08 PM
Our work explores learning a compressed latent representation of text, at the intersection of data compression and representation learning.
We propose an autoencoder architecture that performs residual downscaling and upscaling of hidden representations along the time axis, with a residual low-dimension discrete bottleneck.

Extractive summary: sentences quoted from the sources.

Before this