Scaling Indic Pretraining: 2.8 Trillion Tokens and an Open Data Generation Pipeline
We're excited to introduce Nemotron-pretraining-Indic-v1 approximately 2.8 trillion tokens across 22 Indian languages, releasing soon alongside the open, GPU-accelerated data generation pipeline used to create it.
Key points
- Authors: AI4Bharat — Mohammed Safi Ur Rahman Khan, Mitesh Khapra, Anoop Kunchukuttan.
- Developed by NVIDIA with IIT Madras, India's premier engineering institute, and its AI4Bharat lab, this release gives model builders a substantial new resource for Indic pretraining.
- AI4Bharat, IIT Madras' Indian language research lab, builds open-source datasets and models for Indian languages.
- Our joint open-source release represents a substantial expansion of the open synthetic Indic training data that will be available to the community.
Sources (1)
- [1]Scaling Indic Pretraining: 2.8 Trillion Tokens and an Open Data Generation PipelineHugging Face Blog · Oct 16, 01:44 AM
We're excited to introduce Nemotron-pretraining-Indic-v1 approximately 2.8 trillion tokens across 22 Indian languages, releasing soon alongside the open, GPU-accelerated data generation pipeline used to create it.
Authors: AI4Bharat — Mohammed Safi Ur Rahman Khan, Mitesh Khapra, Anoop Kunchukuttan.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 16, 2026One Communicator, Three Vendors: Extending the MoA Predictive Theory to a Real NVIDIA+AMD+CPU MPI Ensemble
- Oct 16, 2026Welcome Beam: a 501B open model you can own, customize, and scale
- Oct 15, 2026Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint
- Oct 15, 2026unslothai/unsloth v0.1.908-beta: NVIDIA Nemotron 3.5 + Meta Muse Spark Open
- Oct 15, 2026NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning
- Oct 14, 2026mistralai/Timestral-209M
