NVIDIA and AI4Bharat at IIT Madras say they plan to release Nemotron-pretraining-Indic-v1, a corpus containing approximately 2.8 trillion tokens across 22 Indian languages, together with an open, GPU-accelerated pipeline for producing the data. The resource is intended for teams training language models in Indic languages, where the announcement says a shortage of available data remains a constraint.

The reported inventory comprises 2,601.0 billion native-script tokens and 224.6 billion romanized tokens, counted with the Gemma-4 tokenizer. Native-script material covers all 22 languages; romanized text, meaning Indic-language content written in Latin letters, is included for 12. Together, those formats make 34 language-script configurations. The announcement cautions that records can include translations and script variants of shared source documents, so the total does not represent unique English documents.

The pipeline starts with English documents drawn from 15 Nemotron-CC-v2 subsets, including web text, question-and-answer material, mathematics and code. Using NVIDIA’s NeMo Data Designer framework, the workflow translates documents, assesses translation quality, repairs low-scoring outputs and converts a subset of accepted translations into Latin script. The announcement says each record retains a source identifier and outputs are checked against their inputs; checks are also designed to preserve elements such as equations, tables and code.

The release is described as planned, not already available. The teams say developers will be able to use the corpus directly or adapt the prompts, workflows and quality checks to their own source material. The source material does not give a release date or a complete account of the corpus’s filtering and acceptance thresholds.

A Larger Starting Point for Indic Models

A large, openly available corpus could give model developers more material for training across languages that have had fewer resources than English. The announced source mix includes technical subjects such as mathematics and programming, alongside question-and-answer content. That could support models intended to explain technical material or respond to questions in Indic languages, though the announcement does not provide independent evaluations showing how much the corpus improves model performance.

Making the generation workflow available adds a second potential benefit: teams could inspect or adapt how the synthetic data was created, rather than receiving text alone. Source identifiers and input-output checks may help with traceability, while the ability to bring other source documents into the workflow could make the process reusable. These are capabilities described by the project; their practical value will depend on the documentation, code, licenses and quality results in the eventual release.

The corpus is also not presented as a replacement for naturally authored Indic text. Its translations begin with English material, so they can carry over the source’s subject matter and framing without supplying the full range of local knowledge and cultural context found in original writing. The project itself says naturally authored Indic data remains essential.

From Sangraha to Synthetic Data

AI4Bharat is an Indian-language research lab at IIT Madras that develops open datasets and models. The announcement points to Sangraha, its open pretraining corpus for 22 Indian languages, as part of the foundation for open Indic-language model research. The new project adds a large body of translated material and a described workflow for generating more from curated English sources.

The announcement says the 22 languages are those listed in India’s Constitution and that they are the first languages of 97% of Indians, or nearly 1.2 billion people according to the 2011 Census. That population figure is tied to the cited census, not a current estimate. The project also reports that continued pretraining for Nemotron 3.5 Super VL used 10 billion Indic tokens from a 12-language subset of this corpus along with naturally authored Indic text. That training use is distinct from the corpus inventory: token counts depend on the tokenizer, and the retained corpus total is not a training run’s token budget.

“We do not have enough data for Indian languages.”

— Mitesh Khapra, head of AI4Bharat

Release Details Still Pending

The announcement does not specify when the corpus or pipeline will be available, where they will be hosted, or the precise terms under which users may redistribute or adapt them. It also does not provide detailed quality scores, acceptance thresholds, per-language token breakdowns or independent assessments of translation fidelity. Those details will matter to developers deciding how to use the data and to researchers evaluating its limitations.

The reported token totals are corpus counts under the Gemma-4 tokenizer and include accepted rewrites and script variants. They should not be read as counts of unique source documents or as evidence that a model has trained on the full inventory. The announcement gives no performance comparison showing how the corpus affects downstream models, and the relative contribution of synthetic translations and natural Indic text in future training mixtures is not fully specified.

Release and Evaluation Milestones

The next milestone is the planned publication of the corpus and its generation pipeline. Once available, developers and researchers can review the data, inspect its provenance and quality documentation, and test whether its language and subject coverage fits their needs. The source material does not name a publication date.

The teams say they plan to build on the corpus in future Nemotron models. Further reporting will depend on the release materials and subsequent evaluations, including how the data performs across individual languages and scripts and how it complements naturally authored Indic text.

Where I land

My assessment is that the release could be useful infrastructure if developers can inspect the data and reproduce or adapt the generation process. Its reported scale and breadth are meaningful for teams that need material across many languages and technical subjects. But the announcement establishes the size and intended workflow, not the resulting model quality.

The strongest counterargument is that translated synthetic text may not capture local knowledge, idiom or cultural context, and automated checks cannot by themselves show that translations are accurate or balanced across languages. The project acknowledges the need for naturally authored Indic text. I would revise my assessment upward if the release includes clear licensing, language-level quality results and independent evaluations showing gains across scripts and tasks; I would be more cautious if those checks reveal uneven coverage or persistent translation errors.

Key Questions

What is Nemotron-pretraining-Indic-v1?

It is a planned corpus from NVIDIA and AI4Bharat at IIT Madras, reported to contain about 2.8 trillion tokens across 22 Indian languages, plus an open pipeline for generating synthetic training data.

Which languages and scripts does it cover?

The announcement says native-script data is included for all 22 languages. Latin-script romanizations are included for 12 of them, making 34 language-script configurations in total.

How was the data generated?

The workflow uses English documents from 15 Nemotron-CC-v2 subsets. It translates them, assesses output quality, repairs low-scoring translations and romanizes a subset of accepted translations. Source identifiers are retained, according to the announcement.

Is the corpus available now?

The source describes the corpus and pipeline as planned releases. It does not give a publication date or a hosting location.

Do the token totals count unique source documents?

No. The reported totals count retained corpus text using the Gemma-4 tokenizer and can include translations and script variants derived from shared sources. The announcement says they are not counts of unique English documents.

Source: Hugging Face