Africa-Papers: A 10-Billion-Token Multi-Domain Academic Corpus for African Foundation Language Models
To resolve this representation deficit and advance African intellectual sovereignty in artificial intelligence, we introduce Africa-Papers, the largest curated multi-domain Pan-African academic pre-training corpus to date.
ProofPaper ↗
Key points
- Large Language Models (LLMs) depend fundamentally on the diversity, depth, and factual rigor of their pre-training corpora.
- However, contemporary foundational pre-training datasets (such as The Pile, S2ORC, and Dolma) exhibit a severe structural omission: African scientific, medical, legal, and agricultural scholarship constitutes less than 0.5% of global pre-training tokens.
- Comprising over 844,807 full-text papers spanning 10.15 billion tokens across 879 Parquet shards, Africa-Papers unifies four primary scientific streams: (1) continental peer-reviewed journals indexed in DOAJ across 31 African nations, (2) the CGIAR CGSpace agricultural repository, (3) African-authored and African-focused STEM/AI preprints on arXiv, and (4) open-access international publications by scholars affiliated with African institutions indexed via OpenAlex.
- We release Africa-Papers openly on the Hugging Face Hub under the AfriSpeech organization to accelerate continual pre-training and domain evaluation for foundation models globally.
Sources (1)
- [1]Africa-Papers: A 10-Billion-Token Multi-Domain Academic Corpus for African Foundation Language ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 01:14 PM
To resolve this representation deficit and advance African intellectual sovereignty in artificial intelligence, we introduce Africa-Papers, the largest curated multi-domain Pan-African academic pre-training corpus to date.
Large Language Models (LLMs) depend fundamentally on the diversity, depth, and factual rigor of their pre-training corpora.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026CollabSciReM: Aligning Language Models for Scientific Conversations via Science-Aware Reward Decomposition
- Oct 19, 2026Introducing Clairic, a new System One class of extraction models
- Oct 19, 2026Reasoning-Aware Multicalibrated Reliability Predictors for Language Models
- Oct 18, 2026Towards Figurative Language Understanding in Moroccan Darija: An Evaluation of Large Language Models
- Oct 18, 2026Local Evidence, Global Failure: Diagnosing Speech-Vision Temporal Composition in Audio-Visual Language Models
- Oct 18, 2026Aligning Language Models with Indigenous Community Cosmovision through Constitutional AI