Researchers at McGill and Mila, LMU Munich, and Microsoft's AI for Good Research Lab published AfriqueLLM at ACL 2026 (paper in the proceedings, models on Hugging Face): a suite of open models adapted to 20 African languages through continued pre-training on 26B tokens, across five base models spanning Llama 3.1 8B, Gemma 3 (4B, 12B), and Qwen 3 (4B, 8B, 14B). The point of running the same recipe across that many backbones is that it turns a model release into an actual experiment — you get to see which of the two things people assume matters (scale, or prior multilingual coverage) actually predicts how well adaptation works.
Neither does, cleanly. That's the interesting part.
The base model with the worst African support produced the best African model
Qwen 3 has minimal official African-language support and starts from the weakest baselines in the paper: Qwen 3 8B averages 33.47 across seven AfroBench-Lite tasks, below Llama 3.1 8B (35.33) and below the much smaller Gemma 3 4B (40.31). Gemma 3, by contrast, is built around broad multilingual pre-training and starts far ahead — Gemma 3 12B at 54.80.
After identical continued pre-training, that ordering reverses. AfriqueQwen-8B reaches 59.08, edging out AfriqueGemma-12B at 58.82 with two-thirds the parameters, and AfriqueQwen-14B reaches 63.58 — the best model in the paper. The authors call it the "Zero-to-Hero" effect and report relative gains of 74.4%, 76.5%, and 57.8% for the three Qwen sizes against 18.8% and 7.3% for the two Gemma sizes.
Those percentages are worth deflating a little: a relative gain computed against a near-floor baseline is flattering by construction, and Qwen 3 8B's Flores translation score of 21.13 is close to the floor. But the deflation doesn't rescue the intuition, because the absolute post-CPT numbers say the same thing in a way percentages can't inflate. The Qwen-based models end up ahead. The paper's framing — a strong base model matters more than a multilingual one, because capability transfers to new languages more readily than coverage confers capability — survives the arithmetic. That's a genuinely useful result for anyone choosing a starting checkpoint, and it's the opposite of what the obvious heuristic would tell you to do.
Section 5.5 sharpens it further with an unusually clean natural experiment: Qwen 3 4B and Qwen 3.5 4B share an architecture but differ in multilingual scope. The more multilingual base starts much higher (46.01 vs 31.49) and ends higher (57.12 vs 54.94) — but with a smaller relative gain, better translation, and worse math. Multilingual priors buy language-knowledge tasks, not reasoning. Adding more code and math data recovers the math.
The Gemma 3 27B comparison wins two tasks and loses four
The paper's most quotable claim is that AfriqueQwen-14B (63.58) beats Gemma 3 27B (60.71) at less than half the size. The average is right, and the paper names its drivers. It's worth reading the row anyway, because the shape matters.
AfriqueQwen-14B wins on translation by 15.36 points (Flores, 63.77 vs 48.41) and math by 9.64 (AfriMGSM, 45.01 vs 35.37), plus 2.16 on natural language inference. It loses on AfriMMLU knowledge (52.22 vs 55.47), intent classification (77.80 vs 79.70), topic classification (82.63 vs 84.34), and ties-to-slightly-loses on reading comprehension (74.63 vs 74.81). Three wins, four losses, and a +2.87 average carried by the two tasks where 26B tokens of targeted African text and a billion tokens of math should obviously help.
That isn't a debunking — beating a model twice your size on translation and math in these languages is the actual product here, and nobody adapting a model for Swahili translation cares that the unadapted 27B is two points better at topic classification. But "outperforms Gemma 3 27B" reads as a general claim and isn't one.
The comparison the paper handles more scrupulously is against Lugha-Llama-8B-wura, the prior CPT model on the same Llama 3.1 8B base trained on monolingual WURA data only. AfriqueLlama-8B wins 4 of 7 tasks, hugely on math (17.51 vs 9.46) and translation (63.60 vs 49.90) — and loses on topic classification by 6.7 points. The paper states the 4-of-7 split plainly rather than leading with the average.
The recipe's evidence is thinner than the recipe's confidence
The adopted mixture, CMS — monolingual text plus ~1B tokens of Python from CornStack, ~1B tokens of FineMath-4+, and 324M tokens of GPT-4.1-translated web and math content across 11 domains — is chosen in a Gemma 3 ablation, then applied unchanged to five other backbones. Look at what the choice rests on.
At 12B, CMS beats CMSP (the same mixture plus filtered NLLB parallel data) by 0.40 on Flores, 1.26 on AfriMGSM, 0.14 on AfriMMLU, and 1.31 on AfriXNLI. Single runs, no seeds, no variance reported. At 4B the evidence points the other way: CMSP wins Flores (63.34 vs 63.17), AfriMMLU (36.72 vs 35.86), and AfriXNLI (40.44 vs 39.93), with CMS winning only AfriMGSM. And the best AfriXNLI score at 4B belongs to monolingual-only data (40.76), beating every mixture. So the recipe adopted for all six models wins one of four comparisons at the smaller scale and four of four at the larger one, by margins of roughly a point.
The paper's explanation — larger models are more sensitive to data quality, so filtered-but-noisy parallel corpora help small models and hurt big ones — is plausible and matches published mid-training lore. It just isn't established by these numbers, and a hypothesis is what the paper honestly calls it.
Then there's a measurement problem the paper flags itself, in a footnote: the ablation table uses the HuggingFace backend while every other benchmark uses vLLM, so "absolute scores may differ." They differ a lot. Gemma 3 12B's base Flores score is 52.53 in the ablation table and 44.09 in the main table; AfriXNLI is 39.81 and 44.01. The language sets differ too — the ablation covers only CPT languages, the main table all languages per dataset — so the two effects are confounded and neither can be isolated. Either way, gaps of 4 to 8 points separate the two setups for the same model on the same task, while the mixture decision inside one of them turns on gaps of 0.1 to 1.3. The winning recipe is never re-validated under the setup that produces the headline results.
The finding most likely to survive replication is the cheapest one
Buried in the ablation is the result I'd actually bet on generalizing. Training Gemma 3 12B on monolingual African text alone — 22B tokens of it — improves translation and NLI substantially but degrades the reasoning benchmarks: AfriMGSM 24.10 → 23.78, AfriMMLU 48.21 → 46.72. Adding just 2B tokens of code and math reverses it, at every mixture containing them and at both model sizes.
That's a small, concrete, cheap intervention with a consistent sign across two scales and five mixtures, and it speaks to a failure mode anyone doing language adaptation will hit: pour in a large volume of monolingual low-resource web text and you trade away reasoning you already had. Two billion tokens of structured data — about 8% of the corpus — buys it back. The paper's "cognitive anchor" framing for why is speculation, but the effect doesn't depend on the explanation being right.
English and French pay for it, more than the abstract says
The abstract says AfriqueLLM achieves strong results "while largely preserving English performance." Table 4 doesn't support that phrasing. English drops for four of five models — Llama 3.1 8B by 11.0% relative, Gemma 3 12B by 7.4%, Gemma 3 4B by 4.2%, Qwen 3 14B by 4.1% — and French drops for all five, including Qwen 3 8B, the single model that holds English parity (+0.8% English, -4.0% French). The average high-resource decline runs from -1.6% (Qwen 8B) to -10.2% (Llama 8B).
The honest version of the claim is the one the results section gives: Qwen's series forgets least, and there is a real trade you are making. For a model that will be deployed in African-language contexts, losing 10% of English is likely a fine price. It shouldn't be described as not paying it.
The other trade is bluntly reported and deserves credit for being so: transfer to languages outside the CPT mixture barely happens. On African languages absent from both base and CPT training, scores move between -4.8 and +4.4 points, mostly noise. The AfriqueQwen models show small consistent positive transfer (+4.4 and +2.7), which the authors attribute to Bantu-family proximity to Swahili and Kinyarwanda. Twenty languages in the mixture means roughly twenty languages improved, and the limitations section says so — hundreds remain unsupported.
The number practitioners should take away is the compute bill
Appendix B.3 is the part of this paper with the most immediate practical value, and it's three lines in a table. AfriqueQwen-14B — the model that beats Gemma 3 27B — took 64 H100s for 31.1 hours: about 1,990 GPU-hours. AfriqueGemma-4B took 146. The five reported runs total roughly 6,000 H100-hours for the entire suite.
That is a startlingly small bill for a result at this quality level, and it reframes what the paper is. It isn't a compute story or a scale story; it's evidence that competitive language coverage for twenty underserved languages is now a few thousand GPU-hours and a well-chosen data mixture away, on hardware a single university cluster or a modest cloud budget can reach. The authors also report Model FLOPs Utilization of only 26–34% and note intermittent gradient-norm spikes they didn't chase down — so the bill isn't even optimized.
Two smaller things the tables reveal that the prose smooths over. The learning-rate sweep picks 5e-5 as the rate that "optimally balances the retention of prior knowledge with the acquisition of new linguistic features," but in that same table AfriMMLU — the knowledge benchmark — improves monotonically as the rate drops, reaching 37.2 at 1e-5 against 34.7 at the chosen 5e-5. The choice maximizes translation and math and gives up about 2.5 points of knowledge; it's a translation-weighted pick, not a balanced one. And UniMax sampling is described as capping each language near 1.07B tokens, but the six lowest-resource languages can't reach that cap even at four epochs — Tswana and Oromo top out around 370M, a third of the budget the top languages get. Oromo still posts some of the largest gains in the paper, which is either encouraging about how little data it takes or a sign the benchmark has a low ceiling there.
One claim in the conclusion doesn't match any table
The conclusion states that AfriqueQwen-14B surpasses "the 54B NLLB-MoE on translation." The only NLLB-MoE-54B number in the paper is 65.72 SSA-COMET, in the ablation table, on CPT-covered languages. AfriqueQwen-14B's Flores score is 63.77 — in the main table, over a larger language set. The two aren't comparable, and within the ablation table the model that actually clears 65.72 is AfriqueGemma-12B at 66.23, which isn't the model the sentence credits. The underlying point is probably fine — adapted general LLMs matching a dedicated 54B translation MoE on these languages is a real result, and the document-level translation numbers in Table 6 make a stronger version of it, where three Afrique models beat a task-specific fine-tuned baseline on data they never trained on. But as written, the claim rests on comparing two numbers the paper deliberately kept in separate tables.
The strategic frame is worth naming, because this blog has now seen the same bet placed from three continents. AfriqueLLM's answer to "how does an underserved language get a good model" is the same as Sakana's answer for Japanese: don't train one, adapt someone else's open weights. The contrast with NII's from-scratch LLM-jp-4-33b and its 11.7 trillion tokens is the whole argument in two numbers — 11.7T versus 26B, roughly a 450× difference in tokens for a model that is competitive in the languages it targets.
There's an under-discussed bonus in how that bet was placed here. Adapted weights inherit their base model's licence, so the paper's best model is also its most freely usable — Qwen 3's Apache-2.0 travels to AfriqueQwen, where AfriqueGemma and AfriqueLlama carry the Gemma Terms and the Llama Community Licence with them. For anyone who has had to work out where a model sits on the openness ladder before building on it, the recipe that wins on benchmarks happening to also be the one that wins on licence terms is a rare alignment of incentives.
The obvious weakness of an adaptation strategy is that it rides a treadmill: this paper's newest base model is Qwen 3.5, and Qwen has shipped several generations since. But that cuts in the strategy's favour more than against it. A from-scratch 11.7T-token run can't be redone when a better architecture lands. A 2,000-GPU-hour recipe can be rerun on next quarter's checkpoint — and the durable contribution here isn't the six models, it's the finding that you should point that rerun at whichever base model is strongest, not whichever one claims your languages.