Context Length Has Been Frozen at 1M for 30 Months

Here is a number from the arcade dataset that surprised me when I went looking for it.

Gemini 1.5 introduced a 1-million-token context window in February 2024. Today, eleven models in the dataset advertise a 1M context: Claude Opus 5, Inkling, Gemini 3.6 Flash, Gemini 3.5 Flash-Lite, GLM-5.2, DeepSeek v4 and V4-Flash, MiniMax-M3, Laguna S 2.1, and more. Every major lab, three model generations, thirty months.

Not one has ever exceeded it. Not by a little. The maximum context in a dataset of 241 releases has been exactly 1.0M since early 2024, while parameter counts went from 176B to 2.8T and prices fell by two orders of magnitude. Everything else in this industry moved. That number sat still.

So when Pokee-Isaac 28B launched today claiming 10 million tokens, the interesting question isn't whether it's impressive. It's whether the wall was ever technical.

Why 1M held

The plateau isn't a coincidence, and it isn't a conspiracy — it's where the arithmetic stops working.

A standard decoder-only transformer runs softmax attention at every layer, and every layer keeps a KV cache proportional to sequence length. Double the context, double the cache, at every layer, for every concurrent request. Somewhere past a million tokens the memory required to hold one conversation exceeds what you can profitably dedicate to it, and the economics of serving collapse before the architecture does. 1M is roughly where "we could" and "we can afford to" diverge.

Which is why the more useful frame is: the labs advertising 1M weren't hitting a capability ceiling. They were hitting a serving-cost ceiling — and, as the LFM2.5 post noted yesterday, the way past a KV-cache ceiling is not to scale but to stop paying per-layer KV cache in the first place.

Pokee describes Isaac's architecture as proprietary and non-decoder-only — and that is, frustratingly, the entire disclosure. The technical report does not specify the architecture, the attention mechanism, or the memory-management scheme. A technical report that withholds the mechanism is a marketing document with citations, and it matters here more than usual, because the 10M claim rests entirely on the undisclosed part. Everything else — the price, the single-GPU deployment, the RULER score — is downstream of whatever they aren't saying.

For what it's worth, "non-decoder-only" plus 10M context plus single-GPU deployment points hard in the same direction as LFM2's hybrid convolution design: most layers holding fixed-size state rather than a cache that grows with the input. That's inference on my part, not something Pokee has confirmed. Two releases in two days, both arguing that the deployment envelope is set by architecture rather than parameter count.

The provenance nobody led with

One detail deserves more attention than it got: part of Isaac is fine-tuned from Qwen3.6-27B, under Apache 2.0, with the remaining weights trained from scratch by Pokee. The company is careful to say it isn't a conventional Qwen fine-tune, which is fair — you don't get from a 1M-context decoder-only model to a 10M-context non-decoder-only one by fine-tuning.

But it lands squarely in a pattern this blog has now hit twice in three days. Sakana's Namazu built Japan's sovereign-AI offering on DeepSeek and Kimi K2.6. Pokee, a US startup, built its frontier claim partly on Alibaba's weights. Chinese open models aren't just competing on price any more — they're becoming the substrate other companies build products on, in Tokyo and in San Francisco. That is a form of influence no API business can purchase, and it costs Alibaba nothing per Pokee query.

It also puts a useful floor under the skepticism. Whatever Isaac's 10M mechanism turns out to be, it's built on a base whose capabilities are independently known and documented — which is more grounding than a from-scratch claim from a startup would carry.

The claims, and the one that counts

The number that matters is RULER, and the choice of it is the most credible thing about this launch.

Advertised context length is the most inflated statistic in AI. A model can accept a million tokens and stop meaningfully using anything past 128K — and needle-in-a-haystack tests, the ones labs usually publish, are easy to pass while degrading badly on anything requiring actual multi-hop reasoning across the window. RULER exists specifically to measure effective context against nominal context. It is the benchmark most likely to embarrass whoever runs it.

Pokee ran it at full 10M and published 93.3%, and claims competitors fail beyond 2M. Choosing the test designed to catch you is weak evidence of good faith — not proof, but the opposite of what a company inflating a context number would do.

The price is the second story

$0.15 input / $1.00 output per million tokens undercuts everything in the pricing table this blog built last week — including GPT-5.6 Luna at $0.20/$1.20, which had just been cut 80% and was the cheapest frontier-lab offering on the board. Claude Fable 5 is $10/$50.

A 28B model priced below the cheapest thing OpenAI sells, claiming ten times the context of anything Anthropic or Google ship, deployable on a gaming GPU. If even half of that survives contact with independent testing, it's a sharper version of the commoditization argument than any Chinese open-weight release has made — because this one isn't undercutting on price despite a capability gap. It claims a capability lead.

Where the skepticism belongs

Every number above is Pokee's own, on Pokee's own evaluation — the standard caveat this blog applies to Alibaba, SK Telecom, Mistral and everyone else. Two specifics deserve flagging:

"Frontier-class" is doing enormous work for a 28B model. The agentic leads on BFCL v4 and τ³-bench are explicitly "in Pokee's evaluation." Nothing here has been reproduced by Artificial Analysis, LMArena, or any third party.

It's closed — from a company whose last model wasn't. Pokee open-sourced PokeeResearch-7B in October 2025, weights on Hugging Face and code on GitHub. Isaac is closed source, described as "for now." That's a reversal in the direction most labs travel, and it means the 10M claim can be tested through the API but not audited in the weights.

What makes this more checkable than Qwen3.8-Max's benchmark-free preview, though, is that Isaac is live, at published prices, on an OpenAI-compatible endpoint. Anyone with $20 and a long document can start falsifying the RULER claim this week. A bold claim you can immediately test is a categorically better artifact than a bold claim you can't.

What to expect next

• Independent RULER runs at 10M are the whole story. Not BFCL, not τ³-bench, not the price. If a third party reproduces 93.3% at full context, this is the most significant architectural result of the summer; if effective context collapses at 2M, it's the most expensive marketing number of it.
• Expect the 1M plateau to break generally, not just here. A wall that held for thirty months held for economic reasons, and the moment one vendor demonstrates a serving cost structure that goes past it, "1M" stops reading as a spec and starts reading as a limitation.
• Watch whether Isaac opens. Pokee has open-sourced before. A 28B model at this claimed capability, released openly, would land directly in the barbell's local tier — and a 10M-context model that runs on a 4090 is a genuinely different proposition from a 2.8T model that needs eighteen H100s.

References: Pokee Console — Pokee-Isaac 28B · Pokee AI · PokeeAI/pokee_research_7b on Hugging Face · Pokee-AI/PokeeResearchOSS on GitHub · MarkTechPost — PokeeResearch-7B · related coverage: LFM2.5-2.6B · The Price War Nobody Is Actually Fighting · Open Weights You Can't Actually Run · the arcade dataset