Mirendil introduced Fern, its first model: "pretrained from scratch" with a context window of up to 10M tokens. Fern 0.1, the first version, was finished with a short round of post-training, and "new versions ship frequently." The model card lists a base model called Mirendil-Base, 0.1 to 0.5T parameters, text and code input, and access "coming soon." Mirendil calls it "our most efficient model, built for focused, high-volume agentic tasks." Reports describe Mirendil as founded by former Anthropic researchers, which raised a $200 million seed round at about a $1 billion valuation from Andreessen Horowitz and Kleiner Perkins, with NVIDIA participating, and is reportedly in talks at about $5 billion.
Why 10M tokens
Mirendil's argument is "context is intelligence": a model's capabilities grow with compute, and compute can go into parameters or into context. More parameters hold more static knowledge and support more sophisticated reasoning steps, while more context lets a model use more task-specific information: its own past experience, more specifications, more documents and code, more data looked at directly. Fern's architecture, training and serving can support up to 10M tokens at once, about 23 MB of code or roughly half a million lines, a codebase on the scale of the Megatron-LM library, or dozens to hundreds of complete agent traces, or days of an agent's work.
Prediction keeps improving with context
Cross-entropy in bits per byte on Mirendil's repositories, from Mirendil's announcement. Lower is better; dashed lines are post-trained models.
The first test is plain next-token prediction. Mirendil concatenated its repositories into a single 42.9 MB stream and read it with overlapping sliding windows, measuring context in bytes so models with different tokenizers compare fairly. Fern's loss falls from about 0.20 bits per byte at roughly 64K bytes of context to about 0.155 at 10.2M tokens, still flattening gently rather than rising. The other models were evaluated up to about 1M tokens, their full advertised window or beyond, while Fern continues to a 23 MiB window. Against the base models, Fern's curve sits below DeepSeek-V4-Pro-Base (about 0.163 at the longest window) and Nemotron 3 Ultra Base (about 0.172). It sits above Kimi-K3, which scores lowest on the chart, about 0.14, though Kimi-K3 is post-trained, as are GLM-5.3, Beam and Inkling, which Mirendil says generally score slightly worse on bits per byte than their base versions.
Finding information: RULER
Accuracy (%) of Mirendil-Base on five RULER tasks, from Mirendil's announcement. Each cell averages 100 examples (20 at 10M), with greedy decoding, thinking off and exact string matching.
As a basic test of access to the long context, Mirendil scores the base model on RULER. Single-needle retrieval is 100% at every length up to 10M tokens, multi-key retrieval is 90% at 10M (98% to 100% from 512K to 4M) and retrieving all needles stays at 95% or better. Variable tracking dips to 83% at 2M and 75% at 4M but reads 100% at 10M, the cell with only 20 examples. Multi-hop question answering (HotpotQA) is the weak spot: 60% to 65% through 4M and 45% at 10M, down from 65% at 32K. The five-task average moves from 93.0 at 32K to 87.0 at 10M, driven by that last row. Mirendil says more comprehensive long-context results are coming.
What Fern is for
Mirendil pitches Fern at the focused agentic work of people building AI: working across larger codebases, agentic coding that runs commands and keeps going until the tests pass, research code such as training loops and data pipelines, debugging a training run from long logs and stack traces, reading entire agent or training traces end to end, and auditing a whole pretraining shard for duplicates, benchmark contamination and junk. For another long-context release this month, see Kimi K3.1 Pro, with a 1M-token window.