2026-07-31

Thinking Machines' Answer to 'Should Inkling's Weights Even Be Public'

AIOpen SourcePolicy🌍 North America

Thinking Machines shipped Inkling's weights two weeks ago with a line that read as routine self-assessment: a 6.4-point drop in refusing harmful requests, described in one word as "competitive." This week the lab published the actual framework behind that judgment call β€” "A Safe Path to Open Weights" β€” and it's a more serious document than a model card footnote. It's an attempt to answer the question this blog's own open-weight-ban coverage kept circling without a clean answer: what would it actually take for releasing a model's weights to be the responsible choice, not just the available one?

The premise: open weights are a public good and irreversible at the same time

The post doesn't hedge on why openness matters β€” "the knowledge of what AI should do lives with the people doing the work, so they should be able to shape their models directly," and without strong open models, "training and deployment expertise risks becoming concentrated in a few labs." But it pairs that with the plainest possible statement of the downside: once weights are public, "anyone can use them, including bad actors," and unlike an API you can throttle or revoke, a release can't be taken back. The framework exists because both of those are true at once, not because one cancels the other.

The concrete worry it leads with is cybersecurity: Anthropic's Claude Mythos Preview reportedly found thousands of previously unknown vulnerabilities across every major operating system and browser and wrote working exploits without human guidance β€” a capability this blog covered from the incident-response side β€” and Thinking Machines' framing is direct: "Will our society be safer when open-weights models allow everyone to have superhuman cybersecurity skills?" They don't answer that rhetorically. Their answer is the rest of the post.

Two separate questions: is the model safe, and is the ecosystem ready

The framework splits into two axes that get evaluated independently. Is the model safe β€” robust testing for what harmful tasks it can complete, how accessible that capability is, and what happens once guardrails are stripped out. Is the ecosystem ready β€” layered defenses, from individual organizations patching fast to defensive tooling built on top of capable models. The honest part is admitting these pull in opposite directions: readiness is built by giving capable models to defenders, but "every expansion of access also expands who can misuse them." There's no version of this that resolves the tension; there's only a process for staging through it deliberately.

How Inkling was actually tested

This is the part with real specificity, not just principle. Three internal evaluation tracks: dangerous dual-use domains (CBRN, offensive cybersecurity, testing both what the model knows and whether it can operationalize that knowledge), a broad misuse suite covering direct harmful requests and agentic tool-use behavior, and a multimodal content evaluation across 17 languages and text/image/audio inputs, pairing harmful prompts with benign look-alikes.

Then four external red-teaming partners, each assigned a distinct lane: Scale AI on general policy-violation probing, Handshake AI on vulnerable-user interactions (suicide, self-harm, eating disorders, child safety), FAR.AI on CBRN and cybersecurity elicitation, and Apollo Research on loss-of-control behavior β€” scheming, evaluation awareness, sabotage. None of the four found capabilities that would meaningfully increase real-world risk beyond what's already achievable with existing open-weight models.

The most important test methodologically is the third: adversarial fine-tuning. Because open weights can be stripped of their safety training by anyone with the compute to fine-tune them, Thinking Machines treats refusal behavior as not a durable safeguard for an open release, and tests accordingly β€” training "helpful-only" variants of Inkling and Inkling-Small optimized to comply with harmful requests instead of refusing them, then re-running the dual-use evaluations. The variants showed no new uplift on CBRN and cyber tasks. That's the actual finding behind the model card's understated "competitive" β€” the refusal rate can be stripped out entirely and the model still doesn't reveal capability meaningfully beyond what other open weights already offer, which is the specific bar Thinking Machines set for whether releasing weights adds material incremental risk, not whether the model is risk-free in an absolute sense.

The open research bet: can dangerous knowledge be filtered out at training time?

The most consequential paragraph in the post isn't about Inkling at all β€” it's a hypothesis about what dangerous capability actually is. The default assumption in the field treats dangerous capability as an emergent property that scales with general intelligence, which is why cyber-range benchmark gains get treated as a capabilities milestone worth celebrating. Thinking Machines pushes back: they hypothesize that a meaningful share of dangerous knowledge β€” particularly in biology β€” rests on specific empirical facts (protocols, reagents, conditions, results) learned from particular documents in the pretraining corpus, not derived through general reasoning. If that's right, that knowledge can potentially be filtered out at the document level without degrading the model's general capability elsewhere.

They cite early evidence for this β€” document-level filtering of CBRN-related content from pretraining, work credited to Anthropic, EleutherAI, and the UK AI Security Institute, which reduces harmful-capability eval scores while leaving unrelated capabilities intact. They're careful to call this an open question, not a solved one: general reasoning might let a capable-enough model re-derive what was filtered, and the line between "dangerous" and "ordinary technical" knowledge may not cut cleanly. But if decoupling holds up, it changes the shape of every future release decision β€” from "how risky is this model" to "how much of that risk is even necessary to ship."

A staging ladder with fine-tuning access as its own rung

The practical mechanism is a widening-access ladder, and the interesting addition is a rung between "API access" and "open weights" that didn't have a clean name before: hosted fine-tuning, via Thinking Machines' own Tinker platform. It gives users real customization β€” their own data, loss functions, training loops β€” without handing over the weights themselves, so the provider keeps the ability to monitor, maintain guardrails, and revoke access. Four roles get assigned across the ladder: defenders get early inference access to patch and prepare (the post cites Anthropic's Project GlassWing, which gave Claude Mythos Preview to trusted organizations before public release, as precedent); vetted defenders get fine-tuning access to build detection tooling (citing the winning DARPA AI Cyber Challenge team and OpenAI's gpt-oss-safeguard classifiers, both fine-tuned defensive layers); vetted safety researchers get white-box access specifically because jailbreak research needs it β€” the post credits exactly this kind of access with the field discovering that refusal safeguards are bypassable via adversarial token-level optimization, removable via a single direction in activation space, and concentrated in just the first few output tokens; and the public gets monitored access before full release, to surface misuse patterns while there's still a lever to pull.

Thinking Machines is explicit that this isn't a conveyor belt that automatically ends in open weights β€” "a model held back today can become less risky as the ecosystem changes," which cuts both directions: evidence can also justify holding a model back for longer than planned. They're equally explicit about what's still missing: what evidence should justify moving between stages, how much uncertainty should block progression, and how "ecosystem readiness" gets measured at all. A more detailed framework β€” evaluations, access criteria, stop conditions β€” is promised as a follow-up, not delivered here.

What Thinking Machines is actually offering, versus asking for

The close of the post is a funding commitment as much as a research statement: Tinker safety grants, financial support for external researchers using the platform to study fine-tuning risks, alongside an open safety-team hiring call. It's also unusually generous with credit for a lab this young β€” explicitly naming Anthropic, EleutherAI, and the UK AI Security Institute for the data-filtering research, and OpenAI for the fine-tuning-safety framework its own gpt-oss-safeguard approach builds on. "No lab builds this path alone" reads like a genuine acknowledgment rather than boilerplate, if only because the alternative β€” every lab publishing a competing, incompatible safety taxonomy β€” is the more common failure mode in this industry.

What to expect next

  • The decoupling hypothesis becomes the thing to watch, not the staging ladder. If document-level filtering of dangerous knowledge holds up at scale, it's a genuinely new lever other labs will want independent verification of β€” expect replication attempts before endorsement.
  • "Helpful-only fine-tune, then re-run the dual-use evals" becomes the expected disclosure for any future open-weight release. It's the one part of this framework hard to argue against on principle, since refusal-only safety was already known to be fragile.
  • Watch whether "ecosystem readiness" gets an actual measurement. Thinking Machines admits it doesn't have one yet; whoever defines it first effectively sets the industry's release calendar for everyone releasing after them.

References: Thinking Machines Lab β€” "A Safe Path to Open Weights" Β· Thinking Machines Lab β€” Inkling announcement Β· related coverage on this blog: Inkling Β· the open-weight-ban fight Β· Anthropic's cybersecurity eval incidents