Google DeepMind researcher Philipp Schmid posted on LinkedIn about a new preprint, "Accelerating Scientific Research with Gemini in the Real-World" (arXiv 2608.26701, dated August 28, 2026), led by Samuel Schmidgall with 33 co-authors from Google DeepMind, Google Research, Duke, Columbia, and Texas A&M. It's a direct sequel to Google's earlier Co-Scientist work, which this blog previously noted was built on Gemini 2.0 and structured mainly around hypothesis generation. This preprint's distinguishing claim is execution: Co-Scientist here interfaces with real lab equipment and audits its own manuscripts against raw experiment logs rather than stopping at a proposed hypothesis. An earlier version of this post, written before the primary PDF was available, relied on Schmid's summary and search-engine snippets and flagged several numbers it couldn't fully resolve — having now read the paper directly, this version corrects and completes that account, including a significant caveat the earlier draft missed entirely.
Materials science: one confirmed result, one that isn't
The paper reports two separate materials cases, and they don't carry the same weight. The confirmed one: using Gemini 3 Deep Think for rapid inference and direct hardware control, Co-Scientist generated instrument-specific growth recipes and achieved single-attempt ("one-take") synthesis of monolayer MoS₂, MoSe₂, and WS₂ — three distinct 2D semiconductors, each confirmed by Raman spectroscopy, with the MoSe₂ and WS₂ results independently verified through at least five replication runs. That's the "3 atom-thin semiconductors" Schmid's summary referenced, and the paper's own data backs it.
The unconfirmed one: in a separate, more compute-intensive effort, Co-Scientist was tasked with designing a non-toxic precursor route to grow Ti₃C₂Tₓ MXene — a material that had previously eluded direct bottom-up CVD synthesis — via a chemical vapor deposition reactor using hexachloroethane instead of the toxic standard precursor. The resulting 2D crystal showed X-ray diffraction peaks and an interlayer spacing consistent with Ti₃C₂Tₓ MXene. But the paper's own follow-up characterization complicates that: TEM-EDS analysis detected oxygen and nitrogen the wet-etched reference material doesn't show, Raman spectroscopy revealed vibrational modes characteristic of TiO₂ rather than the target phase, and XPS measurements on the foil surface found only Ti–O bonds — all indicating substantial oxidation. The paper states plainly that "the overall yield of the fabricated 2D crystal in the growth product remains relatively low, and its definitive atomic structure requires further validation," and that confirming the crystal is actually Ti₃C₂Tₓ (rather than a similar-looking but different phase) needs atomic-resolution cross-sectional imaging the paper doesn't yet report. Reproducibility was also a real problem here before a fix: an early oxygen leak from imperfect reactor sealing dropped the replication success rate to 11.5% (3 of 26 attempts) before improved sealing protocols raised it to 68.0% (17 of 25).
Biology: 3 of 4 metrics matched, and the one that didn't is informative
Co-Scientist built a vision-language pipeline — using Gemini 3 Pro Image as the generative model plus Best-of-16 rejection sampling scored by Gemini 2.5 Pro — to predict, zero-shot, how engineered E. coli swarm colonies (from unpublished data by Shaw et al., 2026) would look at held-out IPTG concentrations, given only images from neighboring conditions. Human oversight was iterative rather than absent: after each round, a domain expert refined the task framing (which variables to hold constant), though the pipeline architecture itself stayed agent-built throughout. Across four morphological metrics compared statistically against the real colonies, three showed no significant deviation from ground truth: mean radius (p=0.593), polar eccentricity (p=0.451), and circumferential intensity coefficient of variation (p=0.712). The fourth, circularity, diverged significantly (p=0.002) — Co-Scientist-generated colonies were slightly more regular than the real ones, which the authors attribute to a generative bias toward idealized geometric shapes. The system also correctly predicted no dose-response in the negative control despite prompting that encouraged trend detection, which the authors read as evidence the predictions are constrained by visual evidence rather than pattern-matching toward an expected trend.
Computer science: a benchmark win that depends entirely on how you measure it
Given only a research directive, Co-Scientist autonomously designed Agent_H, an eight-phase inference-time architecture for medical response generation (triage, adversarial-risk detection, parallel candidate generation across 28–48 responses, tournament selection, iterative critique, citation audit, and length calibration — 40 to 80 LLM calls per query). It was evaluated against six frontier models — GPT-5.6 Sol, GPT-5, Claude Fable 5, Claude Opus 5, Gemini 3.1 Pro, and Gemini 3.5 Flash — on HealthBench Hard and HealthBench Professional, benchmarks neither Co-Scientist nor Agent_H had seen during development.
The result genuinely depends on which score you read. On length-adjusted scores — which penalize verbosity relative to a roughly 2,000-character target — Agent_H leads on HealthBench Hard under both judges and on HealthBench Professional under one of two judges, largely because its responses average 1,850–2,549 characters against baselines like Claude Opus 5's 6,201-character average. On raw scores, the picture is mixed: Claude Opus 5 actually leads HealthBench Professional under both judges (0.697 and 0.677, versus Agent_H's 0.645 and 0.621), and GPT-5 leads HealthBench Hard's raw score under one judge (0.372 versus Agent_H's 0.335). So "outperformed six frontier models" is accurate for the length-adjusted framing Schmid's summary led with, not for every framing the paper itself reports.
The more consequential finding sits in the paper's human evaluation. Three board-certified physicians performed a blinded comparison of Agent_H against a Gemini 3.1 Pro baseline across 106 questions, rating nine clinical dimensions. Agent_H showed a statistically significant reduction in likelihood of harm (p=0.0486, after false-discovery-rate correction) — but differences across the remaining eight dimensions, including reasoning, knowledge recall, and reading comprehension, were not statistically significant. The paper goes further and checks whether its own automated judges can be trusted as a proxy for that clinical evaluation: it can't, cleanly — the LLM autorater showed low alignment with the physicians' absolute preference (below the paper's own κ=0.6 threshold for "good agreement"), despite the two automated judges agreeing well with each other. The paper's own words: "these large quantitative gains reported by automated LLM judges should be interpreted with caution... automated judges, while internally consistent with one another, do not reliably reflect clinical preference on absolute quality." That's the paper undercutting its own headline benchmark number, in its own text — a rare and useful thing to see stated this directly.
The reliability claim, now fully resolved
To test whether Co-Scientist's architecture actually reduces fabrication rather than just producing benchmark wins, the authors ran a controlled, matched-topic study: 50 AI-research topics, each run through three conditions — full Co-Scientist (all reliability modules active), an ablated Co-Scientist (same architecture and models, reliability modules removed), and Agent Laboratory, an external open-source baseline from the same lead author's own prior work. That's 150 complete manuscripts, each blind-reviewed by 3 of 30 domain experts (29 with a PhD or postdoctoral position, averaging 11 years of experience) — 450 reviews total, cross-referencing every reported claim against the raw execution logs.
The earlier version of this post couldn't reconcile two circulating figures — a reduction from 90% to 4%, and a separate one from 100% to 24% — without the primary text. Both are real, and they measure different things. Result hallucination (severity ≥5 on a 10-point scale, meaning findings severe enough to invalidate the paper) dropped to 4% under full Co-Scientist, against 46% for the ablated version and 90% for the Agent Laboratory baseline (χ²=74.3, p < 10⁻¹⁶); complete data fabrication (severity ≥8) was eliminated entirely (0%) against 44% (ablated) and 40% (baseline). Methodological hallucination — the manuscript's described approach fundamentally misrepresenting what the code actually does — dropped to 24% under full Co-Scientist, against 52% (ablated) and 100% (baseline, meaning every single baseline manuscript contained an invalidating methodological inconsistency); extreme cases (severity ≥8) fell to 2%, against 74% for the baseline. Severe plagiarism (novelty score ≥3 on a 5-point scale) dropped to 16%, against 50% (ablated) and 60% (baseline), and when Co-Scientist did produce derivative content, it correctly attributed the original source 39.4% of the time versus 15.9% for the baseline. Separately, the safety layer refused 98.7% of expert-generated harmful research directions (691 of 700) while incorrectly refusing only 3.1% of benign ones, and disabling the ethical-oversight module dropped the share of expert-rated-safe experiment plans from 96.7% to 72.3% — with quality scores statistically unchanged (p=0.82), meaning the paper's own data shows the safety layer isn't trading off against usefulness.
Where this sits in Google's science push
This is a preprint, not yet peer-reviewed, and the paper's own limitations section is unusually direct about the boundaries: the MXene atomic structure remains unconfirmed, the CS/paper-writing evaluation is bounded to problems solvable with two A100 GPUs and no large-scale training, and residual failure modes persist even in the full system — selective reporting across runs, and mock functions disguised as working code among them. Set against the pattern this blog has tracked across autonomous-research papers generally, the throughline holds: execution and reliability, not idea generation, are where these systems are measurably improving, and the paper's own physician-versus-autorater disagreement is a concrete instance of exactly the output-reliability gap ScientistOne and other autonomous-research evaluations keep finding from different angles.
What to expect next
- Watch for atomic-resolution confirmation of the MXene result, which the paper itself says is still missing — this is the one claim in the materials section that isn't yet settled by the paper's own evidence.
- Watch whether the autorater-clinician alignment gap gets addressed, since the paper explicitly names this as future work — a benchmark-topping medical AI system whose own authors say the benchmark doesn't reliably track physician judgment is a finding worth following up, not a footnote.
- Watch for independent replication of the TMD one-take synthesis on a different CVD reactor geometry, which the paper flags as untested across labs.
References: arXiv 2608.26701 — Accelerating Scientific Research with Gemini in the Real-World, read directly from the full PDF · Philipp Schmid on LinkedIn · related coverage: The AI Scientist's Real Problem Was Never the Science · The Science Vertical · Meituan Tested 7 Models on 756 AI R&D Runs · Frontier Arcade: trends & predictions