Google published results today for AMIE (Video) — a real-time video configuration of AMIE, its research medical AI system, built on Gemini and Project Astra. Unlike the text-based AMIE that preceded it, this version watches: it perceives non-verbal cues, guides a patient through physical examination maneuvers over the camera, and reasons diagnostically, all while holding a natural spoken conversation. In a 300-consultation randomized study, clinical evaluators rated it on par with board-certified physicians across the core clinical competencies — and rated it significantly better than those same physicians at one specific thing: eliciting physical signs and guiding patients through exam maneuvers. Both the headline result and its limits are worth taking seriously, and Google's own writeup is unusually direct about the second half.
The engineering problem: talking fast and thinking slow, at once
The interesting design problem here isn't medical, it's architectural, and it's one this month's harness coverage keeps running into in other domains: a single model can't simultaneously hold natural conversational pace and do careful multi-step reasoning, because reasoning takes time and silence erodes trust in a live conversation. AMIE (Video)'s answer is an asynchronous multi-agent architecture, three specialized agents running continuously in parallel:
- Talker agent — the patient-facing voice, optimized for low-latency, natural conversational flow, incorporating guidance from the other two agents as it becomes available rather than blocking on it.
- Planner agent — runs in the background, continuously updating the differential diagnosis and management plan, flagging information gaps, re-prioritizing what to ask next.
- Perception agent — continuously reads the audio-visual stream for clinically relevant non-verbal cues (visible distress, physical findings, auditory signals) and contextualizes them against the ongoing conversation.
Notice what this is: the same "decouple the fast interactive layer from the slow reasoning layer" pattern that shows up whenever anyone builds an agent that has to act in real time and think carefully — it's the medical-domain instance of a harness design problem the industry keeps solving from scratch, domain by domain, rather than as a shared primitive. AMIE (Video) is built on top of Gemini and Project Astra rather than being a new foundation model — this is an orchestration and evaluation story, not a scaling story, which is worth naming plainly since it's easy to read "Google AI" headlines as always being about a bigger model.
The study, and the two clean results
Google ran a randomized, three-arm Objective Structured Clinical Examination (OSCE) study — the same standardized-patient format used in actual medical training and licensing — spanning 100 clinical scenarios across five body systems (cardiopulmonary, abdominal, HEENT, neurological/psychiatric, musculoskeletal), with 15 trained patient actors conducting 300 consultations split across: AMIE (Video), AMIE (Text) as a baseline to isolate what audio-visual actually adds, and ten board-certified primary care physicians consulting over the same video interface. A separate, independent panel of 20 experienced PCPs scored every consultation blind, using established clinical rubrics plus case-specific criteria.
Two results stand out cleanly. Parity on the core competencies: history-taking thoroughness, diagnostic accuracy, management appropriateness, and communication quality all came back statistically on par between AMIE (Video) and the physicians, and AMIE (Video) matched or beat its own text-only sibling on the same measures — the video modality wasn't a regression, which given how much harder real-time audio-visual perception is than parsing text, was not guaranteed. A specific win on physical examination: AMIE (Video) scored significantly higher than both PCPs and AMIE (Text) at eliciting physical signs and proactively guiding patients through exam maneuvers over video.
That second result is the one worth sitting with rather than just repeating. An AI outperforming licensed physicians at a hands-on clinical skill, even conducted virtually, is a genuinely striking claim, and the study doesn't fully explain the mechanism — plausible candidates include systematic checklist-following without impatience or time pressure, a lower bar for asking a patient to repeat or re-demonstrate a maneuver, or simply that AMIE's exam guidance was written into its training more thoroughly than any individual PCP's variable in-the-room habits. It's a result to watch replicate, not a settled fact about AI versus doctors.
Patients liked it — on a metric that deserves a second look
Patient actors "strongly preferred" the video interface over text-based chat, rating it easier to use and more effective for communicating concerns — an unsurprising result given how much diagnostic information a written description of symptoms discards. More notable: patient actors rated AMIE (Video) favorably on empathy, rapport, and confidence in care, ahead of both the human physicians and AMIE (Text). Worth a flag here: at least one third-party summary of the underlying paper we found while researching this describes patients preferring PCPs specifically on "rapport and partnership building" as a narrower sub-metric — which would sit in tension with Google's own topline framing. We couldn't independently verify which reading is right (arXiv is not reachable from here), so treat the empathy/rapport finding as Google's own reported result, not yet cross-checked against a source we could access directly.
The limitations section is doing real work
Credit where it's due: Google's own writeup states the caveats plainly rather than burying them. Every consultation in this study used trained patient actors, not real patients — skilled actors "cannot fully replicate the complexity and unpredictability of real clinical encounters," and the 100 scenarios were limited to conditions that can be authentically acted, which by construction excludes presentations where audio-visual perception would matter most diagnostically. Automated evaluation, run separately from the human study, found "occasional perceptual and reasoning errors" even where overall conversation quality scored high, and the system — built on Project Astra, which Google explicitly calls a prototype — has intermittent technical issues that disrupt conversational naturalness. Google's own conclusion: "studies with real patients and real clinical conditions" are "an essential next step before any conclusions about real-world utility can be drawn." That's the correct scope for a randomized-but-simulated OSCE study, stated by the people with every incentive to round it up.
This isn't AMIE's first pass at real-world evidence, either — Google references an existing clinical feasibility study with Beth Israel Deaconess Medical Center for text-based AMIE, and an ongoing nationwide randomized study with Included Health evaluating AI in real virtual care. AMIE (Video) sits upstream of both: a capability demonstration that has to clear real-patient validation before it's a clinical claim rather than a research one.
What to expect next
- Watch for the physical-exam-guidance finding to replicate — with real patients, where compliance, embarrassment, and pain thresholds behave nothing like a trained actor's.
- Watch Project Astra's prototype status resolve or not. AMIE (Video) inherits every limitation of the platform it's built on; a maturing Astra is a precondition for this becoming more than a research demo, independent of anything AMIE-specific improving.
- Watch whether other labs publish a comparable audio-visual clinical study. OpenAI, Anthropic, and Microsoft all have active health efforts; a second lab replicating "expert-level video consultation, in simulation" would tell us whether this is a genuine capability threshold or an artifact of Google's particular architecture and evaluation design.
- Watch the regulatory and real-world-study path, not the paper. Google is explicit that this is research, not a product — the real signal is whether the BIDMC and Included Health studies eventually report on video-capable AMIE with real patients, and what that shows.
References: Google Research — Advancing AMIE towards expert-level audio-visual clinical consultations · arXiv — Towards Expert-level Medical AI for Real-time Video Consultations (2608.09861) · related coverage: The AI Scientist's Real Problem Was Never the Science · Prime Agent and the RLM Harness · Frontier Arcade: trends & predictions