the-agent-will-see-you-now
Safety-verified acute stroke screening from an audiovisual neuro exam — for clinical use.
Core trick
Fuses frame-sequence vision findings with Whisper speech transcription, and a verifier blocks any normal exam call lacking supporting evidence.
Description
Safety-verified acute stroke screening from an audiovisual neuro exam — for clinical use. A clinician (nurse, resident, or attending) performs a focused neurological exam normally; MedBridge captures the resulting video clip — moving images and audio — and its multimodal Evidence Extractor fuses Claude vision over frame sequences (facial asymmetry, arm drift over time, gaze tracking) with Whisper over the audio (speech clarity, naming, commands) into source-attributed evidence; the Clinical Planner works the NIH Stroke Scale one item at a time; the Safety Verifier refuses to call an exam "normal" without the evidence to support it and fires an emergency escalation on stroke red flags. Time-critical ✓ ("time is brain"), clinician-facing ✓ (the physician stays the accountable decision-maker), safer (a missed or over-claimed stroke finding is the liability the Verifier prevents) ✓ — grounded on the public-domain NINDS NIHSS videos, auto-segmented into 218 gold-labeled item clips.
csbrendan/the-agent-will-see-you-now★ 00 forksPythonApache-2.0Created Jul 18, 2026 · last push Aug 29, 2026
README excerpt
<div align="center"> [](https://youtu.be/OxmiFSo7HUg) **▶ [Watch the demo on YouTube](https://youtu.be/OxmiFSo7HUg)** · MedBridge — the AI co-clinician </div> --- **MedBridge: "The Agent Will See You Now"** — an audiovisual AI co-clinician for time-critical neurological screening and clinical documentation, for clinical use. Built at the **Abridge × Anthropic × Lightspeed "Future of Agentic AI in Healthcare" Hackathon**, hosted at **SHACK15** and organized by **Cerebral Valley**. 🩺 Ambient AI captures the conversation but misses the physical exam. MedBridge watches *and* listens: as a clinician performs the NIH Stroke Scale, a pipeline of specialized agents turns the audiovisual exam into a standardized, evidence-grounded score — auto-drafted to the patient chart for rapid clinician review and sign-off. **Under the hood:** - **Whisper (local)** — speech → transcript for spoken items - **Claude Opus 4.8 (vision)** — spatio-temporal analysis for facial asymmetry, arm drift, gaze - **Claude Opus 4.8 (planner)** — works the stroke scale one item at a time - **Claude Haiku 4.5** — one clear, calm instruction per step - **Claude Opus 4.8 (safety verifier)** — independent check that escalates red flags and keeps every finding backed by evidence The bigger idea: extend ambient intelligence into the visual modality via wearable glasses — giving an off-site neurologist object