Helixstreet — autonomous genomic fleet

Inspiration Every genomic sample that reaches a lab starts a relay of humans: compliance review, PII scrubbing, pipeline babysitting, annotation lookups, record-keeping — days of coordination per file. We wanted to know what it looks like when the relay is replaced by a governed fleet of agents that the file itself wakes up. Not a chatbot with tools — a system with no chat loop anywhere . What it does Drop a raw sequencing file (BAM) into a Cloud Storage bucket and walk away: Intake agent screens the patient manifest with Model Armor (prompt-injection, malicious-URI, PII filters) and de-identifies it with Cloud DLP . Variant-calling agent dispatches Google DeepVariant as an asynchronous Cloud Batch job, checkpoints state to a Firestore Memory Bank — and the fleet goes dormant. Minutes later the VCF lands; Eventarc wakes the fleet again . The analyst agent parses the real calls and cross-references 1000 Genomes in BigQuery. The pharmacist agent screens variants against a curated CPIC drug-gene reference (e.g. "CYP2C19 poor metabolizer — clopidogrel may not activate") and attaches peer-reviewed papers it fetches live from Europe PMC. The steward agent persists a digital twin behind a hash-chained audit ledger whose integrity the dashboard re-verifies on every refresh. Measured result: raw reads to fully-annotated twin in ~9 minutes, zero human touches — and three samples processed concurrently by the same fleet. Every reasoning step is an OpenTelemetry trace in Cloud Trace. How we built it Four principles shaped the architecture: Right-sized hybrid model tier. The high-throughput intake gatekeeper runs on Gemma 4 31B (Gemini API backend, native function calling; graceful fallback to Gemini if the key is absent), while the four reasoning agents run on Gemini 3.5 Flash. At population scale, screening every sample with a frontier model is waste; a small open model is the production answer. Events decide when , the model decides how . Routing between pipelines is deterministic (Eventarc + a regex router); behavior inside each stage is agentic — Gemini 3.5 Flash (Vertex AI, global endpoint) driving tools through Google's Agent Development Kit ( Agent , SequentialAgent , Runner ). Dormancy over polling. The async gap while DeepVariant runs is bridged by Firestore state, not a waiting process. The same mechanism would carry a multi-week job. Identity at the platform layer. Three least-privilege service accounts: the analyst can read BigQuery but cannot launch compute; the dashboard reads state but cannot invoke agents; the fleet endpoint accepts exactly one identity — Eventarc's. Zero API keys in the codebase — the single key (Gemma's Gemini-API key) lives in Secret Manager. Governance as data. Agent registry in Firestore, audit entries written by the agent that did the work, hash-chained so tampering breaks the chain. Stack: Gemini 3.5 Flash · ADK · DeepVariant · Cloud Run · Cloud Batch · Eventarc · Cloud Storage · Firestore · BigQuery · Cloud DLP · Model Armor · Cloud Trace · FastAPI · Streamlit. Challenges we ran into Five production incidents, each of which hardened the system: Model placement — gemini-3.5-flash serves only from Vertex's global endpoint, while Batch jobs are regional; one env var was doing two jobs. The idempotency trap — we claimed events as processed before success, so a failed run swallowed its own Eventarc retry. Claims now release on failure: exactly-once on success, self-healing on error (later proven live when a quota-exhausted run failed, was redelivered, and completed untouched). GCS-FUSE's vanishing directories — empty directories have no backing object and evaporate when the kernel cache expires, killing DeepVariant's final write minutes into the job. We back the output prefix with a placeholder object before dispatch. Quota-aware agents — concurrent pipelines cannibalized a thin per-minute model budget; model-level 429 backoff (15s→120s) plus right-sized concurrency fixed it. Unpinned dependencies — an upstream release broke Cloud Build mid-hackathon; all dependencies are now pinned from a verified environment. Accomplishments that we're proud of A genuinely autonomous multi-agent system: upload in, governed digital twin out. Real heavy compute (DeepVariant on Batch), real enterprise guardrails (Model Armor + DLP), real population genomics (1000 Genomes), real clinical references (CPIC). The analyst agent noticed — unprompted — that VCF and BigQuery coordinates disagree by exactly one base pair (1-based vs 0-based) and reasoned through it. A join-by-position ETL would have silently returned zero matches. 3 concurrent patients → 3/3 twins, 6/6 pharmacogenomic findings each, zero errors. What we learned Autonomy is an infrastructure property before it is a model property: exactly-once event handling, claim release on failure, durable state across dormancy, and platform-level identity are what let an agent be trusted to act alone. The model supplies judgment inside stages; the architecture supplies the right to act. What's next for Helixstreet — autonomous genomic fleet Whole-genome scale on larger Batch shapes, per-department registries with A2A hand-offs, dbSNP joins in BigQuery to assign rsIDs to called variants, and managed Vertex AI Memory Bank for cross-session personalization.