Data Ingestor
Data Ingestor turns the hours a data curator spends hand-reformatting every CRO's differently-shaped Excel/CSV into a one-click review.
Core trick
Claude proposes column mappings with per-field confidence from a bounded sample, while deterministic Python extracts every exported value and a validator checks declared constraints.
Description
Data Ingestor turns the hours a data curator spends hand-reformatting every CRO's differently-shaped Excel/CSV into a one-click review.
The pain I keep meeting at the scientists I work with: the ingest parser is hardcoded to one vendor. Each new vendor needs its own bespoke parser, and when a vendor quietly changes its Excel, last quarter's parser breaks. Maintaining that pile is the real cost.
Data Ingestor has no per-vendor code. You declare the fields you need; Claude judges the file's structure — where the table actually starts, whether a sheet is row-per-record or key-value — and proposes a column mapping with honest per-field confidence and a plain-English reason for each pick. Anything uncertain is flagged yellow, never silently guessed. Nothing is saved until a human clears every field.
The design rule: Claude judges, Python reads. The model may see a bounded sample in order to judge, but every exported value is read from the file by deterministic code; a no-LLM validator re-checks it against your declared constraints.
Beyond a wrapper: a learning loop (a repeat file from the same lab auto-maps at full confidence with zero Claude calls), a canonical schema + vendor-alias crosswalk with provenance, and a headers-only privacy mode.
README excerpt
# Data Ingestor **Upload any lab's messy CSV/Excel → Claude maps its columns to the fields you asked for, with honest per-field confidence → you review and confirm. Manual reformatting becomes a one-click review.** Every contract research organisation (CRO) ships assay data in a different shape — different column names, units, date formats, header positions. A data curator spends hours hand-reformatting each file before it can be used. Data Ingestor does the first pass: you declare the target fields you want, Claude proposes a column mapping with a plain-English reason and a confidence score for each field, anything uncertain is flagged yellow, and **nothing is written until a human clears every flag.** Built for the **Built with Claude: Life Sciences** hackathon (Builder track). Standalone, open-source (MIT), runs entirely on synthetic data. --- ## The three principles 1. **Claude proposes, human disposes.** Claude only returns a *proposed* mapping via structured output, with per-field confidence and a reason. The human confirms; only then is anything saved. *"Trust the numbers"* — the LLM never touches production truth directly. 2. **Never guess silently.** Missing unit → flag it. Ambiguous column → offer ranked alternatives with % confidence, never a bare "I don't know". A blank or out-of-range value is flagged, not quietly accepted. 3. **Nothing saved until every field is clear.** Confirm and export are blocked while any uncertain (yellow) field remains — and the se