PrivateDoc
PrivateDoc is an offline, on-device visual assistant that uses ML Kit OCR, ExecuTorch, MiniLM, and Llama 3.2 to scan, index, and answer questions about private documents. Its strict RAG pipeline provides source-grounded answers. Powered by SnapDragon.
Core trick
Runs ML Kit OCR, MiniLM retrieval, and on-device Llama 3.2 offline, highlighting retrieved OCR boxes on the scanned image as evidence.
Description
PrivateDoc is an offline, on-device document intelligence assistant designed for sensitive personal paperwork, running on a Samsung S25 Ultra phone powered by Qualcomm SnapDragon. It combines ML Kit OCR, ExecuTorch, MiniLM semantic retrieval, and an on-device Llama 3.2 1B LLM to scan, classify, index, and answer questions about prescriptions, contracts, bills, IDs, and other documents without requiring an Internet connection. A strict retrieval-augmented generation (RAG) pipeline ensures every response is grounded in retrieved OCR evidence, with supporting text highlighted on the original document to improve transparency and reduce hallucinations. By performing OCR, retrieval, and language generation entirely on the device, PrivateDoc preserves user privacy while delivering fast, reliable, and source-backed document understanding.
README excerpt
# PrivateDoc **English** · [中文](README.zh-CN.md) > An **offline, on-device** document assistant for sensitive family paperwork. > Scan a prescription, WiFi card, bill, ID, or contract; build a local searchable > index; and answer questions with evidence highlighted on the original image — > **without any data ever leaving the phone.** `On-device · Offline · Source-grounded` --- ## Why it matters Families constantly need to look things up inside private paper documents — medication labels, prescriptions, contracts, bills, IDs. Cloud OCR and cloud chat tools are risky for this data, because raw images, extracted text, and the user's questions can all leave the device. PrivateDoc keeps **everything local**: the app manifest requests **no Internet permission**, and OCR, retrieval, and LLM generation all run on the phone. ## Highlights - **Three-tier on-device AI**, all offline: OCR → classification → semantic retrieval → 1B LLM, no network. - **Llama 3.2 1B Instruct (SpinQuant INT4)** running on-device through **ExecuTorch**, verified on a Galaxy S25 Ultra. - **Strict RAG**: the LLM may only rephrase retrieved OCR snippets; a grounding check + `NOT_FOUND` sentinel + deterministic template fallback prevent hallucination and block medical/legal/financial advice. - **Robust engineering**: per-query embedding cache, KV-cache reset, EXIF-aware downsampled image decoding, graceful degradation — it actually demos without crashing. ## Architecture ``` CameraX captur