Project Hermes
Hermes is an Android translation assistant that scans text or speech and converts it into translated, accessible output on-device.
Core trick
Transcribes speech and extracts text from camera frames on-device, translates locally with QNN-accelerated models, then reads output via Android TTS.
Description
$29
README excerpt
# Hermes Hermes is an offline multimodal translator for Android. It translates spoken language and camera-captured text directly on-device using ExecuTorch and Qualcomm QNN acceleration, without relying on a cloud API during inference. The project is built for real-time travel, accessibility, and field communication scenarios where network connectivity, privacy, or latency make cloud translation less reliable. ## What It Does Hermes supports two translation flows: - **Voice translation**: record speech, transcribe it locally, translate it locally, then speak the translated result with Android TTS. - **Camera translation**: freeze a camera frame, extract the important visible text, detect its language, translate it locally, and show the result in a scrollable panel. Both flows let the user choose the source and target language. Camera mode can also detect the source language automatically after OCR. ## How It Works ### Voice Mode ```text Microphone audio -> Whisper Base QNN transcription -> Qwen3 QNN translation -> Android Text-to-Speech ``` Voice mode uses a native QNN Whisper path for speech-to-text and a native QNN LLM path for translation. The app logs each stage so latency can be measured on-device. ### Camera Mode ```text CameraX freeze frame -> ML Kit OCR -> important text selection -> ML Kit Language ID -> Qwen3 QNN translation -> scrollable translation panel + optional TTS ``` Camera mode focuses on the most important visible text rather than translating