Lantern: Model anything
Lantern turns a smartphone into an AI-powered 3D scanner. Using on-device AI with ExecuTorch, it generates accurate 3D meshes in real time without cloud processing, LiDAR, or external hardware, making 3D scanning fast, private, and accessible.
Core trick
Fits per-frame affine scale to ARCore sparse depth, then fuses monocular depth maps into a TSDF volume exported as a metric mesh.
Description
$25
README excerpt
# Lantern — Phone Scan → 3D Point a phone at a real object, get a clean, correctly-scaled, **CAD-ready `.glb` mesh** — reconstructed **on-device**, with the depth network running on the Snapdragon Hexagon NPU via ExecuTorch. Built for the **Qualcomm × Meta ExecuTorch Hackathon** (Jun 27–28, 2026). ## How it works Faithful reconstruction (not generative): **TSDF fusion of ARCore-aligned monocular depth maps.** ``` RGB + ARCore pose + raw depth + intrinsics │ ├─► Depth-Anything-3 Small ......... dense RELATIVE depth (affine-invariant) │ (DA-V2 Small = fallback) NPU-optimized for SM8750 (~43 ms/frame) ├─► affine scale/shift solver ...... fit metric ≈ s·pred + t vs ARCore sparse │ metric depth → real meters (solve s AND t) ├─► TSDF fusion (Open3D) ........... average many views → one clean surface ├─► marching cubes ................. raw .glb mesh └─► import_and_clean.py (Blender) .. watertight + normals + m→mm → CAD-ready .glb ``` Two depth sources by design: the **monocular depth net** gives a dense, smooth depth in unknown units; **ARCore** gives sparse-but-metric depth. The affine solver marries them so the mesh is both dense *and* correctly sized. See [`roadmap.md`](roadmap.md) for the full execution plan, decision log, and risk register. **Depth net: Depth-Anything-3 Small** (Apache-2.0, 24.7 M params, 518×518) is the host default and the on-device target — Qualcomm AI Hub already exports it NPU-