Skip to content

Utama AI

Utama AI — Step into Singapore's past through the lens of your iPhone.

Code Video Cerebral Valley

Core trick

AR anchors a 3D character on a detected surface, while Gemini Live streams speech in both directions for real-time conversation.

Tags

culture/heritagevoicevision

Description

Utama AI — Step into Singapore's past through the lens of your iPhone. Point your camera at any flat surface and watch Sultan Hussein and his guardian lion materialize before your eyes in stunning AR. Have a real-time voice conversation with the Sultan — powered by Google's Gemini Live AI — as he recounts the founding of modern Singapore, while his lion prowls, roars, and even triggers an immersive VR flashback sequence. Built for the National Heritage Board, Utama AI transforms static history into a living, breathing encounter you can talk to.
pebblepaw/UTAMA_AI★ 00 forksSwiftCreated Mar 7, 2026 · last push Mar 8, 2026

README excerpt

# 🦁 Utama AI
_Talk to history. In augmented reality._

Built for the Gemini 3 Singapore Hackathon.

What if you could stand face-to-face with historical legends and hear the story in their own words?

Point your iPhone at any surface and Sang Nila Utama — the 13th-century Srivijayan prince who founded Singapore — materializes before you in AR, his lion prowling at his side. Ask him anything. He'll answer, in real-time, in his own voice — powered by Google's Gemini Live API.

Trigger a lion roar. Unlock an immersive VR flashback to the moment that gave Singapore its name.

## Demo

<a href="https://www.youtube.com/watch?v=f3ol2L_EAtY">
  <img src="VIDEO/THUMBNAIL.png" alt="Video Thumbnail" width="300">
</a>

<p>
  <a href="https://www.youtube.com/watch?v=f3ol2L_EAtY">Watch video here</a>
</p>

## Features

- **AR Character Placement** — Sultan and lion appear on any detected surface via ARKit plane detection
- **Real-Time Voice Conversation** — Bidirectional streaming audio with Gemini Live API; the Sultan responds in character with his own voice
- **Skeletal Animation Sync** — Character animations driven by voice amplitude (talking, gesturing, idle states)
- **Lion Roar Trigger** — Say something about the lion and it roars on command
- **VR Flashback Scene** — Ask the Sultan to show you the lion encounter and the app transitions into an immersive VR cinematic with gyroscope tracking
- **Fully In-Character AI** — The Sultan stays in 13th-century persona, references the Malay A

Team