Advicehub
AI clone with lip sync realtime - web search, computer use and sentiment analysis for loading state expression cloning
Core trick
Streams OpenAI Realtime voice audio and sends the reply to D-ID for lip-synced avatar video, with sentiment tuning the persona's mood.
Description
AI clone with lip sync realtime - web search, computer use and sentiment analysis for loading state expression cloning
README excerpt
# Deckard 🎭
> **Voice and Likeness Clone** - A real-time AI avatar platform combining voice conversation, sentiment analysis, and video generation to create lifelike digital personas.
---
## Overview
Deckard is an advanced AI platform that creates personalized avatar clones capable of real-time voice conversations. The system combines OpenAI's Realtime API, sentiment-driven mood dynamics, and lip-synced video generation (via D-ID) to deliver an immersive conversational experience.
### Key Features
- **Real-time Voice Conversations** - Seamless audio streaming with 24kHz PCM quality
- **Sentiment-Aware Personas** - Dynamic mood changes based on conversation tone
- **Multi-Agent AI System** - Coordinated specialists for web search, sentiment analysis, and browser automation
- **Lip-Synced Video Generation** - D-ID integration for realistic talking avatar videos
- **Computer Use Capability** - Browser automation for web research and expert search
- **Image Analysis** - Upload and discuss images within conversations
- **Three Unique Personas** - Joi, Officer K, and Officer J with distinct voices and personalities
---
## System Architecture
```mermaid
graph TB
subgraph "Frontend - Next.js"
UI[User Interface<br/>React Components]
WS_CLIENT[WebSocket Client<br/>Audio Streaming]
VIDEO[Video Player<br/>Persona Display]
end
subgraph "Backend - FastAPI"
WS_SERVER[WebSocket Manager<br/>Session Handler]
subgraph "AI Agent Syst