Skip to content

Edtools Labs

Edtools Labs is an AI tutor powered by Claude Opus 4.7 that teaches STEAM through voice, live drawing on a shared canvas, and smartphone sensors—so any phone becomes a simple interactive lab.

Code Video Cerebral Valley

Core trick

The tutor alternates one spoken sentence with canvas strokes, and pulls phone sensor readings into predict-tilt-reflect lab cards.

Tags

educationvoice

Description

Edtools Labs is an AI tutor powered by Claude Opus 4.7 that teaches STEAM through voice, live drawing on a shared canvas, and smartphone sensors—so any phone becomes a simple interactive lab. Goal: democratize rigorous, hands-on science learning globally, in a learner’s own language.
soyandresalcedo/Edtools-labs★ 00 forksTypeScriptGPL-3.0Created Apr 23, 2026 · last push Apr 27, 2026

README excerpt

Edtools Labs

AI agent designed to teach STEAM to anyone by leveraging "hands," voice, and
smartphone sensors to turn any device into an interactive lab platform. The
goal is to democratize this kind of education.

This repository contains a Claude-powered kinematics tutor that draws on a
canvas and narrates every step with synchronized captions.

What you see

A student picks a chip or types a kinematics question. The tutor speaks one
short sentence and then draws on an Excalidraw canvas, alternating speech and
strokes step by step. When body intuition would help, an inline Lab card
appears in the chat: predict → tilt the phone → reflect, and the lesson
auto-continues with a handoff back to the teach mode.

Architecture

flowchart LR
  subgraph Client[Browser]
    UI[Sidebar + Chips + Input]
    Hook[useLessonStream]
    Speech["useSpeech queue, prefetch=1"]
    Draw["draw worker, serial Promise chain"]
    Canvas[Excalidraw 800x600]
    Tilt["useTilt / simulated tilt"]
  end

  subgraph Server[Next.js route handlers]
    Lesson["/api/lesson SSE"]
    Tts["/api/tts proxy"]
  end

  subgraph External[External APIs]
    Claude["Anthropic Claude Opus 4.7"]
    Eleven[ElevenLabs TTS]
  end

  UI -->|question + lang + history| Hook
  Hook -->|POST JSON| Lesson
  Lesson -->|messages.stream + tools| Claude
  Claude -->|tool_use deltas| Lesson
  Lesson -->|"SSE tool_call events"| Hook
  Hook -->|speak text| Speech
  Hook -->|"draw_* / clear_canvas"| Draw
  Speech -->|POST text| Tts

Team