Beethoven
Takes unfinished scribbles from Beethoven and composes the rest.
Core trick
Feeds a sketch image to a multimodal model that outputs structured score JSON, validates it, and renders distinguished inferred and newly composed notes.
Description
Takes unfinished scribbles from Beethoven and composes the rest. Displays the notes inferred from the scribbles and also notes generated by the model. As the music is playing, you can see the notes being played.
There is also a separate blender script that takes the generated sheet music and turns it into a visualization that is supposed to emulate how Beethoven with his loss of hearing must have felt the music as he was composing it.
README excerpt
# Beethoven A handwritten Beethoven sketch becomes a playable string quintet—with visible distinctions between uncertain readings and newly composed notes. Upload a sketch and historical context. GPT-6 Astra interprets the image and proposes a structured score. The app validates it, engraves all five parts, plays the music, and explains each bar. Visualization video needs to be generated separately by downloading the sheet music and running the script. | Sketch + prompt | Playable score + visualization | | --- | --- | |  |  | ## Try it ```sh npm ci npm run build npm start ``` Open [localhost:3000](http://127.0.0.1:3000). The saved Beethoven example needs no credentials or regeneration. Click **Play**, a note, or a bar. **Source** shows the original image and prompt; **Sheet music** exports MusicXML. For new generations, copy `.env.example` to `.env.local` and configure either the authenticated local Codex backend or an OpenAI API key. See [generation setup](docs/ENGINEERING.md#generation-setup). Never commit credentials. ## What does what | Component | Responsibility | | --- | --- | | [Astra task](lib/realization.ts) | Read the manuscript image; propose notes, origins, and bar explanations. | | [Validation](lib/compose.ts) | Check the structured result; allow one repair attempt. No fallback composition