r/ClaudeAI
•
u/Sourcecode12
•
22h ago
Built with Claude I built a musical instrument you play in thin air (Try it yourself)
I used Claude Fable 5.2 to build a musical instrument you play in thin air. It's called Music Maker. Open it in your browser, turn on your webcam.
How it works:
You turn on your webcam, pinch with both hands and pull them apart. A string stays in the air where you left it. Sweep a fingertip across it and it plays.
- Strings and notes: each string gets its own instrument and its own scale (major, minor, pentatonic, blues, Dorian, Hijaz). Where you cross the string decides the note, and longer strings hold more notes. A string can also play chords instead of single notes.
- 24 instruments: guitar, electric guitar, harp, oud, qanun, sitar, piano, electric piano, organ, marimba, kalimba, steel pan, violin, cello, strings, flute, trumpet, voices, bells, synth, bass, 808 bass, drums and hand drums. All of them are played the same way, drums included.
- Expression: rest a finger on a string to sustain a note or roll a drum. Pinch a string and pull it sideways to bend the pitch, then let go to snap it. Pinch empty air and drag to sweep a filter and echo over the whole mix.
- Building a track: there are six pre-made beats you switch on one part at a time (kick, snare, hats, bass, chords). A looper records what you play and keeps it going while you add the next layer. A "follow chords" mode retunes every string to the current bar, so whatever you play fits.
- Recording: you can record video with sound or sound only, in landscape or vertical, and save MP4/WebM, MP3 or WAV. You can also export each beat part and each layer as separate stems.
- Setups: you can save a whole stage (strings, instruments, beat, key, layers) and load it back in one click, or start from a ready-made layout.
- Changing the loop Style: You can record your loop, export it as WAV or MP3, import it to Suno AI and turn it into any style you want.
How it was built:
- Hand tracking: MediaPipe's hand landmarker runs in the browser and gives 21 points per hand. A pinch is measured as the gap between thumb and index fingertip relative to hand size, so it works at any distance from the camera. The camera picture never leaves your machine.
- Gestures: this took the most iteration. A pluck is a fingertip crossing from one side of a string to the other, with a small dead zone so tracking jitter doesn't trigger notes. A finger that settles on the string is a hold. A pinch on the string is a pull.
- Sound: every sound is synthesised live with the Web Audio API, with no samples. Plucked strings use Karplus-Strong. The rest are built from oscillators, FM and filtered noise, with a generated reverb.
- Timing: the beats and loops run on a look-ahead scheduler tied to the audio clock, and loop layers can snap to the grid.
- Export: video uses MediaRecorder on the canvas. MP3 is encoded in the page, and stems are rendered offline, faster than real time.
- Packaging: the whole thing is one HTML file, with no build step, backend or account.
- Process: I made it with Claude over one long session. I described what I wanted, tested each version, and fed back what was wrong or missing.
Why:
I wanted to experiment with AI tools and motion tracking, and see what happens when you combine them with music. The result is an instrument you play with nothing but your hands and a webcam.
Limits:
It needs Chrome or Edge on a computer, decent light, and your hands about 40 to 80 cm from the camera. The instruments are synthesised, so the violin won't fool anyone.
Try it: https://hashem-al-ghaili.github.io/music-maker/
Code: https://github.com/Hashem-Al-Ghaili/music-maker
What it cost to build:
- Tokens: about 31 million in total across 83 requests. Roughly 30 million of those were cached re-reads of the same context, which are billed at a fraction of the normal rate. The model wrote about 286,000 tokens of output, which is the code, the tests and the replies.
- Where it went: about 44% on writing code and running tests, about 43% on the model re-reading its standing instructions at every step, and the rest on conversation and research.
- Money: it was built inside my Claude MAX subscription, with no API bill and no other paid services. Hosting is free on GitHub Pages.
- Time: Around 2 hours from start to finish.
Feedback welcome. Feel free to download the code and build on it.