# MusicGen *A theremin obsession, an Arduino prototype and computer vision, turned into a browser synthesizer with MIDI in and out.* - Year: 2024 - Built with: Vue.js, Vite, Tone.js, Three.js, Pinia, Tailwind - Updated: 2026-09-07T20:55:52.903Z - live: https://musicgen.abchaudary.me - repo: https://github.com/abdullahchaudary/musicgen > MusicGen turns a theremin obsession into six browser instruments built on Tone.js and Three.js, with MIDI in and out and camera-driven sound. ## What it is MusicGen is a browser instrument collection: six playable modes, full MIDI input and output, and two ways to generate sound with your hands, touch and a camera. Tone.js handles the audio graph, Three.js renders a visual response tied to whichever instrument is active, and none of it needs anything installed beyond the page itself. ## Where this started (origin story) The theremin is the reason this exists. It's an instrument played without touching it: two radio-frequency oscillators, one fixed, one wired to an antenna, and a player's hand acting as one plate of a capacitor with the antenna as the other. Moving a hand closer changes that capacitance, which detunes the antenna's oscillator by a small amount. Mix the two oscillators together and the difference between them, a "beat frequency," lands in the range a person can hear. Move your hand, the beat frequency shifts, the pitch changes, and none of it required contact. A second antenna does the same thing for volume. It is, quite literally, music made out of the capacitance of the air between a hand and a wire. I built a crude version with an Arduino to understand it properly, then decided a physical build was the wrong direction if the goal was letting other people try it: a circuit only I owned reaches exactly one person. A browser tab reaches anyone with a link, so I started building the same idea as a web instrument instead. ## Building the theremin The first version mapped a pointer's X and Y position to frequency and amplitude on a canvas, the same two axes a real theremin's two antennas control, so a position on screen is a pitch rather than a step on a scale. I added colour to those points next, then wired Tone.js underneath to actually generate the tone instead of just drawing it. `TouchTheremin.js` is where all of this lives, tracking every point by touch identifier rather than by count, up to ten simultaneous contacts, with every contact's own cleanup path so an ended touch can't leave a dangling oscillator running. ## A keyboard with velocity, in both directions Alongside the theremin sits a full synth keyboard, velocity-sensitive, and wired to the Web MIDI API in both directions: it accepts a real MIDI controller as input, translating note-on velocity straight into how hard a voice is struck, and it sends MIDI out too, so it can drive external hardware or another piece of software instead of only Tone.js's own synths. Neither the theremin nor the keyboard settles for one timbre. Both share the same effects chain, distortion, delay, reverb and chorus, plus ADSR and oscillator-type controls, all exposed as live parameters rather than fixed presets. ## Playing music with light Four of the six instrument modes generate sound from a camera feed instead of touch: a hidden video element samples pixel colour across a grid, and each cell's colour and brightness maps to both a Three.js object's position on screen and a Tone.js voice. Where the theremin turns hand position into pitch through capacitance, these modes turn light and colour in a room into the same thing through a camera. It's the same idea, computer vision standing in for the antenna. ## Stack Vue wraps the app and its routing; every instrument is its own class owning a Tone.js audio graph and a Three.js scene, so switching modes swaps both the sound and the visual at once. It installs as a PWA (`vite-plugin-pwa`), and the interface fades its own controls out while you play, so the instrument is the only thing left on screen. ## Outcome Six instruments, MIDI in both directions, ten-point touch and four camera-driven modes, built and pushed in four days. What started as a way to let more people try a theremin turned into something used well outside music: a friend of mine who designs textiles and fashion used this project as the basis for a set of interactive garments built for autistic children, work that earned him a distinction at university. I built an instrument. What it became was someone else's material. ## Links Live: [musicgen.abchaudary.me](https://musicgen.abchaudary.me). Source: [github.com/abdullahchaudary/musicgen](https://github.com/abdullahchaudary/musicgen). One of [the five browser experiments](/projects) on this domain built the same way: fast, for the sake of finding out.