RETINA RAVE

I taught a computer to listen to music.
This is what it sees.

Hi, I'm Toma. I like music and math and built this for fun.

Retina Rave is a music visualizer that runs in your browser. It listens to whatever you are playing and steers a camera through real mathematical objects: the shapes on screen are not drawn to look mathematical, they are the mathematics, computed live. Beats, drops and sections are detected as they happen, and a director picks the scene, remembers what each part of the track looked like, and crossfades between them. Nothing is random and nothing runs on a clock: every motion on screen traces back to the music.

The scenes

Keys 1–6 force one; the arrow keys step forward and back; 0 hands it back to the director.

Support

It is free and there are no ads. If it made your night better, a tip helps cover hosting and adding new scenes.

Venmo @toma-kraft Cash App $toma5

Follow

Clips and new scenes land here first. What changed in each release, in plain words, is on what's new.

Instagram @retinarave YouTube @retinarave

How to use it

On a computer: play music in another browser tab, open retinarave.com, press Share a tab and pick that tab with its audio. Press ? at any time and the machine explains what you are looking at and what is moving it. On a phone: put the music on a speaker and let the page listen with the microphone.

Firefox and Safari cannot share a tab's audio — their share dialog offers windows and screens only, and the sound is left out (Mozilla bug 1541425, open since 2019). The page notices and offers the two ways in that work there: press Listen with the microphone and, in the browser's prompt, pick the device that carries what the computer is playing — Monitor of … on Linux, Stereo Mix on Windows where the sound driver offers it, a loopback driver such as BlackHole on a Mac — or play a file from this device. Chrome, Edge, Brave and the other Chromium browsers share tab audio directly.

How it works and what it does not do

Everything runs on this device. The audio is analysed inside your browser tab, frame by frame, and the picture is drawn by your own graphics card; nothing you play is sent anywhere, because there is no server on the other end to send it to — this site is a static page. When you press Share a tab, the dialog that opens is Chrome's own, and the page only ever receives the one tab you tick, with its audio, for as long as you leave it shared. The microphone works the same way: the sound stays in the page. A file you drop on the card is decoded in the page and never uploaded.

There are no accounts, no cookies, no analytics, no trackers and no ads. Nothing from a third party loads on these pages until you press play on a clip on what's new, and the recording below is the same: made on your device, never uploaded. Close the tab and there is nothing left behind.

Why? I built this for myself — I like music and math — and wanted to share it. Free and private is the point, not a phase.

Recording

Press R while it runs and R again to stop: what you saw and heard is saved as a video file on your own device, through your browser's normal download, with a small @retinarave mark in the corner. The recording is made on your computer and stays there; nothing is uploaded, there is no upload or share button, and this site never receives it. The music in your clip is the music you were playing, and the rights to it are yours to clear before you post it anywhere.

License

The code is open: MIT, with one clause of my own — if Retina Rave, or anything built from it, runs at a party, a show, a festival or any other event, I get in free, with a guest. That is the whole deal; it is written up as the Retina Rave License and sits at the top of every file this site sends you. For the curious, the whole thing — the engine, the scenes, the engineering notes — is public at github.com/tbkraf08/RetinaRave.

The fluid under every scene is a re-implementation of Pavel Dobryakov's WebGL-Fluid-Simulation (MIT, 2017); its notice is at /THIRD-PARTY.md. Ideas from the forks that map music onto it are credited in the engineering notes.

The mathematics NAV3's drop is built on is not mine: the parabolic root points of the Mandelbrot set and their internal angles are Douady and Hubbard's (Étude dynamique des polynômes complexes, 1984–85), the implosion of a Julia set at such a point and its phase are Pierre Lavaurs' (thesis, Orsay 1989), the Fatou coordinates and the horn map at the parabolic point follow Mitsuhiro Shishikura (Bifurcation of parabolic fixed points, in The Mandelbrot set, theme and variations, LMS 274, 2000), and the horn map's translation constant as written here is Inou and Mukherjee's (Non-landing parameter rays of the multicorns, Invent. Math. 204, 2016). Nothing of theirs is copied; the formulas are re-derived in the scene's own code and credited there, in the README and in /THIRD-PARTY.md.

The ears' vocal detector — the small net that hears whether someone is singing, on your device like everything else here — was trained on the Jamendo Corpus for Singing Voice Detection (Ramona, Richard & David, 2008; CC BY 4.0), 93 Creative-Commons pieces from jamendo.com with voice / no-voice annotations, and — so a solo piano or a string quartet stops reading as a voice — on MusicNet (Thickstun, Harchaoui & Kakade, 2017; CC BY 4.0) and MAESTRO (Hawthorne et al., 2019; CC BY-NC-SA 4.0 — this site is non-commercial) as no-voice examples. The weights ship with the engine; the music does not. The notices are in /THIRD-PARTY.md.

▶ open the show

Retina Rave · a zero-dependency WebGL2 engine, native modules, no framework.