RETINA RAVE

What's new.
Every release since the site went public, newest first.

v0.46 v0.44 v0.43 v0.42 v0.41 v0.40 v0.39 v0.38 v0.37 v0.36 v0.35 v0.34 v0.33 v0.32 v0.31 v0.30 v0.29 v0.28 v0.27 v0.26 v0.25 v0.24 v0.23 v0.22 v0.21 v0.20 v0.19 v0.18 v0.17.1 v0.17 v0.16 v0.15 v0.14 v0.13 v0.12.1 v0.12 v0.11 v0.10 v0.9 v0.8.1 v0.8 v0.7 v0.6

Three kinds of change: a new scene (or one rebuilt), something the engine learned to hear or show, and tuning — looks retuned, thresholds moved. Everything here runs on your device; nothing on this page phones home.

v0.46 new scene

A new scene: the pool inside the dancing set, stirred by its own edge

There is a new scene on key 8, NAVPOOL, and like the seventh it is one you have to ask for — the director never picks it. It is the navigator from key 7, the same set rolling, dancing and coming apart on a drop, with a pool of ink inside it. The edge of the set is the wall of the pool: the fluid lives only inside the filled shape, and when the shape moves, the wall moves with it. The first version stirred the pool straight from the music — every beat threw small whirlpools into the water, and the result curled and swirled far too much, tight crinkly spirals that had more to do with the drums than with the shape. That is gone entirely, not turned down. The music still drops ink into the pool along the path the set's own centre traces, a drop still throws a ring of ink outward and a kick still shoves at the wall, but nothing else in the audio is allowed to spin the water. What moves it now is the edge, and the edge works like a paddle: when the set turns, grows or shrinks, the fluid next to the wall is pushed the way the wall is going, and it keeps moving after the wall stops, coasting and slowly settling the way water does when you stop stirring. The wall is slippery rather than sticky, so the flow runs along it instead of being dragged to a halt. What you see is smooth streamers that follow the lobes and turn with the set; after a fast spin the band next to the rim rolls up into a ring of small eddies that fade away. The dance is the seventh scene's, exactly. The pool scene had been built on a draft of the navigator that never shipped, one where a lift in a percussive song only pressed the ball out to the edge and waited for a slam that, on this engine, never comes — so in the pool the set often never came apart at all. Replayed against four songs, the two scenes now move identically frame for frame once the music is playing, and every come-apart lands on the same frame in both. From here on they are tied together: a change to how the seventh scene moves has to be carried into this one too, or the tests fail. The honest limits. With the music no longer stirring it, the ink near the rim is thinner on real songs than it was. While a song is just starting the pool scene holds its dance still until it hears music, so on a song with a long quiet opening the two scenes can sit a hair apart for the first few seconds.

for the curious, the engineering notes: §149 · §151 · §152

v0.44 new scene

The quiet song: a drop that waits, and one that swells

The seventh scene — the navigator on key 7, the one the director never picks and you have to ask for — got four more passes, and every one of them came out of a single song: a slow solo piano piece that the engine had been reading completely wrong. First the scene itself learned to be quiet. In silence the ball now rolls back to the plain circle at the heart of the shape and the scene's own clock stops, so the rays and spokes stand still instead of spinning through a rest. The orbit dots that made the set look like a scatter of little circles only appear when there is a map on screen and the ball is away from the edge; the petal layer stays inside the petals; a hit can no longer throw the ball past the wall; and the scene only crosses into a smaller bulb when the music is genuinely building, not whenever it drifts near a gateway. Then the engine. This one is not the new scene's — every scene on the page inherits it. A piano has no sub-bass, and the drop detector had been arming on exactly that: it counted a bass that was never there as having dropped out, and then fired on the next loud chord. A slam now has to show the sub back and the loudness ladder standing at a drop's level before it is allowed through, and the second detector, the one that reads the map, goes through the same test. Across the nine songs in the library that is five false slams gone to none, fifteen false reads down to five, and not one real drop lost — a few of them land a fraction of a second later than they used to, which is the sub coming back. In its place there is something gentler: an energy shift, a sustained lift that is not a slam, is now a field any scene can read, and the new navigator holds its drop picture up for as long as the energy stays up rather than for a fixed eight beats. On the piano piece that is the difference between a picture at 47 seconds that should never have happened and one at 52 that holds to 1:07, which is what the music does. Small thing, same release: the sound now starts on your first click or key press, instead of staying silent on a freshly opened tab. The third pass was the look. The held interior goes black with the petals only on the way in, the depth of the come-apart is calibrated per drop so it eats visibly on a quiet song and still has somewhere to go on a loud one, the direction sweeps around the wedge over four bars instead of sitting still, and the whole dance is scaled by how percussive the music actually is — on a piano, barely at all. The last pass is the one to watch for. On a delicate piece an energy shift no longer explodes anything: it SWELLS. The ball runs out to the rim and through the gateway into a smaller bulb, the bands rise, the picture opens — and nothing comes apart. The piano piece went from two implosions to none and three swells. The tiling inside the set, which had been reading as a stack of hard rings, now fades out with depth to a floor that is never quite black. The honest limits, as always. Nobody but me has watched any of these four passes — the scene still needs work and it is on a key of its own for that reason. The swell draws nothing of its own yet beyond the crossing and the lift. One song's intro sits right on the threshold between swelling and imploding. The escape-time colour uses one number for every gateway. The alternative colour mapping carries none of this. And the site itself has been sitting several tags behind: everything since the singing detector went in arrives here at once.

for the curious, the engineering notes: §142 · §143 · §144 · §145

v0.43 new scene

The navigator that rolls, and a drop that implodes

There is a seventh scene on the page now, on key 7, and the director never picks it — you have to ask for it. It is a copy of the navigator the page opens on, with its whole way of moving thrown out and rebuilt. The navigator you already know walks a hand-drawn map: a short list of interesting places inside the shape, worked out in advance by me, and a route between them. This one carries no map at all. Wherever it happens to be sitting, the shape itself hands it a coordinate — how tightly the orbit there spirals in, and which way it turns — and the music steers in that coordinate directly, which means it can be anywhere in the set and still know which way is out. Loud presses it toward the edge of the lobe it is in, so the spiral tightens as the track opens up; the sung note, or the brightness of the mix when nobody is singing, turns it around the rim; the harmony and the key turn it further; the bass and the kick push it outward on every hit; a section change mirrors the whole picture top to bottom. When the music builds, it slides along the rim until it is sitting on a gateway — the pinch where a smaller bulb is budded onto the one it is in — and crosses through into the smaller one; when the energy falls away again it rolls back out the way it came in. Nothing there is looked up in a table. The shape is asked, every frame, where its own edges and gateways are. The drop goes through the gateway instead of around it. At the pinch itself the orbit neither escapes nor settles — it crawls, and the set's interior opens into petals, which the scene draws as bands of the time an orbit spends before it leaves. On the drop the scene walks to that exact point, holds there while the petals come up, and then steps just outside, into the hair-thin wedge between the two lobes, where the filled set is no longer solid: it comes apart from the inside out, and the dust eats inward as the step grows with the slam and fills back in as it shrinks. The distance breathes with the beat, and how far out it goes is how hard the drop hit. Then it eases back and the ordinary rolling picture returns. Three rounds of retuning came out of watching it on a big screen. It was alive but ten times quieter than the navigator it was cloned from, so it now lives against the rim rather than in the middle, with springs and gains set to a beat rather than a bar, and a grammar of its own: the bass and the kick pulse the spiral's tightness, the lobes turn a little every bar and sway across a phrase so the outline is never the same two bars running, and the snare twitches it from side to side. It was spending too much of the machine on the implosion and the picture went soft during drops, so the implosion's budget now follows the same quality governor as everything else, and the glow that was washing the shape out has been pulled back to the very rim — the boundary and its spiral arms read against a dark interior where before there was a lit blob. And it was wandering too deep: far enough down a chain of bulbs, the set stops looking like the set and becomes a rosette of little circles that barely changes. It now stays home, in the main body and the bulbs attached directly to it, holds a smaller bulb for a couple of bars once the music stops asking for it, and never pins itself flat against the wall. Being honest about it. Nobody has watched this but me, and the eye it has to convince is at home in front of the big screen — that is the whole reason it is on a key of its own and out of the rotation. On real tracks the implosion eats in much more thinly than it does on my test timeline, and the knob for that has not been set per song. Two songs in my library have a drop that the engine does not raise inside the window I measured, so they never implode at all and I do not yet know which of the two detectors is at fault. During a drop this scene costs several times what the home navigator costs, so on a weaker machine the governor will soften the picture for those few seconds. And one song still washes out to a flat blob at the deepest setting it is allowed to reach. The home navigator, on key 3, has not changed by a single byte.

for the curious, the engineering notes: §140

v0.42 new scene

The cloud follows the singer

Watching the swarm of grains on key 4 with a song I know well, I could not see the singing in it. The kick, the snare and the hat each had a voice in that picture and the singer had a hairline: a quarter of the light moved onto a ring so thin that under a real drum kit you would never find it, and the one thing it did with the melody was slide that ring in and out by less than its own width. The detector was not at fault, and I measured that before I touched anything — on a song the network had never heard it lit through all of the singing and almost none of the rest, and named the note within a semitone nearly every time. The signal was arriving clean and the scene was spending it on nothing. Now the register the singer is in is a band rather than a hairline, lit in the colour of the note being sung and drawn with bigger grains, and the rest of the cloud — the drums' flashes included — gives up nearly half its light to it while a voice is sounding, then takes it all back within a breath of the voice stopping. The drums recede under a held line instead of shouting over it, and the moment the singing stops they are as bright as they ever were. The note is also a height now: the lit band floats up out of the body of the cloud on a high note and sinks under it on a low one, carrying the whole swarm a little with it, in every formation the scene has. The drum grammar underneath is untouched — not one kick, ring or hat moved — this is only a question of who gets the light while somebody sings. One old mistake came out in the washing. The view this engine draws through is mirrored top to bottom, which nothing on screen could ever tell you, and the one place it mattered was the rise: the line the last release lifted for a sung note had been lifting it for a low note and pushing it down for a high one. Both rises are the right way up from here. There is a ninth song in the library, a rock record from the seventies with a real singer over a real band, and it is the first one I added after the voice detector had finished learning — it was deliberately kept out of that learning, so what it says is the only word I have from a song the network cannot have memorised. It says the detector is good. It found the singing wherever the separated vocal track said there was singing, it called the note right nearly every time, and over the whole song it claimed a voice almost nowhere there wasn't one — less often, in fact, than on the solo piano I keep as a control. The song has the widest quiet-to-loud range of anything in the library and a long stretch in the middle with no voice at all, which is exactly the material this scene needed to be judged on. One caveat on it: the engine reads its pulse at double the speed you would nod your head to, and until I have sat and listened to that properly the song is in the library for watching rather than for measuring against. Being honest about the rest. A solo piano sitting in a singer's range can still make the cloud flicker a little; it is a fraction of what it was two releases ago and it is not nothing. A chopped, shouted sample still reads as a voice with no note in it, so the light it makes has no colour. On one song the lit line through the swarm is bright enough to own the picture by itself. And the seam that puts the whole swarm on a line is still a once-a-section decision rather than something a voice can open on its own — on a song that is sung most of the way through, letting a voice open it would throw away the twenty formations the scene has for the sections of a track. So far the eye on all of this is mine; the one it has to convince is at home in front of a big screen.

for the curious, the engineering notes: §137 · §138

v0.41 engine

The voice finds the note, and the ink rests

The detector that listens for singing was rebuilt, and the thing it got better at is the one it was worst at: the note. Before this release it answered a window at a time and guessed the pitch off the whole mix, and on two of the three sung tracks in my library the note it named was wrong more often than it was right — which means the colour a sung line painted was mostly a colour the song was not in. It now answers on every step rather than every window, and the pitch comes from the network itself instead of being chased afterwards. On the tracks with a clear lead vocal it lands on the right note nearly always, and on the hardest of the three — a shouted sample over a synth — it still lands where it used to. You see it in the two scenes that read the voice: on key 4 the lit seam through the swarm of grains takes the colour of the note being sung, and on key 5 so does the strand lifted out of the nest. Hearing a sung line in the right colour is the whole of what changed there. Being honest about the rest of it. The voice still enters a beat behind the note at a walking tempo, and a shade later than it did in the last release — that is the trade for a network that hears further back before it commits, and it is the gate I most wanted to meet and did not. A solo piano sitting in a singer's range still lights the scenes a little; it is a fraction of what it was two releases ago, and it is not nothing. A chopped, shouted sample still reads as a voice being present with no note in it, so the light it makes has no colour. And the network still knows my own library far better than it knows music it has never heard. The second pool of ink, on key 2, has gone back to the reading I kept returning to. The rig of invisible mouths I built for it last release is gone and the pool is fed the way the first one is, which is the version I had on a recording from a week before and could not stop preferring; it is in the page exactly as it was that afternoon, proved frame by frame against the old file rather than rebuilt from memory. One thing goes with that: the pool no longer reads the voice at all, because reading it was the rig's own idea. The grains and the nest carry the voice now, and the first pool never did. Last, the thing that had been quietly taking frames from everything else. The pool of ink is a fluid simulation, and it was being run under every scene on the page whether anything on screen was drawing from it or not — on a machine with graphics power to spare you would never notice, and on one without it that is a real slice of every single frame spent for nobody. It now runs only while a scene that actually draws from it is on screen. On the slowest renderer I have, the dust and the navigator each got better than a tenth of their frame rate back, and that is the dropped frames I went looking for. There is one visible consequence and it is the right one: when a pool scene comes in now, it fills from dark instead of fading up the picture the pool happened to be holding from minutes ago.

for the curious, the engineering notes: §132 · §133 · §134 · §135

v0.40 engine

The listening moves to its own thread

Everything the page does to hear the music — the drums, the bass, the key it is in, the texture, the pulse and the voice detector that arrived in the last two releases — now runs on a thread of its own, beside the one that draws. Nothing a scene is handed changed, and I proved that rather than claimed it: the old arrangement and the new one were run side by side on the same sound, on every track in my library, and every value a scene reads was compared one at a time, exactly, never within a tolerance — not one of them differed, and the shot-by-shot proofs that pin each scene's pixels all came back the same to the byte. What changed is that the thread that draws no longer stops to listen. Watching a shared tab, which is how I watch nearly all of the time, drawing gets back almost all of the time the listening used to take from it, and more again when the voice detector is doing its work too. On this machine you will not see the detail level move, because that time was margin rather than a step; on a machine nearer its limit it is the difference between a frame arriving and a frame arriving late, and it is the same margin the softness I went looking for last release came out of. Playing a file the way the test harness plays one keeps the listening where it was, because there the sound is handed in and read back inside the same frame, so every pixel proof still runs the path it always ran. If the new thread will not start, the page falls back to the old arrangement whole, with nothing missing. The readout in the corner says which thread is listening and what it costs, and a switch in the address bar forces the old one, so the two can be watched one after the other on the same music — and the new one is the one I would keep. Being honest about what moved and what did not. One thing moves: the part that follows the beat is now told about a kick a hop of the thread after the ears found it, which on everything I measured shifts the tempo it reports, and the place it puts the beat, by less than an eye reads on a beat line — but it is a shift, and it exists only while listening live; the kick itself still carries the moment it happened. And one old limit is worth naming because it now looks like a new one: a single saved page opened straight off the disk cannot start this browser's audio plumbing at all, which was already true of the page before this one — served over the network, the way the site is, it is the real thing.

for the curious, the engineering notes: §131

v0.39 engine

The voice reaches the scenes

The singing the engine learned to hear last time now reaches two more of the scenes, and the detector itself had a second pass. On key 4, the swarm of grains lights where the voice is: the grains carrying the sung register brighten and take the note's colour, so a held line draws a lit seam through the cloud. On key 5, the nest of circles grows a strand — one fibre lifted out of the bundle, its height set by the pitch, its hue the note being sung, its brightness the voice against the rest of the mix. Both go quiet the moment nobody is singing, and each has a switch in the address bar that turns the read off, so the before and the after can be watched one after the other on the same music. The detector itself was retrained. Its old failing was a confident false alarm: a solo piano or a lead synth sitting in the voice's range read as singing, and on a piano recording it had never heard it lit up on better than a quarter of the frames. It now lights on about one frame in fourteen there, and the other instrumental controls fell with it — a track of synth leads from a seventh of its frames to a twenty-fifth, a drum track to under one in a hundred. That was bought with caution: it also misses more of the quiet singing than it did, about a tenth of what it used to catch, and that is the trade I would rather have — a scene that stays dark through a piano solo is honest, a scene that pulses at a piano is a lie. It is still a small network hearing a third of a second at a time; it still answers about an eighth of a second behind the note, and on music it has never heard it is right about four times in five. A shouted, chopped vocal sample reads as presence with no pitch, so the light it makes has no colour. One bug went with all this, and it turned out to be the camera rather than the sound: on key 5 the voice was quietly nudging the frame on every sung note, which on a vocal line reads as a pump. The voice is now read after the camera has decided where it is looking, so the strand lights and the nest sits still. Nothing is fetched and nothing leaves your device: the network's numbers are part of the page, as before. Two quieter fixes went in with it, both from asking why the picture had gone soft. The part of the engine that decides how much detail to draw was reading the pause while a track is read off the disk as the machine struggling, and then spent the next half-minute climbing back out of a hole that was never there; it now waits out a load and carries on from where it stood, so a scene opens at the detail it was already drawing. And when a file's vocal part was separated beforehand and is the thing feeding the picture, the small network is no longer asked for an answer that is thrown away on the same frame — the only listening being done is the listening you can see.

for the curious, the engineering notes: §126 · §127 · §128 · §130

v0.38 engine

The engine learns to hear a voice

Retina Rave now listens for singing. A small neural network — about forty thousand numbers, trained on a public corpus of Creative-Commons music annotated for voice and no voice, and shipped inside the page itself — reads the sound as it arrives and answers three questions many times a second: is someone singing, how loud against the rest of the mix, and on what note. The second pool of ink, on key 2, uses all three, and it is a different scene from the one that shipped last time. The band grammar I put there a release ago moved too fast to watch, so it is gone; in its place the pool is fed by a rig of invisible mouths — twenty-three of them, lining the two sides and the top and set symmetrically inside the frame, the way a lighting designer hangs a truss rather than aiming one lamp at the middle. Each part of the music gets its own mouths, and a run across the rig reads as a chase. The voice is now the lead channel: the mouth that draws the melody rides on the singer instead of on a metronome, its height is the pitch — high notes high on the screen — and its colour is the note being sung, taken around the circle of fifths like every other harmony colour here. When nobody is singing, the lead goes quiet rather than inventing a line. The drop no longer throws a shockwave across this pool; the rig answers it instead. The listening costs the picture nothing — the detector runs beside the rest of the ears, in well under a tenth of a millisecond per step, and the weights are part of the page, so nothing is fetched and nothing leaves your device. The on-device promise on the landing page is unchanged. Being honest about what it is: the network is small and hears only a third of a second at a time, so it is quick rather than clever. It knows the eight tracks in my own library well; on music it has never heard, a solo piano or a lead synth sitting in the voice's range can read as a voice on a fifth of the frames, and a pitch taken off a full mix lands within a semitone about three times in four when there is a clear lead vocal, less when the voice is a chopped sample riding over a synth. There is a second, more accurate route that only works on my own machine: playing a file, the page can read a vocal part that was separated offline beforehand, which hears the voice on its own with no mix to confuse it, and a switch in the address bar flips between the two so the difference is visible. A shared tab is always the live network — which is how I watch almost all of the time, and why it was built first. The little readout in the corner says which of the two is feeding the picture.

for the curious, the engineering notes: §120 · §121 · §122 · §123 · §124

v0.37 new scene

Six scenes, six keys, and the arrows

The scene list is down to the six I actually watch, and they sit on the first six number keys: 1 the pool of ink, 2 a second pool that reads the bands of the music, 3 the navigator the page opens on, 4 the dust, 5 the torus, 6 the mandala. The arrow keys now step through them in both directions — right is forward, the same as n, and left goes back — which is what the two buttons in the bottom bar have always done. Eight scenes I never looked at have left the page: the polytopes, the ladder, the first torus, the two older navigators, the waves, the superformula and the sand plate. Nothing was thrown away — they are all still in the repository, frozen at the version they last ran in — and the page that loads is a quarter smaller without them. The second pool, on key 2, is new and experimental: where the first one takes every sound as a splash of ink, this one gives each band of the music its own kind of coupling. The sub-bass is a field that heaves the whole pool instead of a dot, every kick launches a pair of spinning vortices that travel across the screen, the mids decide how stiff the ink is, the highs are the surface — a grain, a dip, a shine — the harmony draws with a pen, and the simulation’s own clock breathes with the sub, drops to half speed in the hollow before a drop and double-steps when it lands. It was tuned on one track, the newest in my library, rather than on all eight, so treat it as a sketch rather than a finished scene: the director never picks it, you have to. And sharing a tab now opens on the pool of ink rather than the navigator, and stays there while it settles.

for the curious, the engineering notes: §116 · §117 · §118

v0.36 new scene

The drop as a shockwave — and FLUID joins the rotation

Two things for FLUID, the pool of ink. First, the drop is now a shockwave: on the beat the drop lands, a ring of velocity leaves the centre of the screen and travels outward at the song’s own beat speed, carrying the ink with it — the centre empties physically, the ink piles up at the front, and the drop’s kicks refill the middle over the next beat. How hard it throws is how hard the bass slams back, ranked against the track’s own kicks, so a soft drop is a softer throw. It fires only on a confirmed drop — the bass has to return — so a breakdown that teases never fires it. It works the same from a file and from a live tab capture; on a live night the only difference is the ear that finds the drop. Second, FLUID now takes part in the director’s rotation: it bids for sub-driven passages under a sure key in a dark mix, next to the navigator, the particles and the torus, and stays at least 30 to 90 seconds like every scene. Nothing else moved — every other scene renders exactly as before, frame for frame, and n still cycles to FLUID whenever you want it.

for the curious, the engineering notes: §113 · §114

v0.35 new scene

A pool of ink under every scene — FLUID

There is now a fluid under every scene: a pool of ink that the music stirs, simulated on the graphics card every frame, built after Pavel Dobryakov’s WebGL-Fluid-Simulation (MIT, github.com/PavelDoGreat/WebGL-Fluid-Simulation). The new scene FLUID shows that pool as itself, lit like oil on water — press n to cycle to it, or open the page with #scene=12. Where the ink enters is the note: the bass sits at the song’s key, its fifth to one side, its fourth to the other, the odd notes out at the walls. How hard the music hits is how big the splash: every kick lifts the ink, the snare shears it, the hats sprinkle it, and the loud bars throw more than the quiet ones. The song’s key is the colour, and it stays put once the ears are sure of it. The drop clears the pool — a beat of empty water, then the drop’s kicks refill it. Pads and chords keep a faint floor of ink moving so a quiet passage is never a black screen, sized to the track’s own range. All of this was tuned on every track in the library, one at a time. The other scenes are unchanged: the fluid only reaches a scene that asks for it. And the glitch rows — the screen tear on a drop — can be switched off: press G, or open the page with #glitch=0 for a bookmark that stays off.

for the curious, the engineering notes: §104 · §105 · §106 · §107 · §108 · §109 · §110 · §111

v0.34 new scene

NAV moves with the music

The home scene, NAV, was rebuilt to move with the music. The picture no longer washes out to white at the loudest bars: the brightest moments are now the most detailed, with a dark interior and a bright rim, and white is spent on the snare and the drop. The colours sit on the song’s key — minor keys land on green — and step once per phrase instead of drifting around the wheel. The set jumps on the real kicks, about once a bar, breathes with the beat, presses to the rim when the sub-bass is in and relaxes when it drops out, leans with the bass note, and the exterior filaments turn once per bar instead of twice a beat. At each phrase it walks to a new part of the set instead of sitting in one place for a whole section, preferring busier shapes when the music is loud and calmer ones when it is quiet, and every drop lands on a new ray. The kaleidoscope mirror is off on NAV, so the set itself is what you see. The previous navigator is still a key press away on 9 for anyone who prefers it.

for the curious, the engineering notes: §97 · §98 · §99 · §100 · §101 · §102

v0.33 engine

Firefox and Safari find their way in

Firefox and Safari cannot share a tab's audio — their share dialog offers windows and screens only, and the sound is left out. The page now notices and shows those browsers the two ways in that work: listen with the microphone, picking the device that carries what the computer is playing (“Monitor of …” on Linux, “Stereo Mix” on Windows, a loopback device on a Mac), or play a file from this device. The message after a share with no sound says the same instead of a shrug. Chrome and the other Chromium browsers are unchanged.

for the curious, the engineering notes: §96

v0.32 engine

A slower director, and sharper recordings

The director now moves between three scenes — NAV, DUST and TORUS — and lets each one play for at least thirty seconds to a minute and a half before it looks for a change, so a scene has time to settle into the music instead of flipping every few bars. Drops still cut back to NAV at once and a build still parks there, waiting. The other scenes are still a key press away. A forced scene no longer sticks across a reload: open the page and the director is in charge again. Recordings made with R keep the full picture quality for the whole take and come down as one file with their notes inside.

for the curious, the engineering notes: §92 · §93 · §94 · §95

v0.31 engine

A what's-new page, and a line about what this is

Retina Rave now has a what's-new page: one short entry per version, newest first, going back to the first public build, written for someone who watches rather than reads code. The front card says plainly what this is: everything runs on your device, with no account, no trackers and no ads; I built it for myself and wanted to share it. A small line under it points at what changed in the latest version. The about page grew a longer explanation of how it works and what it will never do. Nothing in the visuals or the sound analysis changed in this release.

for the curious, the engineering notes: §91

v0.30 engine

Sure of the key, a kaleidoscope on the beat, and a record button

The colour now follows the song's key only when the ears are sure of it: the bass has to agree with the harmony before the palette commits, so a wrong guess no longer flips the hue mid-track. MANDALA has been rebuilt from the ground up: the kaleidoscope turns one step per beat, kick, snare and hat each have their own voice in it, and the stretch before a drop tightens the fold and drains the colour until the drop lets go. Press R to record what you see and hear to a video file on your own device, and R again to stop; nothing is uploaded and there is no share button. DUST stays in step on tracks where the vocals carry the beat, and the code is now open under MIT with one clause of my own: if it runs at a party, I get in free, with a guest.

for the curious, the engineering notes: §83 · §84 · §85 · §86 · §87 · §88 · §89 · §90

v0.29 engine

Colour follows the chord

Major bars pull the palette warm, minor bars leave it alone, so the hue breathes with the chord changes instead of sitting on one key for the whole track. The tension field is now measured against the track itself over the last half minute, so a quiet intro and a wall of noise both use the whole range and a drop reads as a drop on every song.

for the curious, the engineering notes: §81 · §82

v0.28 tuning

A haze before the drop

In TORUS2 the dreamy stretch before a drop now hazes the nest of rings, and the drop clears the fog in one go. DUST's hi-hat voice joins the double-time accent: when the rhythm doubles, the rim flicks harder rather than faster.

for the curious, the engineering notes: §79 · §80

v0.27 engine

Everything since v0.15 goes live

The first update to reach the site in two weeks: every change from v0.16 on arrives at once. New in this one: the engine notices when the music hesitates on a beat it will not commit to, and treats it as a build-up; DUST's double time accents the picture instead of driving it faster; CHLADNI's figure follows the bass from the very first seconds of a track instead of waiting for a sub note that is not there.

for the curious, the engineering notes: §75 · §76 · §77 · §78

v0.26 tuning

Thresholds measured against the right number

A statistic deep in the ears had its sign backwards, so several thresholds had been calibrated against the wrong end of the distribution; they have been re-based and hits land more honestly. CHLADNI now throws its sand by how hard the kick hit, and one of the reference tracks had its beat grid re-aligned by a few hundredths of a second.

for the curious, the engineering notes: §72 · §73 · §74

v0.25 engine

Every hit has a size

A kick and a snare now carry how hard they were hit, so DUST's flash ring and TORUS2's wave scale with the hit instead of firing at one size. The scenes' snare is the snare lane alone, so a hi-hat is no longer mistaken for one.

for the curious, the engineering notes: §70 · §71

v0.24 engine

The snare the ears could not hear

The snare and the rim are now heard as a rise in the two middle bands, so a rim shot that used to vanish is seen and a hi-hat that used to be called a snare is not.

for the curious, the engineering notes: §69

v0.23 engine

True loudness, the right key, and a smoother beat

The beat's nudge is now a smooth velocity profile, so the picture bounces instead of jerking, and the clock no longer wanders off a live tempo. The key the colours follow is the song's tonic: until now the engine had been reading the fifth above it, so every key-coloured scene was a fifth off. Loudness is measured the way broadcast meters do it, so a drop that comes back louder after a quiet breakdown actually reads louder on screen. The kick that hides under a bass note is found, the hi-hat fires cleanly on sparkly passages, and the quiet void before a drop is drawn as one.

for the curious, the engineering notes: §61 · §62 · §63 · §64 · §65 · §66 · §67 · §68

v0.22 engine

DUST pass 2, and the half beat

In DUST a grain's frequency band is now its place in the formation, a repeated sound fades while a new one pops, and the harmony shows in the colour. The beat clock lets the low end decide where the half beat falls, which brings a fast track that had drifted half a beat off back onto the kick.

for the curious, the engineering notes: §58 · §59 · §60

v0.21 new scene

DUST rebuilt: read the song with the sound off

DUST has been overhauled against one standard: someone with the sound off should be able to roughly reconstruct the song from the picture. The swarm moves on the beat grid, the kick, snare and hat each have their own sharp voice, the stretch before a drop pulls the formation tight and the drop releases it, and each section of the track has its own formation.

for the curious, the engineering notes: §57

v0.20 engine

The beat clock moves onto the raw samples

Tempo and beat phase are now worked out from the audio samples themselves, on the audio's own clock, rather than once per video frame, and filtered so one odd hit cannot yank the beat line. This is the new default: the beat holds steadier through a build-up and lands on the drop.

for the curious, the engineering notes: §56

v0.19 engine

What comes next, in one place

The engine keeps one queue of the events it expects: the hits the bar pattern predicts, the next bar line, the drop it sees coming. Scenes read what comes next from one place and can get there in time.

for the curious, the engineering notes: §55

v0.18 engine

Seeing the drop coming, live

A build-up and the drop at its end are now detected live, from a shared tab, without knowing the track in advance; NAV rides it by default and leans into the edge of the set as the drop approaches. The small head start the picture takes on the sound applies to files only: live, your eye wants it exactly on time.

for the curious, the engineering notes: §53 · §54

v0.17.1 tuning

Forty milliseconds early

When you play a file, the picture now runs a hair ahead of the sound, because the eye wants to see a hit a touch before the ear hears it. TORUS2 moves onto the new reactive drums.

for the curious, the engineering notes: §52

v0.17 engine

The bar remembers

The engine fingerprints each bar as it goes by, so it knows a section coming back when it starts and can predict where the next bar's hits will fall. The kick, the snare and the hi-hat are heard as three separate lanes, so a scene can give each one its own move.

for the curious, the engineering notes: §50 · §51

v0.16 engine

Live mode, measured

The delay between a sound in a shared tab and the moment the page hears it has been measured, and the clocks now run ahead by that much, so what you see lands on what you hear.

for the curious, the engineering notes: §49

v0.15 new scene

CHLADNI, and the engine grows ears

A new scene: sand on a vibrating plate. The bass note draws the figure, every kick throws the sand into the air, a slide melts one figure into the next, and when the bass stops the sand floats free. You can also play a file from your own device now, decoded in the page and never uploaded. Under it all the engine has new ears: a sharper read of onsets, the bass note and the song's structure, tuned on a dubstep track with a map of its drops.

for the curious, the engineering notes: §48

v0.14 new scene

GIELIS: twelve shapes that breathe

A new scene built on the superformula: twelve nested shapes, one per note, that pinch shut on every beat and open again in the silence. Each kick sends a bump travelling round every ring, the whole nest turns once every sixteen beats, and it fills the screen as the track builds.

for the curious, the engineering notes: §47

v0.13 tuning

NAV2 bumps with the beat

The Mandelbrot set now bumps with the beat: with no beat it relaxes towards a circle, and as the beat comes in it spirals into its own detail. The edge of the set is always moving with the music.

for the curious, the engineering notes: §46

v0.12.1 tuning

Thin waves and fat waves

In MAXWELL the bass sends out fat waves and the highs skinny ones, and a quiet breakdown keeps rippling instead of going dark for seven seconds.

for the curious, the engineering notes: §45

v0.12 new scene

MAXWELL: every sound a wave

Every sound now sends out its own wave from the centre, in the colour of the note being played, so double time looks like double time. The constant carrier is gone, and so is the lattice of small circles in the background.

for the curious, the engineering notes: §44

v0.11 tuning

The wave remembers its note

MAXWELL after its first viewing: quiet when there is no sound, each wave in the colour of its own note instead of two colours that swap, and a wobble that follows the bass so dubstep looks like dubstep.

for the curious, the engineering notes: §43

v0.10 new scene

MAXWELL: the four equations that dance

A new scene: Maxwell's equations solved live on a grid. Twelve charges sit at the twelve notes, a magnet turns one notch a beat, and every drum hit is a real ripple of light. The N key cycles through the scenes.

for the curious, the engineering notes: §42

v0.9 new scene

POLYTOPE: the cage dances

The four-dimensional polytopes dance: the bass shoves the cage round, the rhythm runs in beads along its edges, and the twelve keys are its colours on the circle of fifths.

for the curious, the engineering notes: §41

v0.8.1 engine

This is what it sees

The front page now says what this is: I taught a computer to listen to music, and this is what it sees. Below it, one tile per scene — click one and it plays live behind the card before you share anything.

for the curious, the engineering notes: §40

v0.8 new scene

NAV2: the melody draws the path

A second walk over the Mandelbrot set where the melody drives the camera: when the tune goes up the whole shape floats up with it, a swirl in the music curls the spiral arms tighter, and a build presses the picture against the very edge of the set until the drop breaks through.

for the curious, the engineering notes: §39

v0.7 new scene

TORUS2: alive with the music

The Hopf torus rebuilt to move with the music: nothing inside the nest of rings falls dark any more, every kick flashes the quiet fibres at the core, the hi-hats run a shimmer round each ring, the colour follows the key, and the whole thing turns once every sixteen beats with a bounce on the beat. It takes the old TORUS's place.

for the curious, the engineering notes: §36 · §37

v0.6 engine

Retina Rave goes public

Eigenwobble becomes Retina Rave and goes live at retinarave.com. It works on a phone now — put the music on a speaker and let the page listen with the microphone, with a touch bar in place of the keyboard — and there is an about page that says who made it and why.

for the curious, the engineering notes: §34 · §35

▶ open the show

about & support · Retina Rave · a zero-dependency WebGL2 engine, native modules, no framework.