Nudist Audio Newsletter #12
How Fermenter works
Hey fermenters,
It’s been a big week for me! In case you missed it, the new plugin I’ve been working on for 2 years just came out last week :D
It’s called Fermenter, and it’s a strange new kind of timbral shifter. An all-round living tone dimension explorer.
In my spare time I’m actually a fermentation enthusiast! A bit addicted to kimchi, saurkraut, kombucha, pickled radishes, etc. So the name felt close to home, and also kind of a good description of what it sounds like.
I have been so, so grateful for the incredible reaction to this plugin. You guys are seriously awesome. People are already making some beautiful + weird music with it - keep sending it to me! And thank you to all you newsletter readers for being on my team.
If you haven’t heard it yet, here’s a little taste of what it can do.
piano, toggling between dry/wet:
guitar, modulating the timbre over time:
Ok so: a lot of people have been asking me how Fermenter works. Which is understandable; it sounds pretty strange, and like it might be doing something spectral, but it also has zero latency. And is it really a new algorithm?
So here I’m going to endeavor to explain it to anyone interested!
HOW FERMENTER WORKS
I couldn’t figure out how to write this succinctly. So,
TLDR: Fermenter works by breaking down down audio into thousands of tiny chunks, and then reassembling them in a slightly different way. The specific way that it reassembles them is what makes that timbre-shifting sound, and also makes it sound kinda funky and messed-up :D.
Ok, now for the slightly more detailed explanation.
Fermenter can sound like a formant shifter in some cases, but under the hood, it doesn’t work like other formant shifters. It’s actually closer to a grain resampler.
What do I mean by that? If you’ve ever used a grain resampler or grain synthesizer, you know that it basically splits audio into a bunch of little chunks, and then plays them back at different delays, pitches, gains, etc. You might have ~20 grains playing at a given time. The cacophony of all of these bits of audio playing at once is what makes that cool cloudy sound.
Now imagine you create a new grain every single sample. That’s 48,000 times per second. This is entering a strange temporal regime: not control-rate or audio-rate but sample-rate, the very finest time scale at which audio is rendered by a computer.
To understand this, let’s take the simplest case - that these grains have zero delay, zero pitch shift, zero gain change. They’re all exactly the same as the original sound.
What would this sound like? Well, think about it. Since they have zero delay, all 48,000 grains are playing the audio at the current moment, with no modifications. That means we’re just layering the current audio sample on top of itself 48,000 times, or in other words, turning up the audio by about 94 decibels.
Ok… so that’s a VERY big gain boost, but otherwise not so interesting.
Now imagine that each grain is pitched down by, say, 12 semitones. We play them all at half-speed, but still play one every sample. What would that sound like?
For reasons that I don’t have time to fully get into, this actually sounds like an extreme lowpass filter, and not a pitch-shift. With normal, low-ish numbers of grains like 5 or 20, we would still be able to hear those individual pitched-down pieces (this is sort of how some pitch shifters work). But when we’re playing 48,000 pitched-down grains at once, they start to all blend together into, as it happens, a lowpass filter!
That’s the magic of doing things at sample-rate: effects can transform in unexpected ways.
But here’s the crazy thing: when you then very slightly deviate the characteristics of these grains from one another - for example, their pitch - by a minuscule amount, all of a sudden, a wholly new and distinct sound pops out of the muck.
Why? Because the grains start to have interaction effects. In other words, when you play a whole lot of similar-but-slightly-different pieces of audio on top of each other, the result can sound distinct in a complicated and hard-to-predict way, depending on how those individual pieces relate to each other.
This new sound can be a lot of things, depending on exactly how these grains are different from each other. For example, if this deviation is at a normal, slow speed, this starts to sound kinda like a chorus!
But I’m not interested in normal things. I’m interested in weird things. So what I did is deviated the parameters using the incoming audio wave itself.
In other words, when the incoming audio wave goes up, the pitch of the grains goes up by just a tiny bit. And then when it goes back down, the grains’ pitch goes back down.
So now, all the grains have roughly the same pitch (in our case, -12 semitones, but the “timbre” slider controls this pitch), but are slightly deviated from each other using the pattern of the incoming audio wave. High-frequency inputs will create high-frequency deviations, and vice versa. A spiky transient on the input will create a quick jump in some of the grains’ pitches.
I found that when I did this, the new sound was a version of the incoming audio whose timbre or “formants” had been shifted, but whose fundamental pitch remained the same.1 I had no idea that this is what would happen before I did it - it was just a pure experimental accident.
And it also did… weird papery sonic stuff, that’s hard to describe… especially to polyphonic sounds. At this point, I knew I was on to something cool.
Still, there was a lot of muck surrounding the cool sound. So most of the work of building the plugin was figuring out how to get rid of that muck and just isolate the core “interaction effect” sound that came from deviating these parameters at sample rate.
Fermenter has a LOT of pre- and post-processing before and after this main grain thing. And I mean a lot. My final signal-flow diagram for this plugin has seventeen processes, and many of these are composed of multiple sub-processes. They’re specially-designed filters, compressors, band-splitters, envelope matchers, and other fun things, all with precisely-tuned parameters to make this weirdo core grain thing sound as good as possible.
This got complicated, fast. Which is why it took almost 2 years to make, and which is also why I’m pretty sure no one has ever done this before (and probably would have a hard time replicating it from just this description).2
But for the original, core sound, I had no idea what it would sound like before I heard it. This one was blindly discovered. And I think that’s just the coolest thing in the world! You can just have a math idea and ask “huh, what would it sound like if I did that?” And sometimes, it actually sounds good.
There are some interesting things about this algorithm. For example, even at zero pitch-shift (the “timbre” slider set to 0), we’re still deviating the characteristics of all these grains. So, unlike most formant shifters, Fermenter actually still does stuff.
I often like this zero-timbre mode the best. It can sound like a weird lofi filter-y thing, or a noisy gate-y thing, or even a distortion (try type “C”). I also really like when you set the timbre ever so slightly away from zero, like 0.1 or 0.2 semitones.
And then there’s the strangest knob of all - the “contort” knob, which adds a sine wave at a specified frequency to the deviation thing I described above. But depending on whether this sine wave is at control-rate or audio rate, and depending on what parameter of the grains we’re deviating (the “type” selector), and depending on how much of the normal deviation is going on (the “amount” knob), it can sound like several totally different effects. It’s my favorite knob in the plugin.
As you can maybe tell, this entire technical explanation gives pretty much zero insight into how to actually use Fermenter in music, or predict what it will sound like when you turn a specific knob.
You just have to trust your ears! Turn the knobs, and listen to what comes out and how the sound changes, and react to that however you will.
Hoping your gut is full of fermented foods and sounds this month.
Cheers,
Caleb
“formant shifting” is a concept that technically only makes sense when we’re talking about vocals. The idea is that the vocal tract resonates at particular frequencies, and if you know some physics, you can change what those frequencies are without changing the fundamental pitch, making it sound like the vocal tract is bigger or smaller.
So... it doesn’t really make sense to talk about “formant shifting” an electric guitar. Much less a drum kit. If you play a chord through a standard formant shifter... it usually just breaks. It doesn’t know what to do.
But then, there are some plugins that call themselves “formant shifters” that work on polyphonic sounds, which are usually doing a specific thing to the spectral envelope. In sum: people use the word “formant” to mean all sorts of completely inconsistent things.
If you do, all I’ll say is good luck!
You might also be wondering “wait, doesn’t it take like a lot of CPU to play tens of thousands of grains on top of each other?” Yes. Yes it does. The other big part of this project was figuring out how to get the CPU down to useable levels. I won’t get into the weeds here, but for reference, the first version used almost all of my Apple M4’s processing power. And that’s already a powerful CPU by any measure. It took some SIGNIFICANT finagling to get that down to 5-15% on most computers.
Can't buy this quick enough but bruh do you even Kefir?
This looks amazing Caleb.
Big fan already of nudistort - I'll make sure to check this out!