Iirc I disabled the spectral mode in clds object as it was unusable. you can ‘edit’ the object to re-enable.
Though you’ll need to edit the firmware if you want to alter/improve is behavior.
Soundmagic Spectral
thetechnobear
#14
toneburst
#15
I see. Presumably you could reduce the number of bands, as a trade-off between quality and buffer-length.
a|x
toneburst
#16
Oh, sorry, 8192 is the length of the buffer that gets updated every 512 k-rate cycles.
Do you think the Axoloti could resynthesise a 512-band fft at s-rate?
a|x
watamacha
#18
im aware of 3 methods for arbitrary fft resynthesis: ifft, additive, or vocoding.
ifft may be possible but is probably about as hard as the original fft. here's one approach using the regular fft to construct the ifft.
As far as additive or vocoding, you create a sine osc or send noise through a bandpass with amplitude equal to the band for each band of the fft. this would be equivalent to having either 512 sine oscs or 512 band passes running simultaneously. so good luck.
However, if the band centers have an integer harmonic relation (i.e. are parts of the overtone series of the lowest band's center) and the lowest band is centered at f, the resynthesis could be done by constantly updating a wavetable with length 1/f*48000 with a weighted sum of the wavetables for each band's sine. the harmonic requirement guarantees that all of the sine waves will complete an integer number of periods per each period of the fundamental, guaranteeing that the end of the table will line up with the start of the table regardless of the weightings, preventing the introduction of unwanted transients. Whether this is any more useful in practice isnt something I can comment on but its an option. Still probably too much going on for the axo since for an f0 of, say, 30hz, this will require 1600*512 multiplications AND additions per wavetable recalculation.
valmir
#19
Hi! Check this out.
https://sebiik.github.io/community.axoloti.com.backup/t/community-pedalboard/3641/4?u=valmir
The latch mode MIGHT sound a little bit like that.
No spectral things were harmed in the making of this patch.
toneburst
#20
Interesting...
The fact that MI Clouds is able to do FFT and resynthesis (with some mangling in between), using a similar Cortex M4 MCU suggests it may also be possible on the Axoloti, though.
I wonder what resynthesis method Olivier went for.
a|x
watamacha
#21
since clouds is open source you might be able to figure that out pretty easy.
Another method that might be worth considering is using a combination of FM and additive synthesis, since FM produces interesting spectra which can be modelled with 1st order bessel functions. I'm envisioning a sort of "smart" additive resynthesis which uses non-sinusoidal waveforms combined in the following way: start with the spectrum you want to resynthesize, with harmonically related bands. Create an oscillator with fundamental frequency at the center of the lowest nonzero band, then subtract the pre-calculated spectrum for said oscillator from the original spectrum and repeat. said oscillators could be saws, triangles, and squares (I assume square would be the most efficient), and/or some simple FM signals. this would only really work if you're very careful about phase or do all your spectrum calculations calculations using complex numbers since complex spectra are linear and real spectra are not.
yet another tool you could use is to introduce noise equal to the minimum amplitude of any band and then reduce all bands by that amount, that way you get a guarantee of one band being 0 and a decent chance of a good number of bands being negligibly distant from 0, creating a sparse spectrum. Since sparse fft tells us that sparse spectra can be easier to compute, its fairly reasonable imo to assume the same can be said about the resynthesis of sparse spectra.
since spectra are linear barring phase issues, your design space for spectral resynthesis basically consists of using something to create enough of the partials you need and subtractive to cut out ones you don't. to get the partials you need, you can use any mix of fm, additive, wavetable, or noise synthesis.
spectral analysis isnt my main math squeeze so I'm unsure if such resynthesis has been attempted previously, whether it works, and whether some of the more specific mathematical statements above are correct
edit: added some stuff. check this paper on evolutionary FM/subtractive resynthesis of dynamic spectra out.
toneburst
#23
That’s intriguing. I’m afraid, though I know some of the terminology, my grasp of the mathematical principles involved is tenuous at best.
In terms of which method would be most interesting, for me, it would depend on how much potential it offers for creative disruption/mangling, rather than necessarily how well it performs in terms of fidelity of reproduction.
a|x
watamacha
#24
I'm reading mangling to mean unpredictable but interesting results, like blindly patching a modular drone. I think the most sonic possibilities will come from using FM, and you wont really need to know the special math for FM stuff since you can naively compute the spectra ahead of time (naive here means you can ignore the cool math and just make some preset fm waveforms and compute the spectra). However, if you use additive resynthesis from cheap waveforms you'll leave yourself as much dsp room as possible to do cool subtractive stuff on it, or things like wavefolding and pwm, which can still be set up in an unpredictable and creative way but without the high risk of stuff sounding bad that you get with FM. This is because for all the possible sounds from FM mangling, only some of them will produce harmonically simple enough waveforms to sound good.
I'll explain the math a bit for you. Usually we view audio signals as amplitude with respect to time. However, a combination of cool things said by old crazy nerds like euler, laplace, and fourier let us write those same functions as amplitude with respect to frequency. The coolest thing is that we have functions to convert between these 2 domains (domain here means the same thing it usually does, the x axis on your graph). spectral resynthesis is about finding easy and fast ways to convert from the frequency domain to the time domain. Instead of trying to mathematically convert from the frequency domain back to the time domain (inverse fourier transform) we synthesis (the resynthesis part) a wave that looks almost the same in the frequency domain (the spectral part). Since sine waves have only 1 frequency, they're the simplest way for humans to think about spectral resynthesis because you just have to make a sine wave for each point in your spectrum (graph where x is frequency and y is amplitude) and make the frequency of the sine wave equal the x coordinate of the point, and adjust its volume to the y coordinate of the point.
Unfortunately, sine waves are really hard for computers to make. Luckily, square waves arent. We can basically do the same thing as before, because square waves are just a bunch of odd harmonics with amplitude inversely proportional to frequency. In other words, a square wave at frequency f is sorta like a sin(f), and its a little more like a sin(f)+sin(3f)/3, and even more like a sin(f)+sin(3f)/3+sin(5f)/5, and so on. Because of this a square wave is very cheap for a computer to work with not only in the time domain, but in the frequency domain as well, since we can ignore all the harmonics higher than 20khz, so we have a pretty limited number of them to deal with and that number decreases as the frequency of the square wave increases. Note that I omitted some scaling factors for readability, for those factors just look up the square wave fourier series.
now I'll change the wording for how we think about spectral resynth. suppose our target spectrum is T(f), aka a function of frequency that we want to get, and that our resynthesized waveform has a spectrum of S(f). earlier we said we want T(f) = S(f), but if we subtract S(f) from both sides we get T(f)-S(f)=0. Even though we're in the frequency domain, spectra are still functions and pretty much act like it, so we can do a lot of the same stuff with spectra as we can regular functions.
The other thing we can do is say that if we add 2 simple waveforms, the spectrum of the sum is the same as the sum of the spectrum (i.e. the fourier transform is a linear transform).
If we think about our sine wave resynthesis again, what we're really doing is subtracting the spectra of a bunch of sine waves from our original spectrum in the hopes of getting close to 0. We can do the same thing with square waves, the only difference is that if we add a square wave at frequency f with amplitude A we need to subtract A/pi from the original spectrum at frequency f, A/3pi at frequency 3f, etc.
So to resynthesize a discrete spectrum (i.e. a spectrum with a finite number of points, where the x coordinate is the middle of a band and the y coordinate is the amplitude of said band) with square waves, all you have to do is look at your lowest band and make a square wave at the center frequency of that band, and adjust its volume to match the amplitude of that band. The only extra step is that you now have to subtract the higher partials before moving onto the next band, but the subtraction is easy to calculate and your next step is still to just add a square wave at the current lowest frequency band thats got an amplitude that isnt 0.
There are 2 things that complicate this, one of which matters and one of which doesnt, so I'll discuss the important one first.
the fourier series of a square wave isnt really what its spectrum is gonna look like, because it assumes the square wave goes from negative infinity to positive infinity on the time axis. What happens in real situations is that these peaks in the spectrum (called kronecker deltas, or places where the function jumps from 0 to something else and back to 0 instantly) get smoothed out and end up looking like normal distributions. that doesnt really matter since you can just run your square wave through the spectrum analyzer you're using ahead of time, and as long as you're comparing spectra that were produced by the same analyzer you should be ok.
the thing that doesnt matter for square waves but would matter if you wanted to try other waveforms is that when your x axis is frequency, your y axis is actually complex. That sounds scary, but think of it like this: your x axis (frequency) is now the middle of a cylinder. your amplitudes are vectors that point towards the wall of the cylinder, and the length of the vector is still the amplitude of that partial frequency, but the amount the arrow is rotated in the cylinder is the phase shift of that partial. if the arrow is pointing straight at you, it's got no phase shift. if its pointing away from you, then its got half a period of phase shift (pi radians or 180 degrees). if you phase shift a periodic signal by 360 degrees, you're basically spinning the cylinder around a full cycle, and all the arrows end up pointing in the same direction as they did before. The only thing this affects in our earlier model is how spectra are added and subtracted. say you have an arrow in a spectrum at frequency 220hz with amplitude 1, and its pointed towards you, so its not phase shifted. now say you add another spectrum with an identical arrow, but with 180 degree phase shift so it points the other way. if you add the spectra together (aka add the signals together) then the result is 0 at 220hz because the vectors cancel out, but since our earlier model didnt include phase shift it just reads 1+1 and spits out 2. luckily, this doesnt matter for square waves because all the partials of a square wave have no phase shift. there are other waveforms (in fact infinitely many) that also have this property, and for each one of those there are infinitely many other waveforms that don't have this property.
So we luckily dont have to worry about phase cancellation with perfectly tuned square waves, but the tradeoff is that if one of the square waves gets detuned we'll get an interference pattern. The solution to this is to sync all of the square waves to the lowest one, but that only works if all the square waves we use have a frequency thats an integer multiple of the frequency of the lowest square wave.
If you want a function written for you let me know, but understanding all this should give you enough to write the function and is honestly necessary to do anything interesting with it
toneburst
#25
Wow, that's pretty comprehensive explanation, thank you very much, @watamacha..!
I have to say, it's going to take some digesting, but I'm working on it 
I think I've got the bit about the importance of phase information resynthesising additively, though. I'd heard about this before, but now I have more of a grasp of why it's so important, in terms of phase cancellation effects.
I'm still vague on the bit about using square, instead of sine waves, I'm afraid. I can understand why they're easier to calculate than sines, and that if using them instead of sine waves, you'd need to remove the extra partials. What I'm still unclear on is how the partials are removed/filtered out, presumably in the time domain.
a|x
I think that the transform with squarish waveforms @watamacha is looking for is the Walsh Hadamar transform. Note that Walsh functions where used in early RMI Harmonic synth instead of sines, hence a bunch of characteristic overtones.
Calculating sines is an issue, but having sine functions in a table is the way to go.
In fact, the sine2t table contains a 4096 sample sine that is used in many objects.
toneburst
#28
Interesting...
Isn't the issues with sine tables that the largest the phase-accumulator value, the 'steppier' the sine becomes, given a fixed sample-rate?
Maybe in practice, that's not an issue.
a|x
I don't understand what you mean, can you draw something?
Tables are generally interpolated.
Blindsmyth
#30
Wow what discussion! I'm hyped but at the same time realizing that all of this is way beyond my capabilites. but it's still interesting though!
At the moment I'm getting used to the grainverb by @johannes again wich sounds pretty good actually. But what I'm missing compared to the spectral freeze or paul stretch is this smashing of transients and consonants into long noisy trails. Any Ideas how I could implement this in a cheap way that would complement the grainverb?
Some kind of vocoding/envelope followers with white noise as carrier maybe?
toneburst
#31
I'm probably misunderstanding, but my thought was that as the phase accumulator increment gets higher, you'll end up skipping entries in the table, and the more entries are skipped the more approximate the waveform becomes, until eventually the resulting waveform becomes essentially random.
I must admit I haven't really got a grasp of the numbers involved, though. It could be that, with that table length, and the sample-rate the Axoloti runs at, the table will never skip values.
a|x