Elastique audio and samples interpolation

#1

Is the Axoloti capacity big enough to emulate the zplan Elastique samples interpolation? Is there any sample around?

#2

Z plane is patented isnt it? So no I dont think you should expect a 1-1 port. There are tables with 1,2 & 3 point interpolation in the library. Not sure what zplane uses, but the ones in the ibrary are pretty okay :slight_smile:

As I told you the other day, just get started om the patching, get to know the Axoloti and find out what it can do by trying. Axoloti can do many many things all ready, you just need to explore it.

Axoloti does do sample interpolation, but again probably not the exact type form Z-plane. Look through the library and see what you have, play with it, get familiar with it :slight_smile:

#3

Z-plane Elastique is indeed patented, but no one seems to know how it works. However, when it comes to other Time Scale Modification algorithms, I can help out there (It's my PhD area).
Elastique doesn't just interpolate between points as it maintains the pitch of the audio being interpolated.

For a basic explanation, TSM algorithms work by framing the signal and then manipulating the ratio between the analysis shift/hop and the synthesis shift/hop. The differences between methods is how each tries to maintain coherence within the signal.

Generally, methods are broken into Frequency domain and Time domain methods, each with strengths and weaknesses. Frequency domain methods are better with harmonic and complex material, where as Time domain methods are generally better with transient rich material. The Phase Vocoder and extensions (Flanagan and Golden, Portnoff, Laroche and Dolson etc) make up most of the frequency domain methods. These methods modify the phase progression between frames to maintain horizontal phase coherence. OLA and it's extensions SOLA, WSOLA, etc (Roucos and Wilgus, Verhelst and Roelands) maintain coherence by overlapping frames at points of maximal similarity.

For implementing TSM on the axoloti, I would first look at trying to implement the SOLA algorithm (Roucos S. and Wilgus A.M., “High Quality Time-Scale Modification for Speech”, IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 493-496, March 1985).
If there is an efficient FFT on the Axoloti (I haven't looked in a long time), the phase vocoder should also be possible.

If you would like to see some MATLAB code of these and other methods, check out my github at www.github.com/zygurt/TSM . I also have some algorithms on there that will be published within the next fortnight. Frequency Dependent Time-Scale Modification allows for time scaling of multiple frequency ranges by different amounts. This one definitely won't work on the axoloti.

When considering real time, you will run into restrictions regarding the amount of memory available. My advice would be to write the algorithm and then assign the rest of the available memory as a circular buffer.

I hope this all helps. Let me know if you want clarification on anything.

#4

@Zygurt

Thanks for the details. I have had this obsession with real time time stretching for years, but not on phd level, mostly like a hobby. But it is a very interesting subject, one of the areas of sound design I find most interesting.

I am all ready in way over my head here, with the things you describe. I have written many many simple time stretch algorithms in both Pure Data and a few for Axoloti, which have become better over the years.

After writing those simple algo's I wanted to try something new, like FFT/phase vocoding. But when it comes to FFT/phase vocoding, the math/technical level of things seems to escalade pretty quickly, to a level where I cant keep up. I am bachelor in social education, which is humanoire, so my math is not really that great, haha :slight_smile:

Anyway, it have been discussed a bit, the limits and expectations of Axoloti, but we haven't seen anything concrete come out of it yet:

@Zygurt I think actually there is a phase vocoder in one of the Mutable objects. Dont remember which one of them.

#5

Yeah no worries at all. It took me a bit to get my head around it even with an Engineering background.

When I started in TSM, my project was to write a real time TSM library for Extempore, which is a live coding language. Dealing with multiple smaller buffers to create larger ones that allow for this type of processing was part of it. Collecting a large buffer, and then processing it would certainly be the way to go about it. Can't say that I like the thought of optimising an assembly FFT implementation though.

Overlap-Add (OLA)(I should have said that before) methods work with small buffer sizes, so that could be an option. You don't get fine control over the speed with only 16 samples though.

A filterbank version could be interesting though, as that can be implemented in the time domain. There is a method muTVS that does this, before a Hilbert transform. Far too much processing. Hmm.

#6

Sorry in advance for the wall of text. As part of my PhD I’m doing some research and I would appreciate your involvement in a short study. I’m currently collecting data on the quality of a variety of Time-Scale Modification (TSM) methods that change the speed of an audio file without modifying the pitch. This research will inform future TSM methods and lead to higher quality time scaling.

To be involved, you’ll need access to a computer with Windows and good quality headphones/speakers. Follow the link below to download a zip folder containing the testing software (approx. 2 GB). Contained within this folder is a Readme on with how to install the required library (approx 1GB download) and do the testing. The program will take a little while to startup after each reboot. There are also Information and Consent forms within the zip.
https://drive.google.com/open?id=1CcfwvrVRhlzif6r7Zr2YWiJupca3DfC2

The testing comprises of an initial test set (5 minutes) along with at least 1 of the remaining 26 sets (20-25 minutes). The number of additional sets that you complete is completely up to you. During the testing you would be listening to an audio file and rating the quality of the file using a slider. You are able to save and return at a later time. After each set or save, the results will be automatically emailed to me. The testing has ethics approval through Griffith Uni Ethics with Reference 2018/671. If you would like to be involved, please print, fill out and send me a copy of the Consent form to timothy.roberts@griffithuni.edu.au.

If you are interested or have any questions, let me know. Additionally if you know someone who may be interested I would appreciate you forwarding the link and passing my details. There are a lot of files that need multiple ratings, and many ears make light work.

Additionally, if you are interested in checking out some of my research thus far, this is a link to a library of TSM methods implemented in MATLAB. https://github.com/zygurt/TSM It includes a pre-compiled program, for Frequency Dependant Time-Scale Modification (FDTSM), which uses the same runtime library to scale 10 frequency regions between 50-200% using an interface similar to that of a graphic equaliser. https://github.com/zygurt/TSM/tree/master/FDTSM/FDTSM/for_redistribution_files_only
If you experiment with this, let me know what you think.

#7

I am obsessed with time stretching, so this sounds interesting:)

Any plans to make something for Axoloti, based on the results?

Would love to do the test, but I only have OSX in the house.

Anyway, wish you good luck with it :slight_smile:

#8

I could certainly make something for the axoloti, I just haven't looked into it for a while.

I also have the testing for OS X, but it requires using a 30 day trial, rather than a runtime library, and then manually sending the results to me. If you're interested, head to mathworks.com and download a trial of MATLAB.
I'm just in the process of updating the OS X version, because I noticed that it wasn't up to date, so I'll post it again soon.
The Windows version above is fine though.

#9

Ok, so here is the link to download the zip folder. (~2GB)
https://drive.google.com/open?id=1Fe_-h1yD4WUOMmz-piCdfXRP_6ATrFCR
Inside you will find a How to guide for getting everything setup, and Ethics information and Consent forms.
If you do some testing, please fill out the Consent form and email it back to me at timothy.roberts(at)griffithuni.edu.au
Also, after completing each set, please email your user file in the Results folder to me at the same address.

#10

All right, so I now have a much better option. It's online, requires no additional installs, each set is only a 45 MB download and should only take 10ish minutes. You can find it here. http://timrobertssound.com.au/TSM/index.html