Polyphonic voice assignment

#1

I'm not really sure if this should be in the 1.0.10 thread or separate, but it is not really directly related to 1.0.10, although commit 90adea74fbbc4dbf676832f2c28b408c607c89aa (GUI: polyphonic voice allocation - keep cycling when repeating one note) made me think of it.

What has struck me about polyphonic patches in Axoloti is that there is no control over the voice allocation. Many synths have a couple of assignment modes, often with slightly obscure names like Poly 1 and Poly 2. In practice there are a number of things that could be configured, and in such a flexibile system as the Axoloti I think it should be possible.

Things such as:

  • Should playing voices be robbed for new notes when the polyphony llimit is reached?
  • Should the same voice be played when playing the same note, or should a new voice be allocated?
  • If more notes than voices have been played, and notes are released, should the notes that were once silenced be played again (in essence, a polyphonic version of the monophonic behavior that when two notes are played, and the second one released, the first one is heard again).
  • etc

Also, the sustain pedal behavior could be taken into account. A handy pedal mode is 'sostenuto', whereby the sustain pedal only holds notes that were playing at the time the pedal was pressed, newly played notes aren't sustained. This allows you to play a couple of notes as a drone and then play on top of that. (It mimics the middle pedal on certain grand pianos which have this feature).

I feel it's not possible to make the voice allocation choices once and for all, it depends on the patch what is suitable. For instance, if playing a repeated note, if the patch is a drum or piano sound, it more accurately mimics the corresponding acoustic instrument if the same voices is triggered - and that also avoids stealing voices that are already playing. On a pad sound, it can tend to cut off the release of the previous note, so having a cyclic assignment might be better. So, these choices should be up to the user.

It would also be nice to have a panel in the UI where the individual voices lit up as indicators so you could see which voices are in fact allocated.

What I'm thinking is an object with no actual connections, with a number of parameters in the form of switches which control the allocation modes. If no such object is present in the patch, a default allocation scheme is used. Such an 'allocation' object could also have a row of indicators for the actual voices playing.

I don't know if anything like that is planned. I don't mind looking into it in due course if there's any interest. I feel voice allocation is a fairly important thing to have control over.

Sub Patching Topology Question
#2

Yeah this has been raised a few times.

Main issue is this code is currently embedded in the Java app, and also linked closely to midi handling.
its not hard to do, just needs careful thought on how to 'represent' it

#3

as I mentioned above, representation is tricky,

have you thought how you would represent this to the user?

heres my thought that I've had on this in the past, but not come to any real conclusion.

Axoloti

allocation strategy is held as a property of the patch (subpatch in reality), that then triggers different generated code, which can be found in patch.java
it would be fairly easy to add new ones, but obviously cannot be extended by the end user.
I think the other weakness is the link between midi and voice allocation, yes, this is most often useful but not always.
(also I don't like that some of the stealing/re-use logic is held in the midi/in objects, it should all be in one place)

one oddity in Axoloti, is generally, Axoloti is not event based, yet it handles midi as an event based, makes allocation 'easier' but not in line with other axoloti models.

note: midi handling is subject to change, currently it gets an immediate callback on receipt of midi messages, but this is 'wrong' and potentially problematic, we plan to move it to a queue that will be processed in the dsp execution (e.g. like a k-rate,s-rate handlers)

real strength is its trivial for end users to use...

alternatives, I know 2 reasonable well, Reaktor and Max/MSP.

Reaktor

I really like reaktors model, objects are marked as polyphonic, and wires becomes polyphonic. some built-in objects are polyphonic like midi in, so have a 'strategy' BUT there are objects to go from mono to poly to mono (toV, fromV iirc), to allow you do to as you please.
I love this model, but its quite a distance from how Axoloti has things.
advantage is, flexible, and you don't actually need to have voices in sub-patches.
I will point out, implementing voice strategies is not trivial...definitely and advanced use.
(Reaktor selects max voices on the ensemble level)

Max/MSP

polyphony is handled in Max using Poly~ and poly, the former creates voices, the later (optional) has a voice allocation scheme that makes it 'easy' to link midi to poly~. In some ways this is quite similar to Axoloti, in that sub patches are used to represent a voice.
(though, poly~ is pretty powerful as can be threaded, and have up/downsampling ... might be an interesting way for axoloti to do oversampling for a group of objects)
I will say though, its not trivial for a beginner to use poly~ , Axolotl's approach is much easier to just say ... I want this synth to be polyphonic... and we can't take a step back in this regard.

there are of course, various reasons, why neither the Max nor Reaktor model would sit perfectly with Axoloti. (Max in particular is a bit too 'event based')

a couple of random notes:

  • obviously also need to be 'backwards compatible'

  • Max/Reaktor bring voice allocation under 'patch control', it would be easier to do in Axoloti if we expect allocation schemes to be written as custom objects.

  • we don't have a way of allowing 'classes' to be specified e.g. you can't say I want N instances of subpatch type X to be created.

  • due to above, it might be easier to allow a 'polyphonic custom' strategy, that simply creates the voices, but defers how midi messages (etc?) are routed

  • it would be simpler to change the midi routing to voice (see above) than to tackle the more generic voice allocation.

  • there are some 'generic' things you want/need for poly voice allocation in particular, e.g. knowing voice age... you probably would not want to replicate this in custom strategies.

  • I think its possible to write your own voice allocation system current, its just not very nice. there are a few things you can use like the internal midi bus, and the poly multichannel allocation, to allow an object to decide how to allocate voices. also you can use the polyindex with a patch to do things.
    of course this is not ideal, and is a little complex (though writing voice allocation scheme, in any way, is never trivial) , but its not impossible either.

anyway, thoughts.... how would you represent this?
goal is relatively simple, something that is as trivial to use as it currently is by default. (just mark as polyphonic), yet allows for more flexibility?

BTW: to get a feel for all of this, best to review patch.java to see whats currently being done, and also look at the generated xpatch.cpp code for the different strategies. you'll see its not that difficult.
In fact, I think the first think I ever added for Axoloti, as the "Polyphonic Expression" mode to handle MPE messages in Axoloti, it was a good 'learning' experience.

#4

Ah yes, this would be great. I've come across lots of voice allocation modes that are useful for different things so it'd be great to have them as options for something like the Axoloti where the entire point is versatility. Not sure what Axoloti uses at the moment, but useful ones that come to mind are:

Round-robin: Simplest. Where the voices are allocated cyclically without regard for whether any are currently in use is probably the most basic. Could be used for cool voice variation in conjunction with polyindex. Very Oberheim 4/8 voice-like.

Earliest available: If no voices are currently in use then voice 1 is used. If three notes are held, using voices 1-3 and then the note using the second voice is released, then the next note would use the second voice if that voice has finished its release phase. In a fixed architecture synth it's not too hard to see when a given voice has finished its release, but in the Axoloti how do you decide the voice has finished it's release?

A cool option is deliberate voice stealing where a note that is retriggered steals the voice that was still playing the release phase of the previous instance of that note. It helps avoid overlapping instances of the same note, which of course you wouldn't get on an instrument like the piano. Of course that could be considered desirable for other applications (it can sound great when there's a tiny amount of randomisation to the tuning of each voice so you don't get phase locks).

#5

If we'd split up the general "patch/patcher" object into "patch/patcher mono", "patch/patcher poly", and "patch/patcher mpe" and add an xml field for the voice allocation code, that 'd open it up for more variations of allocators.
That 'd look fine for embedded subpatches.
But currently the mono/poly/mpe setting is part of the subpatch. This 'd break instancing an external poly subpatch. To create an external poly subpatch, you'd need to wrap it into an external subpatch with a "patch/patcher poly" object inside, and then the voice structure inside. Often a useful strategy, since a polysynth will likely have a polyphonic part anyway.

But preserving compatibility will be hard...

#6

yeah, if you make everything embedded this helps, though I will say, we have code that is dependent on 'patch/patcher' (e.g. editor) , and without proper subclassing having derived forms is going to be problematic.

I think there are perhaps 2 parts to this problem...
a) voice management, is it poly or mono
b) voice allocation

(a) really simple, as we only have 'static' voices, i.e. its 1 or N... we don't allow more voices to be added.

(b) currently, is only really handled by midi (shouldn't be, but it is) ... and this is simply a midi handler.
so theoretically, we only need to substitute a midi handler... this could simply be a proxy object.
(i.e. its midi handler is called, and it is passed the proxied object, to call if it wants to allocate to that object)

another option, which we have discussed before, is perhaps we should have a 'midi event' type, this could then be handled in the parent class, and perhaps then pass the event on, to and instance of a voice. this could be relatively easy, as we could use 'channel' as a proxy for voice id.(this is similar to as i mentioned above about using the internal midi bus, but more direct)
this has the advantage midi message routing becomes explicit, rather than the current view where the patcher does not show the midi routing. (also open up midi filtering, and all sorts of midi fx options)

thing I'm not keen on on this, is it propagates the 'midi' to voice , which i think should only be one way... e.g. when your doing sequencer, then midi is irrelevant, but you still might want to be polyphonic.
the only way currently, is to do it all yourself (I've done this, its not that bad), or go via midi.

#7

Well, my spontaneous thought, without having looked at the code or anything, was to add it to the subpatch object. There is already a 'poly' spinner representing the number of voices in the subpatch, so it would be natural to add a 'voice allocation' combo box with various modes.

My initial thought was that the modes could be implemented in the firmware (or as the case seems to be, in xpatch.cpp), and the object would just select among a number of preset modes. On the other hand, looking at the discussion above, given the flexibility of the Axoloti, perhaps we could do better.

There are objects such as sample playback, which refer to a 'memory' object (using the objref type), so perhaps a subpatch object could refer to an 'allocation' object which would do the actual allocation. The idea being that an 'allocation' object would actually have the allocation in code, so that you could write your own allocation algorithm if you wanted to, of course there would be a number of factory objects with useful pre-set algorithms, or perhaps better a single one which was configurable using parameters. Thus the actual API between the subpatch object and the allocation object would be hidden and doesn't need to be restrained to the ordinary data types.

Otherwise it would be cool with some form of 'note' data type, which basically could be MIDI messages stuffed into an int, one int per message. Incoming note data, be it from MIDI, USB, or from an object scanning a string of switches laid out as a keyboard, could then be patched into an 'allocation' object which would accept that type of data at its input. A hidden internal API would then be used to actually trigger the voices.

Perhaps there could be an attribute in the envelope objects, indicating if their release phase should be considered when it comes to allocation. That way you can have both envelopes which do affect allocation, but at the same time you don't have to. On the other hand, it could get very confusing if set up incorrectly, with all voices seemingly never getting relased if you patch something strangely.

#8

Another use case to consider: tonewheel/drawbar organs...

Tonewheel/drawbars would currently partition into

  • one table/alloc object, containing the amplitude of each tonewheel.
  • one "drawbar" object manipulating the amplitudes, midi/in/script could serve this
  • one polyphonic subpatch of tonewheels accessing the amplitude table, without using any MIDI features in the subpatch.

In case of more regular voice allocation one could expose tables containing the voice pitches, velocities and gates. Written by the voice allocator, read by the voices.

Or more generic, another table that is written by the voice. Then it could be the output of any envelope or envelope follower that gets sorted by the voice allocator.

#9

hmm, I think I would 'avoid' have the voice management know anything about envelopes or even gates.
(voices are not necessarily always used for such things)

Ive another idea... which actually is based a little on the Max/Reaktor models,

polyphonic inlets and outlets

(this would not replace the current mechanism, at lease initially, it would sit alongside, and be used with patches marked as polyphonic)

polyphonic inlets

this would be similar to normal inlets, except the 'outside' can address individual voices.
this is like Reaktors toV object, you give it data and a voice, to target a specific voice.
inside the subpatch, these polyphonic inlets, are seen as monophonic.. so no change there.

(in someway this is getting a little like reactors model, the subpatch is marked as polyphonic, and some wires are conceptually polyphonic)

for backwards compatibly we keep existing midi polyphonic behaviour, and the default behaviour of a monophonic outlet to a polyphonic inlet is to copy the data to all voices.

polyphonic outlets?

well this is actually the reverse, currently what happens to outlets of polyphonic modules depends on type e.g. data is summed... but it would be nice have some control of this...
Im thinking here of an object like fromV
what this does is take a polyphonic outlet, and a voice index and has a monophonic output.
thus allowing the parent patch, to be able (with multiple fromV objects, or a selector) take apart the various outputs.
(the default behaviour of a polyphonic output into a monophonic input, is the current behaviour e.g. summing)

what does this mean?
well it means we don't actually needs allocators at all, instead like Reaktor/Max , we just provide a mechanism for patch writers to access voices directly i.e. write there own allocator.
because its 'separate' from voice allocation, they can live side by side (at least initially, we may review in time) - which means compatibility.
its very generic, no assumptions about what a voice is used for.... e.g. its not even assuming the voice is used for audio, it could be anything.
(it would also fit nicely if in the future we make midi data type, as that could then feed in/out using polyphonic inlets/outlets)
I also think its quite simple to implement :slight_smile:

(simple) example of use:
so a user whats a simple polyphonic patch, with note and date input via midi... but wants to take control of how the midi gets translated to a voice

a user creates a sub-patch with 2 polyphonic inlets, note and gate... they may also have mono inlets, eg. for filter cutoff control.
in the parent patch, they have an object with a midi handler , that looks at the midi data directly... selects a voice , and then puts the note and gate data on that specify voice inlet.

there is a question about if this can be done only as a custom object or also as patching.

a custom object is the most efficient, since its the only way to do conditional behaviour
BUT we can also have a patching objects.

this basically would required that the user would (potentially) need to a patching object (toV) for each voice they want to target , the toV object takes a voice id and a data value (so one for each data type), the voice id can either be fixed (using const/i) or derived. the output is a polyphonic output to connect to the polyphonic inlet of the subpatch. using a dynamic value for toV obviously means its only possible to set one value per 'k-rate cycle' or midi handling cycling.

(visually , we should differentiate poly inlets/outlets and wires from mono equivalents... perhaps like a 'double line', as its 'thicker' coz its got more data ?)

one other option object developers have, is to write polyphonic outlet objects...
e.g. one could write a polyphonic midi/in/keyb this could then directly be connected to polyphonic sub patches.
(this is exactly the approach Reaktor takes)

Thoughts?

(sorry the above perhaps is not as clear as it could be, but if it has interest then we can dig into details, and I could make some drawings to clear up any confusion.
but to be clear, its not too dissimilar to Reacktor toV/FromV , an in alot of ways the way poly~ is used in Max when you give it a voice number... so reading those will probably make it reasonable clear)

#10

Can that simplify to array types for inlets/outlets, and a mechanism for poly subpatches to distribute an array inlet/outlet to each voice?
A voice allocator object could have pitch, gate and velocity array-outlets.
A poly subpatch outlet could expose as an array-outlet on the parent object, summing could be an implicit conversion when connecting to a non-array outlet.

This could also be a solution for oversampling/downsampling. Except in case where a polyphonic subpatch operates in oversampled domain and the downsampler needs to be shared for all voices, then we 'd need 2d arrays...

The hard part: which object will decide on the array size of a net, so that run-time bounds checks can be eliminated? If the source outlet decides, two sources could connect to one object that each have a different array size. If we just check for mismatches, still, both the voice allocator object and the poly voices object must be configures for the number of voices.

Sometimes a mismatch between allocator and poly voice object would be useful too, in case where one Axoloti Core handles voice allocation but multiple boards are used to generate the voices. Then the cv/gate arrays could be split and distributed at control rate. But then we need objects to expand/combine/subrange arrays. An array expander object, translating one array-inlet into one outlet per element requires more object dynamism than there currently is. Or it could also take the form of a one array-inlet into one outlet and an array-outlet of the remainder of the array, but also in that case the array-outlet size is only known after connecting the array-inlet. This needs further thoughts...

#11

yeah I kind of envisaged this being implemented as arrays... in some ways.

i don't really see the need at all though for a 'voice allocator', a voice allocator just becomes an object that happens to 'output' polyphonic outlets.

e.g.
imagine a new polymidi/in/keyb it could output a polyphonic pitch,gate,velocity ... and yes by implication, the decision on how it decides what messages to put on which channels, means it is a voice allocator. the cool thing is , don't like it? just implement a different one!
but its important its not just notes, e.g. polymidi/in/cc could also be implemented and again would decide how to allocate these messages to each voice.

with this, arrangement we could have implemented the 'polyphonic expression' modes without resorting to a new 'type of subpatch'

array size, yeah Id assumed this would be calculated from the subpatch voices.
this is easy for direct connect to the subpatch.

I think the question is do you need polyphonic wires upstream?

so... max doesn't do this at all, basically you can only communicate with poly~ as voices, in the rest of the patch, you have to use your own table structures.
in Reaktor, this is controlled at the 'instrument' level, basically you define the number of voices.
(hmm, could be ensemble, would have to double check :slight_smile: ) , this means any polyphonic wire has that number of voices. Id say for us we could say its at 'patch' level...
(this ensures within a patch a poly wire is a known size everywhere, but at the boundary to a subpatch it can change)

bounds checking etc, well I think the 'wire' has to know the number of voices, and we can decide what happens when the user 'access' a wire > poly count. possible options are... ignore or wrap around (modulus) ...its considered a 'patching bug' but we have to do something that doesn't crash :slight_smile:

oversampling/downsampling - yes, this is what max does with its poly~ object, its basically can oversample internally, and then downsample the outlets back to the parent patch SR.

ok, the above is all a little bit 'all over the place' , as yeah, it needs a lot of thought to get it right, but it seems a simple solution (at least conceptually) thats very flexible.

#12

Excuse my ignorance if this already exists, but I'd like to send from the polyphonic input to individual voices. Having a separate sub-patch/voice with different sounds, regardless of assignment regime.. something like the Roland 100M 184 keyboard, or pizmidi midiChordSplit in VST terms.

#13

you could already do this with a custom object, as it can create a midi handler that can split out the voices, and send to different outputs. the main issue you face, is that while the midi handler is event based, the k-rate handler is NOT, this means an individual k-rate port theoretically cannot work, since you might have 2 midi messages arrive within one k-rate cycle that specified two notes. I say theoretically, because its unlikely your going to get midi at 3000 msg/second (the k-rate)... in practice, i suspect a midi/in/keyb with objects patched to split out the messages to different sub-patches would likely work.

going forward a polyphonic outlets (and poly midi objects) would make this much easier!
simply use the polymidi/in/keyb and then use an 'fromV' object to split into different outlets, to be then passed onto the voices.
(different polymidi/in/keyb variants could implement different allocation strategies e.g. RR, last, random etc)

#14

Sorry for reviving this old thread, but i have a very specific question now:

So, i managed to re-write the midi code of a polyphonic subpatch so as to give me sostenuto instead of sustain (see discussion in this thread). However, i can only save changes to that subpatch code after i do embed as patch/object (via right-click on the object). And once i've done that, editing the subpatch via the GUI is no longer possible. That is, there is no edit (nor update) button on the patcher object any more, and right-clicking edit object definition only gives me the raw code of the patcher object, not the pretty window with the subpatch.

Now, I understand this is all intended behavior, but it means that it is vastly inconvenient (i.e. effectively impossible) to finnagle the contents of the polyphonic subpatch after changing it to sostenuto (the intended behavior being "don't mess with the MIDI code of a subpatch!"). I also understand (or think i do) that this is due to the nature of subpatches, which do not store their own midi etc code but refer to the subpatch mode.

So my question, finally, is if there is workaround here? For example so as to...

  • define my own polyphonic subpatch mode some place (or hijack one that isn't used such as 'Mono with bypass')
  • save a subpatch to the library that has a changed MIDI code, and is still editable via the GUI
  • save an object(with changed MIDI code) to the library that allows editing its contents like a subpatch
  • ,,, ???

I realize some of these steps will involve messing with the runtime, I am willing to try that, but i'd need some pointers as to where in my axoloti_runtime/platform linux folder the pertinent files live.

Thanks!