Accessing parameters in subpatches

[ricard]

Ok, this might be a bit esoteric, but I'm working on an editor object inspired by @thetechnobear's Push object, which will allow me to edit parameters on the Axoloti using a connected LCD, encoders and buttons.

I'm using ObjectKvpRoot->apvp.array to find the key-value pairs for the parameters. When I want to change the value of a parameter, i then call KVP_Increase() and KVP_Decrease() in ui.c in the firmware, and poll bit 0x4 of ObjectKvpRoot->apvp.array[foo].ipvp.PEx->signals if any changes made elsewhere (e.g. in the Patcher) are to be displayed.

This works fine when the parameters are in the same patch as the editor object.

However, I'm having problems when I'm trying to edit a subpatch which has a couple of exported parameters. One thing I've noticed is that the list of available parameters in ObjectKvpRoot->apvp.array seems to reflect all parameters in the subpatch (and the top level patch, at least objects that are not subpatches), not just the ones seen in the top level patch.

Secondly, and this is probably related to the first, while changes to the parameter values that I make in the Patcher in the subpatch object are picked up by the editor object (just as when there is no subpatch), the reverse is not true, i.e. calling KVP_Increase()/Decrease() for the key-value-pair corresponding to a given parameter doesn't have any effect (at least not on the parameters in the subpatch object).

I suppose there is a logic here, in that when a change is made to a top level subpatch object, the changes are reflected in the subpatch itself, and it is the subpatch parameters that the editor object is picking up, whereas when making parameter changes, the editor object sends these down to the subpatch directly, and those changes are not reflected in the top level subpatch parameters.

So is there something else I should be doing in order to communicate with the exported subpatch parameters, rather than with what seems to be the subpatch's parameters directly?

[thetechnobear]

the parameters in the table are only the underlying parameters, so yes the ones in the sub-patch.
I had noted the same problems when I did the push object...

I think the current state, is we know parameters handling needs to be reviewed, and axoloti control will provide that impetus... basically, a few of us are starting to now come up against this, due to our work with various controllers etc. (when I did this push... it was just me, so not really a priority, more a future consideration)

e.g. I know @DrJustice is also interested in this area, due to his work with the preset manager.

sorry, no solution...

[johannes]

Moved this to the developer category. This area will get reviewed soon. I the meantime discussion about the required changes/improvement is appreciated.

[ricard]

the parameters in the table are only the underlying parameters, so yes the ones in the sub-patch.
I had noted the same problems when I did the push object...
[ ... ]
sorry, no solution...

But I suppose that means that when making changes to the corresponding parameter values, the changes do actually take effect in the underlying subpatch, they just don't show up in the Patcher UI of the top-level patch? (Yes, that should be easy enough for me to confirm now that I've got everything in place). For me that's a sufficient workaround for now, although it looks like I have to change multiple parameters at the same time if the subpatch has a polyphony > 1.

My main goal at the moment is to create patches that will be able to run standalone without the Patcher UI, so that the Patcher will be used solely for design, and the actual maniuplation of the patches will be done using an editor object, the idea being to create an instrument with its own complete UI.

[thetechnobear]

sorry, its a while ago since I discussed this with Johannes, when I was doing the push object.. and I don't have the push anymore to test what the actual behaviour was.
( I remember johannes mentioning polyphony does confuse issues)

My main goal at the moment is to create patches that will be able to run standalone without the Patcher UI,

Mine too ... and also Axoloti Control needs this, so we will make it happen.

what I had started to do, 'in the meantime' , was to add inlets to sub patches rather than use the 'parameter to parent' and to control that way. personally, because I want to use axoloti without the UI, Ive generally preferred using modulation inlets over parameters, so it was not a big move for me.
if there are some parameters you need to use (as not all are provided as inlets, again something we want to change), then you can use mod sources to workaround.

its not ideal, but meant I could carry on...

as you can imagine parameters are an important aspect of axoloti, so rushing a change through on them, would probably cause more damage than good.

anyway, as Johannes said , now is a great time, to highlight some of your wishes for improvement and requirements for the 'redesign'

my personal ones are... (again from memory, I do need to check this out again!)

• more meta data,
  • make it easier to attach parameters to objects
  • be able to determine hierarchy of objects and sub patches
  • more type/units nfo
• ensure all parameters are registered (some are missing)

perhaps it would also be nice, to be able to review/amend the cc mappings, on the fly to from another controller. use-case : axoloti control mounted to axoloti, select parameter, hit 'midi learn', turn knob on my external keyboard to map it.
(i say perhaps, as I do think, we have to becareful to keep a boundary between performance control vs design... as there is only so much we can do on the fly)

[ricard]

what I had started to do, 'in the meantime' , was to add inlets to sub patches rather than use the 'parameter to parent' and to control that way. personally, because I want to use axoloti without the UI, Ive generally preferred using modulation inlets over parameters, so it was not a big move for me.
if there are some parameters you need to use (as not all are provided as inlets, again something we want to change), then you can use mod sources to workaround.

I'll have to look into that, that's a good idea. And if one still would want them as parameters, you have parameters on the top level, which are simply constant values feeding the appropriate inlets.

I have been thinking that sometimes one would like to have a different value range exposed to the UI than the actual parameter has, and using inlets would be a way to accomplish that.

as you can imagine parameters are an important aspect of axoloti, so rushing a change through on them, would probably cause more damage than good.

Yes, i agree. I'm not expecting any changes to happen quickly, just trying to offer some input.

anyway, as Johannes said , now is a great time, to highlight some of your wishes for improvement and requirements for the 'redesign'

At the moment I have very few suggestions apart from the ones already mentioned in this thread. I'll need more experience with aspects of the Axoloti such as modulation and parameter storage which I haven't even looked at yet.

One thing though I would like would be to have the API for the Control accessable for any UI that one would want to construct, like the Push or as in my case the P6. I think the Control will be a great complement to the Axoloti, but a custom user interface might be something that more people would want to design and also conversely something which more people would be able to design, compared to the Core. So having a well-defined UI API would be a Good Thing.

[ricard]

Ok. Couldn't leave this one alone ... it struck me that somewhere there must be a list of available parameters at the top level of a patch. And in fact there is, patchMeta.pPExch[] contains a pointer to an array of ParameterExchange_t's which correspond to the top level parameters (there being NPEXCH of them).

The way the Axoloti works currently is that for parameters that belong to subpatches, the pfunction member of the ParameterExchange_t contains a pointer to PropagateToSubpage (so we can test this), and in this case the finalvalue member contains a pointer to the corresponding ParameterExchange_t for the subpatch. This one in turn has a pointer to the corresponding ParameterExchange_t for the individual voices in the subpatch in its finalvalue member.

Using this information, we can construct a top-level KeyValuePair for each subpatch parameter, by extracting the name, min and max values from the corresponding KeyValuePair of the subpatch (which can be found among the global parameters in the ObjectKvpRoot). Each voice will have its own ParameterExchange_t, but since they have the same name, min and max values, we can extract that information from just the first voice.

For parameters which are in fact ordinary parameters at the top level, we can just take a pointer to the corresponding KeyValuePair.

This way, we construct an array of KeyValuePairs corresponding to the top level parameters.

I did a quick test of this with a polyphonic subpatch and it seems to work nicely. Twiddling a knob causes the LCD dispay to update, the Patcher UI to update, and the parameter in all voices to update. Conversely, twiddling a knob in the Patcher UI has the same effect, including updating the LCD display with the parameter values. Ultimate success, so far.

Admittedly, some of this will probably break if the parameter architecture is reorganized (and there might be something i've missed and haven't realized), but for now, there is actually a workable way of creating KeyValuePairs for the top level parameters. Furthermore, it can all be done with publically accessable variables; no need to add extra extern declarations, or modify the Patcher code to export existing static variables.

[thetechnobear]

ah, interesting....
I knew about the pExch stuff, and this was the approach taken by @DrJustice on the preset manager.
what I hadn't found was a way to link the two worlds of pExch and ObjectKvpRoot.
this is really handy, as you say pExch gives the structure (sorely lacking in the ObjectKvpRoot), which ObjectKvpRoot gives you the info.

does this also help for those things that are missing in ObjectKvpRoot?
definitely need to have a play with this.

but for that , I need to get my new controller stuff working on the Push2... to test, this, although this faces potentially another challenge...

As the Push2 will be a different device (a PI2/BBB) , my current thoughts are to connect via the Axoloti USB bulk interface e.g. the same way as the GUI gets the data.
this of course means I get exactly the same data as the GUI... but I fear the GUI gets nothing other than data from the board.. because it has the meta data from the patch.
this leaves me with a horrible dilemma

a) enhance the protocol to send meta data

b) load the patch... process in C++
my preferred approach, but we have no xml schema, and it might be a large task decoding the xml, to process what attributes are on what objects, and how they are structure in the patch

c) load the patch ... process in Java
ok, so this approach uses the existing code base, but creates a much smaller lighter interface, perhaps even then just retransmitting data via OSC.
Its a bit heavier than id like... i don't really want to be running java on a PI/BBB...
but has quite a few advantages, including potentially for patch compiling/uploading.
(again possibly with (a or b) but significantly more work)

i kind of touched on this in December

Controlling axoloti via USB

this topic is to discuss options of controlling Axoloti via the USB device port. (though arguably the same method could be used for the USB host port) primarily this is for my own project, but could be useful for axoloti as a whole or other developers doing similar things, so perhaps there is an opportunity for collaboration. the basic idea is for a hosting device (in my case a PI2/Beagleboard) to be able to control the axoloti board, in particular my goals are: parameter control seque…

I guess this is all a bit off-topic , but in some ways, I thing the axoloti binary protocol, is actually quite linked to this topic of parameter display/change and patch meta data.

... anyway, Im working hard on the PI side at the moment with my other controllers, but once thats finished I will move my focus back to this area.

[DrJustice]

I use both use both ObjectKvps[] and PExch[] in the preset manager. Here are my observations and thoughts regarding this issue:

• ObjectKvps[] is missing some parameters (notably the integer spinboxes), otherwise it's "the one" for a blanket global parameter access since it has the parameter names and is therefore part resilient against patch changes.

• ObjectKvps[] has pointers to the PExch[] entries in the sub patches which enables x-ref'ing between the two.

• PExch[] has all parameters for a sub patch, but no names.

• PExch[] will shuffle it's parameters around as modules are moved in a patch and is therefore not reliable in itself.

This makes for some challenges. Currently the preset manager supports three methods of parameter access.:

• Global: uses ObjectKvps[] and is partly resilient against patch changes, but it's missing the integer spinboxes. Only need one preset_manager for the whole patch.

• SubPatch: uses both ObjectKvps[] and PExch[] to find the named parameters for a sub patch, partly resilient against patch hanges, but it's missing the integer spinboxes. Needs a preset manager placed in each sub patch that one wants parameter saving for.

• SubPatchV1: The initial method, uses PExch[] only and isn't missing any parameters, but patch changes invalidates the save files. Needs a preset manager placed in each sub patch that one wants parameter saving for.

Since a module can only access its own parent->PExch[], there is a need for a preset_manager module in every sub patch if one wants to isolate the parameters belonging to that specific sub patch. Therefore I still lobby for an array in xpatch.cpp with pairs of sub patch rootc pointers and a unique and a stable id. The id can be a simple sequence number that gets assigned when a sub patch is added.

In desperation I have made a partway solution to the missing sub patch table: I place a preset_manager in the top level patch and a preset_slave module in each sub patch that I want to know the rootc address of. On the first iteration of the patch, the preset_manager creates a file and registers it's parent rootc pointer and its parent->NPEXCH. On the second iteration, each preset_slave module appends its pair of parent and parent->NPEXCH to the file. On the third (or later) iteration, a full sub patch parent pointer table is then available in the file for preset_manager to read and use on save/load operations. The parent->NPEXCH is stored so that the preset_manager can verify this number to give some resilience against patch changes. This method still fails if the iteration order is changed, by moving sub patches around. It's an indication of how desperate I am for full (conventional) preset management system

The details can be seen in the preset_manager.axo source. The preset_slave.axo has not been published - I consider the method of using the rootc file registry to be too hacky to publish, especially since it can't provide a resilient patch storage.

To round off, part of the problem is the iteration order being determined by graphical placement of the modules. A simple graph traversal with a sequence sort-number added to each node is enough to give a stable, topology based and more "correct" iteration order. It would not only help the preset management (that is better served with some tables and/or meta data though), but is a "must" for a module synth anyway IMHO (could be offered as an option for those who have critical iteration orders established by way of graphical module placement) .

[ricard]

• ObjectKvps[] is missing some parameters (notably the integer spinboxes),

Do you mean const/i objects? Are they parameters, I thought they were attributes, i.e. not editable when the patch is running.

To round off, part of the problem is the iteration order being determined by graphical placement of the modules. A simple graph traversal with a sequence sort-number added to each node is enough to give a stable, topology based and more "correct" iteration order. It would not only help the preset management (that is better served with some tables and/or meta data though), but is a "must" for a module synth anyway IMHO (could be offered as an option for those who have critical iteration orders established by way of graphical module placement) .

Could one not base parameter ordering on some other property, like a hash of the full parameter name (i.e. the instance name + the parameter name, e.g. 'lp_1:freq'). That would make it independent of graphic position, but would not help if you add another parameter.

[DrJustice]

Do you mean const/i objects? Are they parameters, I thought they were attributes, i.e. not editable when the patch is running.

I mean the "int32" parameter type which displays a spinbox, used in e.g. the ctrl/i object.

Could one not base parameter ordering on some other property, like a hash of the full parameter name (i.e. the instance name + the parameter name, e.g. 'lp_1:freq'). That would make it independent of graphic position, but would not help if you add another parameter.

That's how I use the name; storing a name-hash/value pair for each parameter for the Global and SubPatch modes (there's a handy CRC32 function in the firmware). However, using just the global method, there's no way to distinguish between e.g. 'lp_1:freq' in two different sub patches. That would need the mentioned global sub patch table.

I'm very close to having a complete and robust preset management - the xpatch.cpp sub patch table looks like the simplest solution for full, graphics independent, parameter access, provided that the spinbox and any other missing parameters gets included in ObjectKvps[]. I'm still in favour of graph (not graphics!) dependent iteration order, even if preset management potentially can (and should) be achieved without it

Great piece of debugging work on the linker text placement, BTW

[ricard]

Do you mean const/i objects? Are they parameters, I thought they were attributes, i.e. not editable when the patch is running.

I mean the "int32" parameter type which displays a spinbox, used in e.g. the ctrl/i object.

Ah! I haven't noticed that one before. Much of a noob at this still I'm afraid...

However, using just the global method, there's no way to distinguish between e.g. 'lp_1:freq' in two different sub patches. That would need the mentioned global sub patch table.

One way would be if the name of the parameter were augmented with the name of the subpatch, so that when the parameter of a subpatch is exported to its parent, it is called e.g. 'subpatch_1:lp_1:freq'. It also seems to me (without having tried this out in any detail) that the parameters one wants to be able to store the values of should be exported to the parent, with the ones that aren't simply having fixed values.

Great piece of debugging work on the linker text placement, BTW

Thanks!

[DrJustice]

One way would be if the name of the parameter were augmented with the name of the subpatch, so that when the parameter of a subpatch is exported to its parent, it is called e.g. 'subpatch_1:lp_1:freq'.

Yes, that's what could be achieved with the sub patch array I xpatch.cpp. It would practically form the first part of that path - just a sequence number for each created/inserted sub patch would be enough for this purpose (SRAM is precious). One still has to cross reference the global ObjectKvps[] and each subpatch PExch[] to determine the parameter name and owning subpatch, but that's fine.

It also seems to me (without having tried this out in any detail) that the parameters one wants to be able to store the values of should be exported to the parent, with the ones that aren't simply having fixed values.

You mean using the "parameter on parent" option to bubble parameters up to the top level patch? I think there's no single paradigm that will fit all users and situations.

In some cases, it wouldn't matter much for either time or space if every single parameter in the patch/subpatches were saved, in other cases you might not want to save the parameters for a specific sub patch, e.g. a reverb that's being tweaked and improved in its own sub patch file. Also the possibility of saving the parameters for each sub patch separately is interesting; the settings for your synth voice goes in one file, the settings for your effect patch in another file and so on (this is what the SubPatch mode of the preset_manager does).

One thing I have to look at is polyphonic sub patches. Is it so that a parameter in a poly subpatch is instantiated for each voice? If so, I hope there's a single point at which all can be adjusted, and only one instance needs to be saved and loaded.

[ricard]

It also seems to me (without having tried this out in any detail) that the parameters one wants to be able to store the values of should be exported to the parent, with the ones that aren't simply having fixed values.

You mean using the "parameter on parent" option to bubble parameters up to the top level patch? I think there's no single paradigm that will fit all users and situations.

Yes, that's probably very true.

One thing I have to look at is polyphonic sub patches. Is it so that a parameter in a poly subpatch is instantiated for each voice? If so, I hope there's a single point at which all can be adjusted, and only one instance needs to be saved and loaded.

Yes, each voice has its own set of parameters, and they all show up in the global ObjectKvpRoot->apvp.array[] . However, for each subpatch parameter there does seem to be a single ParameterExchange_t whose pfunction is set to point to PropagateToVoices(), which when called as a result of a parameter change distributes the change to the different voices.

I don't know to which array these 'PropagateToVoices' ParameterExchange_t's belong, but i do know that at least for parameters that are exported from a subpatch, at the top level the exported parameters are present in the global PatchMeta.pPExch[] array. Each of the parameters which are in fact exported subpatch parameters have their pfunction set to PropagateToSub(), and the finalvalue member of the ParameterExchange_t is actually a pointer (but cast to an integer) to a ParameterExchange_t which has its pfunction set to PropagateToVoices().

(The ParameterExchange_t's which have pfunction set to PropagateToVoices() use a similar method, where the finalvalue member is actually a pointer to the the ParameterExchange_t's for the first voice, and the rest of the voices are found by stepping from the first one.)

So yes, there is a central point for each parameter which will distribute it to the respective voice parameters, but it's a bit awkward to find.

I mean the "int32" parameter type which displays a spinbox, used in e.g. the ctrl/i object.

I checked them out now. I wonder what they are for really? They are a bit special since the don't have any (practical) bounds to their values, so for instance you can't map a CC to them. Would they be used to for instance set the starting or loop point in a sample so you could adjust it in real time? Most things that you'd want to control don't require that large a range, and a normal ctrl/dial would do.

[DrJustice]

Thank you for unravelling the poly parameter handling! That should give me enough to go on.

The int32 parameter, called ParameterInt32Box in the object editor parameter type selection menu, is bounded if you add in MaxValue and MinValue tags:

<int32 name="preset" description="Preset number to save or load">
  <MinValue i="0"/>
  <MaxValue i="999"/>
</int32>

Those tags are also seen in e.g. in the "sel/sel fb 16" object in the frac32.s.mapvsl parameter types, but when I adopted that and changed the bounds, they had no effect. I also tried the MaxValue and MinValue tags on other parameters, but they had no effect there either.

<frac32.s.mapvsl name="b0" noLabel="true">
  <MinValue v="0.0"/>
  <MaxValue v="64.0"/>
</frac32.s.mapvsl>

BTW, @devs : when trying to add ParameterInt32Box and ParameterInt32BoxSmall from the object editor I get a java error - it has to be done by hand. This is the error (Axoloti 1.0.9, Win 7 64bit):

org.simpleframework.xml.core.ElementException: Value for @org.simpleframework.xml.Element(name=, type=void, data=false, required=true) on field 'MinValue' axoloti.datatypes.ValueInt32 axoloti.parameters.ParameterInt32Box.MinValue is null in class axoloti.parameters.ParameterInt32Box
org.simpleframework.xml.core.ElementException: Value for @org.simpleframework.xml.Element(name=, type=void, data=false, required=true) on field 'MinValue' axoloti.datatypes.ValueInt32 axoloti.parameters.ParameterInt32Box.MinValue is null in class axoloti.parameters.ParameterInt32Box
...java stack dump follows...