Newbie Questions about Patch Loading, Preset saving, Polyphony, Midi Channels of standalone patches

#18

You can save values to SD card using tables. There are examples of how to do that, but with random numbers. I made a patch, with help form community that creates a bunch of random numbers created with a random object and then saved in a table. This can be recalled over and over again and it works. But I am ot sure how to use it for preset system. Cant figure out how to send the different table values to the different parameters. The random table object is only made for controlling ONE parameter, but with many values. It should work opposite, like many parameters with only ONE value for each(Or actually as many values as you want presets).

Link for saving random values patch:

I think one of the Alexs(There are two, I always, mix them up.... Sorry Alex) :smile: in here have been working on saving parameters without the editor. Not sure how he does it or if he got it working.

#19

Guys, I'm not saying its not important... just the only 'paid' axoloti resource is Johannes... so there are lots of goals.
...currently there are 92 posts in the improvements for the patcher UI alone, and many more posts around asking for improvements/more performance etc.

anyway, as I said its 'quite easy' to build this yourself if you are a programmer....
you will see in my Axoloti with Ableton Push video, I directly access and change the parameters completely standalone. this is done by accessing ObjectKvps in ui.c ... so that could be persisted using the fatfs libs to the SDCard.
(Im not sure we can write to the flash easily, Id have to check that, but I think personally id prefer SDCard)

the 'issue' with this, and why I have not pursued it (apart from also being busy) is some objects are not being stored there correctly, and also the current structure is (imo) too limiting. (it needs to be hierarchical, and have a bit more metadata) ... for me this is a prerequisite to any work in this area (including saving/loading) , the parameter code needs a bit of an overhaul which might invalid other work.
I've discussed this a bit with Johannes, but not enough to be able to just do it... as its pretty fundamental and could potentially break things, and i cannot put in my own private fork, as this would immediately be broken by other work that may be done in this area.

anyway, perhaps the show of interest, may mean johannes will turn his attention to this area.

#20

Man it was no critique of you or Axoloti or Johannes. We are just discussing options good or less good with what we have :smile: Or what would be nice to have :smile:

My main focus atm is trying to implement midi cc into the ctrl/i object. I tried once but with wrong program editing the javacode for that object. I think I got right program now so Ill try and edit the java code again and see if I have any luck.

Presets are a bit down on my list, since I dont really use it as stand alone a lot yet. When I have more patches finished the presets would become more important :smile:

#21

I think SDCard is the best medium for storing complete patch parameter snapshots.
There are 2 approaches:

  • simply writing/loading the "PExch" table to a file. This would save all parameters in the current patch, including on-parent parameters of subpatches (but excluding not-on-parent parameters of subpatches), in a binary format. It will only load well if the patch is exactly the same. Basically fairly easy to implement, but lacks preset-edit-ability from the GUI, and migrating such files when making changes to a patch.
  • writing parameter + value lists to a file, perhaps even in a human-readable text format. I think this is what @thetechnobear has in mind. More robust towards patch changes, but still need to be careful with unexpected behavior when loading an older preset into a changed patch. This approach has less chance to work without interrupting the audio computation, since it involves a lot of string parsing.

Both approaches touch an open area in the Axoloti framework. Currently the editor can only write a compiled patch to sdcard, but not open or connect to a patch running from sdcard, and keeping other data on the card (like parameter snapshots) in sync with the editor. The axoloti usb protocol allows uploading a file to sdcard but not downloading from sdcard yet.

#22

actually I think both approaches are possible...

the data could be stored in a binary blob, along with the patch version... this would allow for quick loading (no parsing) when the patch is the correct version. (the 'usual' case) , and additionally a more robust format (where parameters ids etc can change) which is used for migration.
the preset file involved would be pretty smalls, so 'duplicating the information' would not be too problematic.

of course, there is always a potential of audio interference, or even patch anomalies as the presets are saved and loaded due to a number of factors including file io... but I suspect these can be kept to 'reasonable' levels.

there are also lots of other approaches which can trade off memory, file io, cpu load etc... and which take into account when presets are loaded/saved ... e.g. when its performance critical or not.

#23

ughh im so busy with other stuff but cant wait to return to this question. I don't want this crap from Namm 2016..i want my own synth with presets :stuck_out_tongue:

#24

^ You may want to have a look here.

It's an implementation of the first method suggested by Johannes in his posting above. It seems to work fairly well, but adding, removing or changing modules in a patch may render the presets unusable. I.e. only use with finished patches.

If xpatch.cpp had an array of pointers to all the patch root structures, a full global store/recall would be possible. Could this be added? Or perhaps they can be gotten to from a module already, and I'm just missing it?

#25

wow that looks very interesting. Will study it a bit more, but if you have time could you tell me if this needs a specific firmware? Too caught up with my own code right now.

#26

I've tested the preset manager with 1.0.8 and 1.0.9 and those are fine.

It depends on a structure in the firmware, which could theoretically change and break at some version, but I think that bit is fairly stable for now. And if/when it changes [1] it might be to the advantage of the preset manager, which I plan to keep up to date.

[1] I'm advocating a tiny change: a table of pointers to the root struct of every sub patch in xpatch.cpp, to allow a global parameter load/save.

#27

@DrJustice just been taking a look at your preset manager (only code, not played with it yet )

for discussion purposes i.e. its not criticism, its just great to have someone playing in this area, so can discuss these things. ... also I'm interested if my understanding is correct :smile:

I notice your using PExch, Im assuming this is fully populated on the patch (I've not checked, but explains how your PM works)
as such, I think this is not really saving presets, but saving a full set of current parameters values. presets are stored value in 'p' (i know not best name!) and are a subset of the parameters. so 'theoretically' it should only save those, and also I think it should populate 'p', since otherwise if a user does a recall of a preset later, they will not pick up the values that they earlier loaded.
I raised, this as potentially, there is going to be confusion, between the interplay of presetmanger and recall/storing of presets ... and frankly, I think presets already confuse many (including me :wink: )

also... and this may not matter... Id assume PExch changes as your patch changes... so does any patch change invalidate the preset stored?

of course my understanding may be incorrect, as I took a different approach on Push (as I wasn't storing the data), I used the global object parameter table (ObjectKvpRoot) , and just presented this to the user.
however, this also has 'issues' as its only the underlying parameters, so it doesn't have any knowledge of a parameter being on a 'subpatch', no issue for storage, but means from a UI perspective, you cannot show parameters that have been 'placed on the parent'
of course, the advantage for display, was also it has details about max/min , which aren't in PExch... unfortunately I dont think there is a relationship between PExch and the main object table (as far as i remember when playing with this)

here kind of illustrates where the two worlds divide:

 parent->PExch[PARAM_INDEX_square__3_pitch].pfunction = pfun_signed_clamp;
 SetKVP_IPVP(&KVP_instancesquare__3_pitch,ObjectKvpRoot, "square_3" ,&parent->PExch[PARAM_INDEX_square__3_pitch], -1<<27, 1<<27);
 KVP_RegisterObject(&KVP_instancesquare__3_pitch);

thats this shows, is the global table knows about the parameters and all its attributes but only knows at PExch instance, it has no knowledge of where it comes from e.g. parent/Pexch idx , similarly parent->pExch has no understanding of the actually parameter that is created in the global parameter table.

anyway, this is a bit fyi and discussion only... alot of this matter only when building controllers which want to expose the parameters in a sensible way to the user. but I think some of it is 'interesting' to storing patch state.

note: the global parameter table has the object name, which is what I kind of referred to above, this name only changes if the user changes object names, rather than patch structure.
I fear PExch and the index in ObjectKvpRoot are both based on the order things are built in a patch, which in turn is position on the patch canvas vs other objects. (though Ive not verified this)

anyway, perhaps this should be a separate topic, but i thought Id start here... as may not interest you at all :wink:

#28

Ah yes... that's an area open to confusion. These 'presets' are not the same as the original Axoloti presets (please add me to the ranks who gets confused by that - I'm used to the conventional meaning of 'preset' in synths). And since the original Axoloti presets aren't parameters as such, they don't get saved. PExch is fully populated AFAICS, and tests confirm this.

So, this thing should probably have another name, but what should that be? For what it's worth I think 99% of users will expects 'presets' to be a total store/recall of all parameters. So we have patches (which in fixed architecture synths are also called presets), original Axoloti presets, and my attempt at 'conventional' presets.

I assume the above explanation, makes this half redundant, but: Changing the parameters is fine, that's what the preset manager is meant to capture. Adding/removing/changing modules in a patch will invalidate the presets previously stored for that patch. Changing the placement of modules reorders PExch, so that should be avoided too - I strongly suggest that Axoloti implements an iteration order based on the topology of the patch, not the placement of graphical placements (you can have an algorithm from me if you wish). Therefore this PM module is best used for finalized patches, but even so I use it for patches I'm working on just to switch parameter sets (aka recall presets...) during testing. It sure beats having no conventional preset storage at all.

OK - that's something I will look into. If this can help me read and write all parameter values in all sub patches, I'll use that. Otherwise, just an array of pointers to the root structures of each patch would allow total store/recall of all patch parameters.

The issue with presets getting invalidated by patch structure changes could be avoided by having access to patch/module/parameter metadata, such that named parameters could be stored for named modules in a named (sub)patch. That metadata would eat RAM though - best placed in the SDRAM rater that the SRAM then.

In any case, I believe that (conventional) preset save/load is a very important feature that users will expect, so let's find a reasonable implementation and just do it! :smiley:

#29

yeah the presets in axoloti are a very nice feature, but perhaps need a better UI metaphor and tidying up...
they have many advantages, but I grant they are not how we think of presets.

I agree, but when I was writing this, I couldn't think of anything either.

my fear, is this will get users asking for the current presets to be a snapshot (as your PM is) , which we have had already, but I think thats wrong (its been discussed many times here already) , as the current system whilst perhaps not perfect/familar is very useful.

perhaps snapshot manager? is it Reaktor that calls these things snapshots?

yes it does that, as it stores all parameters.
(Ok, Ive seen a couple of objects that do not appear to be registering the the parameters properly... I can't remember which)

again the above table has a name, which makes it less fragile... and I believe this name has to be unique, though its worth checking this. if you look at my Push code, from memory i display it, with the parameter name and the object name its within (IIRC) ... so hopefully forming a FQN.

However... I will repeat, this does all need improving... but making it so its backwards compatible, and adding the extra meta data is no small task (also has to be kept small, as its part of the runtime)
... there is also another possibility, which is to keep this metadata separate, but along side (an in step with) the patch binary... such that the axoloti core doesn't have to load it, but its available for other processes to decode , think, like debug symbols. this might be better, since potentially there is alot of useful meta data, that only editors/controllers need and that for runtime axoloti does not need.
(axoloti runtime, could deal merely with a single index, doesnt care about the rest)

as I said, I suspect when Axoloti Control is being finalised, it will also require this area to be improved, as it will face the same issues when wanting to display data to the end user in a logical fashion.

#30

I've followed that up now, and I can't make sense of the ObjectKvps table. So far, from what I can see it only has top level patch parameters registered, and then not all of them as you mentioned, even though I see the registering calls is xpatch.cpp. Also, the registered parameters are seemingly not consistent with the SetKVP_IPVP() and KVP_RegisterObject() calls in xpatch.cpp, among other things I can only retrieve the keyname of a few of the ipValuePair structs that are registered.

There is of course a good chance that I'm doing something wrong. The code below is what I use to inspect ObjectKvps ATM. It's executed at the first iteration of the preset manager module (BTW, I'm thinking of renaming it to "snapshot manager" as you suggested, to avoid confusion). 

if( start_sequence == 0 ) // Only do this on the first iteration of the module!
  {
    extern struct KeyValuePair *ObjectKvps[];

    LogTextMessage( "preset_manager::krate(): ObjectKvpRoot = %08x, ObjectKvps = %08x, n = %d", ObjectKvpRoot, ObjectKvps, ObjectKvpRoot->apvp.length );

    for( int i = 0; i < ObjectKvpRoot->apvp.length; i++ )
      {
        switch( ObjectKvps[i]->kvptype )
          {
            case KVP_TYPE_IPVP :
              LogTextMessage( "KVP_TYPE_IPVP: .keyName = %s, .value = %08x",
                ObjectKvps[i]->keyname != NULL ? ObjectKvps[i]->keyname : "NULL",
                ObjectKvps[i]->ipvp.PEx->value );
              break;
            case KVP_TYPE_IVP :
              LogTextMessage( "KVP_TYPE_IPVP: .keyName = %s, .value = %08x",
                ObjectKvps[i]->keyname != NULL ? ObjectKvps[i]->keyname : "NULL",
                ObjectKvps[i]->ivp.value );
              break;
            default :
              LogTextMessage( "KVP_TYPE_ = %d", ObjectKvps[i]->vptype );
              break;
          }
      }

I'm running:

Axoloti version: 1.0.9
Firmware version: 1.0.0.1, crc=0x21C46C25, entrypoint=0x20011000
Host OS: Windows 7 64 bit

#31

Its a bit of time (when I was doing the push stuff) since Ive played with this.

as far as i remember though...
you will see all parameters... just remember when you 'place on parent' you are really just exposing the same underlying parameter (i.e. the one in the sub patch) , so that can be confusing.

also as I said, I noticed some objects are not correctly registering their parameters. from memory this was some of the sel type objects. (this is something I think we should fix.)

perhaps try the following...
create a patch with dials in the top level, and also dials in the subpatch (name them all, so you can see whats what).
and see if you get all the parameters....

hopefully, then what we can do is go the other way, and narrow down, what is not being registered... and see if we can do this. i suspect its possible , perhaps you chose a different set of objects to the ones i did.

at some point, I'll create once to just print the stuff out as you did, as I cant use my push object now (traded in for a push 2)...

unfortunately, as I said in my first post, its definitely 'not right' at the moment, there are 3 different structures (albeit related) doing things in this area... and I dont think any are vary consistent, or have the information we need.
the reason for raising this, is I suspect, we will need to tweak it all.... and the last thing I would do is create another one (e.g. adding the tree info your requesting for PExch) without reviewing it all.
as I said, Id put it on hold really, as I know Johannes will also need this updating for the Axoloti Control... and when we discussed it before, he did say the ui bit in firmware, was really just provisional/prototyping until Axoloti Control gets developed properly....
BUT... I thought its useful/important for you to know a little more about how parameters are handled... as Im sure you will have some good ideas for the future too.

#32

Absolutely tb - all info is greatly appreciated :smile:

The addition I suggested for xpatch.cpp is simply a list of pointers to each patch root structure. A simple NULL terminated array would do fine. The metadata stuff we briefly discussed was more theoretical.

From looking at ui.c, ui.h and xpatch.cpp, it seems to partly be a work in progress for the remote. If it's the intention that ObjectKvps should be a registry for all parameters, that will be a great thing to have. However, right now it looks like it's incomplete and can't be used for this.

My test scenario consists of a patch with one sub patch, and I'm only seeing some of the top level parameters in ObjectKvps - still with the caveat that I could be doing things wrong.

I've added the test scenario to the library, so that you or anybody else who knows the firmware and xpatch can have a look and see what's going on:
patches/drj/test/preset_manager_test.axp,
objects/drj/test/preset_manager_test.axo

#33

Of course I had done a silly mistake - It's looking quite good now. :blush:

However the ctrl/i (integer spinbox object) and ParameterInt32Box (object parameter) do not appear to be registered. In xpatch.cpp a KeyValuePair is defined locally for them, but SetKVP_IPVP() and KVP_RegisterObject() calls are never made. Perhaps this could be the case for some other obejcts too? OK this is close, but the cigars will have to remain in the humidor for now.

#34

cool... I started doubting my memory for a moment there

yeah, theres a few of these... I can't remember which ones at the moment...
I need to determine if there is a good reason for this, or if its just an oversight, or perhaps something that was added later and not applied to everything.

perhaps if you maintain a list of the ones you spot... then it will give us something to work off of.

#35

Aye! I shall keep notes of anything missing. I'm writing a new version now, will have something to test later today.

Perhaps it's an idea to split off this part of discussion into a new thread, starting from here?

And don't doubt yourself, tb! You're the veteran and I'm the noob with only a few hours of system familiarisation so far.

#36

A new version of the preset manager is up, please ref. my contribution thread for a description.

The functionality is now very close to 'conventional' presets. Just need two things to make that happen:

1 - Get those currently non-registered parameters fixed. I still have only spotted the integer spinbox, but I haven't been searching for that (too busy programming...).

2 - A list of sub patches, with something to identify each sub patch, e.g. just a sequence number or something like that, so that name clashes in the global save mode can be avoided (that is equally named parameters in different sub patches). I.e. an array of sub patch pointer/identifier pairs.

#37

looking forward to finally try it out. been catching up on things :smile: