Live coding/patching

#1

Note: this is a 'pie in the sky' topic, just a discussion...
(i.e. dont get your hopes up on it being implemented :smile: )

Ive been recently playing with Extempore (http://extempore.moso.com.au), and quite like what it has to offer, and wondered how something similar might be possible be offered on the axoloti core board,
and direction we might go to add some of its ideas to axoloti.

(you could argue, SonicPi and many others offer similar... though many of these rely on Supercollider)

so there are two parts that interest me:

  • hot swapping code
  • other "user" interfaces

(maybe later I'll split these into two separate topics, if there is interest in discussing both)

hot swapping code
the most obvious here, is to do like (apparently) the nord modular... and allow patches to be edited live, substituting in new patches on the fly.
of course, many issues in doing this, to ensure audio is uninterrupted.

Im kind of interested in, how would this be done (approximately, I know its complicated)
what granularity is used? e.g. you could imagine, and entire patch being substituted or individual objects.
the later seems problematic, since you cannot optimise the whole code if you do this, only down to object level, and the STM32 probably needs this optimisation.
how do you keep state? a particular problem if you do an entire code swap....

does something like MCJIT with LLVM handle any of these issues?

is this all or nothing?
e.g. I was thinking of an application, where actually most of the patch is pretty static, but I should like to alter some some elements (e.g. say some calculations) ... something that requires a bit more than just changing values, e.g. introducing new calculations, but does not require changing the call hierarchy.

(random thinking: like changing a pointer to function... I just need to introduce a new function... hmm, Im thinking perhaps this is possible in a similar way to the way a patch currently works. having a special 'upload' section, and then being careful about how I switch between code i.e. you flip between an active code, and uploaded code, at a safe point e.g. next k-rate run.)

other user interfaces
initially, I was thinking about GUI vs languages (these lisp ones are quite nice).
this are interesting, for different use-cases to a graphical environment...

this one is easier to imagine applying to axoloti, and has previously been discussed a bit in this thread , the 'obvious' solution is to create a patch file, and compile it... and some clever programming/language bindings could even expose axoloti objects.

I also wondered about the 'nature' of the interaction between axoloti and a 'client computer'.
e.g. imagine axoloti hardware as been a generator (like supercollider) that is externally told what to do...

I guess this is not dissimilar to what axoloti is currently doing... which is running a patch, that its been created on the computer until is told to stop* and of course, if you have 'live coding' principles, with hot swapping of code, then that relationship between computer and axoloti becomes a more 'continuous one'

(* even running standalone is the same, at some point a computer told it what to do... its just it can carry on doing it autonomously, surviving disconnection from the computer, and even power cycles)

Saving/loading module-code as text?
#2

+1 as a micromodular user ....... live editing (and undo) would save so much time

#3

Yeah, I know live editing is wanted/desirable.... don't need any upvotes :smile:

really I want to just get a discussion started on, what steps are required, and are there baby steps we can be taking to move us this way.
e.g. some of this is potentially related to moving to clang (/ libclang)

#4

Moving to libclang is the first step.
Without interrupting audio, I think it is not possible, some extra time will be spent on saving states and restoring states, generating extra load.
Hotswapping partial code - like a single object - implies the use of function pointers so they can be modified, but non-inlined functions can add significant overhead. Modifying inlined code is
I'm not sure if saving/restoring state is really needed, I think accelerating compile/upload/init would be good enough. Saving/restoring states adds a risk of getting into a state that is not reached anymore by re-initializing a patch.
I think the G2 did not save/restore states when you add an object to a patch.

#5

Yeah, I assumed inlining makes it an 'all or nothing', and similarity I've seen talk that llvm jit produces non optimized code.... and agreed functions generally have a context overhead.

But perhaps this is where trade offs can be made, having functional call at sample level, is obviously a no go, but the a few function calls at k-rate , with caveats may be not too much.

Of course excessive use will cause cpu load, but perhaps you are willing to trade that for other areas of patch complexity.
( perhaps you one f call does a lot of work, that would otherwise be split over lots of objects inefficiently. )

Audio interruption is more a concern, though we are already pushing data back and forth, perhaps as long as the swap is done over multiple k-rate cycles, it can be reduced, so no one k-rate cycle causes an overrun.
( the actual code switch is merely a pointer change , it's the data transfer I see as the issue.)

Of course there also here , is a requirement that the patch leaves enough memory to have two versions of the switchable code. Again a compromise of resources

I'll admit though , the more constraints I place on how and when code can be switched, and how it can interest with the rest of the patch , perhaps the less useful it becomes.

Hmmmmmm.

#6

You might already have found/heard of this.. http://chuck.cs.princeton.edu/ . Although it would need a BeagleBone or Rpi under Linux.. this can do plenty in terms of live coding.

#7

yeah I'm familiar with chuck, though haven't worked out its advantages over and supercollider (both seem popular, and I'm aware of the timing advantages... just unclear on ease of use) ... I have a PI and a 2 BBB (which I will be adding Bela too)

here, though I was interested, in how we might apply some similar concepts to axoloti