Future GUI framework development


#1

I keep getting back to this idea that actually the Netbeans Platform would be a great fit for the patcher:

  • It's in Swing, the development already happens in this editor
  • There are palettes and a graph editor.
  • Moreover,it would be possible to open the patch c-code in an advanced editor with code highlighting etc.
  • It has a module distribution system, with update notifications, which could be used for patches or even the complete build environment

I guess it's a bit of a learning curve, but if I understand it correctly you can use the parts you want and leave the stuff you don't need out.

What are your thoughts?


Improvements/Wishes for the patcher
#2

It's an interesting idea, but I think we'd spend a huge amount of time filtering out features we don't really need. Much of the core model/rendering/usb code would remain the same. It doesn't really buy us anything but brings a huge amount of bloat. The question of the having better syntax highlighting for axo object C code is an interesting one. I think the current design of having the axo object C code embedded inside of an XML file is a mistake. This makes it very difficult to edit using any standard editor mode. For example, if in an editor's XML mode, the C is not highlighted properly. If in C mode, the XML is invalid and the highlighting breaks. It also obscures what names are visible because there is so little context (showing includes for example would make it obvious to the object author what names are visible). In my opinion, the raw axo object source code should be stored in a separate file that is pure C/C++, so it can be edited properly outside of the patcher. If the file format is sane, achieving basic highlighting and completion in the object editor would be easier, and we could do that with a much smaller dependency than Netbeans. The design should make it convenient for power users to use their real editors but also support the "casual" object editor approach.

On the point about Netbeans being used for development: having a hard requirement to use a particular IDE is a huge mistake. It scares away potential contributors. We're not totally there yet (to having a hard IDE requirement), but we're close. It's perfectly possible to develop and build most patcher code without it, but we rely on the Netbeans UI editor for some of our UI views. I find it totally painful and infuriating to use. I'd prefer keeping as much as possible in a human editable text format and keeping things buildable from the command line. This lets developers use their preferred tools as much as possible. If someone wants to use an IDE, great, we should support that but not require it.


#3

Johannes has been considering an 'annotated' C++ for the object for some time, which is also my preferred route.
that said, Im not sure this is what all users would be comfortable with... it is useful having the editor as it means you dont have to remember thing like inlet types etc.

I agree about netbeans, probably is good for somethings, but its a lot to bring in for not a huge gain.

btw: I think probably this more technical things, should be raised in the dev section, I'm sure many users dont care what its implemented in :wink:


#4

NetBeans as base platform: I'm afraid of bloat. It locks the GUI harder to Swing, while I'd like to lock less to Swing. In the future, I'd like to add a way for objects to provision their custom GUI script code, for use-cases like a xy-pad control or more wild stuff. I don't think the NetBeans platform will help towards that direction.

Fileformat: @thetechnobear has done some work on linking from objects to C++ libraries, that allows using plain C++ code. That helps for advanced code, using your preferred editor, but does not create axoloti objects themselves, but those would be only a shim.
The problem with annotated C++ is that it is not strictly defined, parsing the annotations could become a struggle. Another approach is using a templating engine like moustache but that does not provide object metadata for the GUI.
I think the future for the object file format is to become sub-patches, where their c++ code is declared in more atomic objects that show up as a code-editor canvas in the GUI.
But that conflicts a bit with another desire for code-generated objects, it would be nice in some cases if the actual object could be generated by a script, say mixers with different channel counts depending on an object creation argument. That would nest c++-code-nested-in-XML, in, say, JavaScript... Not quite convenient for external editors, unless we do some sort of polyglot (but with divergent functionality as opposed to true polyglots).


#5

How about a 'bundle' file format for Axoloti files?

I don't know if it's possible under Windows/Unix, but in OS X, it's possible to have folders/directories that present as single files.

The bundle could contain a standardised file-structure, and perhaps also an XML file containing meta-information about the patch, but actual C/C++ code would be contained in individual files.

a|x


#6

GThe builtin patcher object editor could then display the meta info from the XML, and pull in any source files found in the bundle for editing, or allow for the creation of new source files for all the current XML sub-elements (K-Rate code, etc.).

The advantage of this approach is that it separates XML and C(++) code, but keeps separate source files conveniently bundled together.

It also provides a container for arbitrary other files (extra C++ libraries, license files, photos of the developers cat, etc.).

a|x


#7

Personally, Ive tended to keep the C++ code in the objects to a minimum, I keep the 'real code' in the C files, the new 'modules' will mean this is better managed, allow code reuse etc, and they are compilable units. so I dont really see a problem in having a bit of interfacing code in XML.

on the other hand, I also think annotating C++ code with comments (as its been done for years by IDEs and things like JUCE) is straightforward, you don't really need to use anything particularly complex, you just make it clear to programmers what is generated code vs user code. (and most programmers are familiar with this concept since its so common)

Id also highlight, like in C++/Java, its likely your going to want to use an IDE (aka object editor) in the future, because we will need to add things like 'UI layout' for objects, and this is tedious to do in 'text', which is why IDEs follow the generated code/comment approach.

as always theres countless options out there, they all have advantages/disadvantages, so we have to be clear what precisely we are trying to improve....

@toneburst, we are hoping/planning to introduce a package concept, this is a level higher than objects, which is probably more suitable for licenses/cat photos etc :wink:

EDIT:
p.s I dont actually have any issues editing the C code in the XML, the only thing I dont like, is that it could be format better
e.g.
it does stuff like (and tends to do the same to my code :wink:)

      <code.krate><![CDATA[   switch(inlet_s>0?inlet_s:0){
      case 0: outlet_o= inlet_i0;break;
      case 1: outlet_o= inlet_i1;break;
      case 2: outlet_o= inlet_i2;break;
      case 3: outlet_o= inlet_i3;break;
      case 4: outlet_o= inlet_i4;break;
      case 5: outlet_o= inlet_i5;break;
      case 6: outlet_o= inlet_i6;break;
      case 7: outlet_o= inlet_i7;break;
      default: outlet_o= inlet_i7;break;
}
]]></code.krate>

if it just added a new line it would be easier to read, and make no difference to the compiler :wink:

      <code.krate><![CDATA[
switch(inlet_s>0?inlet_s:0){
      case 0: outlet_o= inlet_i0;break;
      case 1: outlet_o= inlet_i1;break;
      case 2: outlet_o= inlet_i2;break;
      case 3: outlet_o= inlet_i3;break;
      case 4: outlet_o= inlet_i4;break;
      case 5: outlet_o= inlet_i5;break;
      case 6: outlet_o= inlet_i6;break;
      case 7: outlet_o= inlet_i7;break;
      default: outlet_o= inlet_i7;break;
}
]]></code.krate>

so just force a new line, so that the C code, was separate from the XML tags :slight_smile:


#8

Absolutely with you on the XML formatting.

Badly(auto)-formatted code is a bit of a bugbear of mine.

Also, code that seems to have been consciously written to use as few characters as possible.

And uncommented code.

This isn't the demo scene, guys; you don't get any kudos from me for minifying your code.

:wink:

a|x


#9

I say the above because I'm entirely self-taught, and from an arts background, so I really appreciate clear, nicely-formatted code, that's clearly explained.

a|x


#10

Hi Johannes,

About the Netbeans platform: You may be right about the bloat, I was just looking for at the advantages of the module distribution and ready-made editors etc. Anyway, it was just a thought.

I'm really interested in what you meant by locking less to Swing? Maybe it's an idea to implement something like OSC that lets you control parameters of nodes, based on their names (and whether you link it in the node)? Then you could even control the Axoloti gui over the network.


#11

I dont think we should be building any reliance on the GUI... the gui is just one way to control what is a hardware product.

its better to open up the parameter protocol, so that other GUI/devices can connect/control the axoloti hardware directly. then it will be 'trivial' to write an OSC bridge from the usb (parameter control) to OSC.


#12

That actually sounds like a better idea, would that be hard to do? I've glanced at this stuff in the past but couldn't really make out how to do that.


#13

Swing/Java2D is not very efficient. Marlin in OpenJDK9 may bring significant improvements (OpenJDK9 is scheduled for release 2017/07/27).

Swing essentially does not use the native OS toolkit to paint widgets, but paints lookalikes. Some frictions with Swing are that it is a pain to implement a native file-selector dialog on OSX, or add keyboard shortcuts to simple notification dialogs. Actually even Netbeans also does not use the OSX-style file selector on OSX...
Parameter introspection (that would allow retrieval of metadata over USB) is on my short-term development plan. Refactoring the GUI code so it can run GUI-less is also on my wish list, that 'd allow to create an OSC bridge server.


#14

Johannes, thanks a lot for all your answers, I'll be watching this, it's very interesting! By the way, if I read it correctly Marlin will be a renderer for Graphics2D, in other words it will put Swing on steroids. I'm not completely sure if this will help to have better native integration, but maybe it will be an improvement that way.