User Library: A Proposal

#1

Ever since the release of axoloti, its been fantastic the number of contributions we have had.
But this success (and the popularity of the forum) has meant its getting difficult to find objects and sub patches that have been contributed, and i think we need to organise things a little better.

the proposal below is not the ultimate goal, but something that I think we can do reasonably easily, and Ive time to implement in the client etc... and that later we could migrate to a different solution should we wish to.

Forum: User Library Category

A user will be allowed ONE topic in this category, on the first post, they can list all their contributions. split as :
objects - axs, axo (and associated axh help files)
demos - complete patches
firmware - firmware extensions
these files can be initially attached to this first post, and should include a version number and short description. following post in the topic can be used for further description, examples and also for others to discuss.
(also ensure you put your name/license/description in patch settings!)
we will encourage all users, to like, give feedback to authors
once you contribute, you will be 'granted contributor status' and will be PM'd to ask if you wish to become a collaborator . (see below :smile: )

Note: if we seen more than one topic per user, we will merge topics, we want to keep the number small.

Repository : user library repository (contrib.)

a. we will create a new repo (github.com/axoloti/contrib ?) that will contain user contributions, which collaborators will be able to contribute to.
(collaborators is the term used by github, so hence we need to re-use this)
the license agreement on this repo will be 'free for use', but the contributor retains ownership
collaborators will need to be registered with github.

b. I will extend the Axolotl client (using JGit) to do 2 things:

i. have a sync function which will allow ALL users to download and synchronise the contrib objects, this will sync into the current axoloti document folder as a contrib folder.
ii. allow collaborators to contribute new objects/demos/firmware.
(the hope is to 'hide' git away as much as possible for day to day use, but more advance options will have to be done using conventional git tools)
iii. the user library will appear in the object search windows as 'contrib' and then hierarchy

c. when a user is a collaborator, they should still post on the Forum : User Library, but instead of attaching the contribution directly, they can simply say its in the user library, and name it
e.g. midi/ctrl/launchpad.axo (in user library) . this will of course allow also for discussion for your users.
again, check you patch settings are filled with relevant info

some other note:

  1. We will also need that some collaborators help keep the 'user library repo' tidy, and perhaps 'police' certain aspects e.g. ensure copyrighted material is not committed! (lets call them 'maintainers')
  2. The contributor is responsible for maintenance and ensuring it works on future versions etc.
  3. If you would like to volunteer as maintainer... but are not a contributor, that is cool too.(collaborators will already have permissions to do this anyway)
  4. all collaborators will be able edit/delete other contributions... thats just the way github works. (but of course changes can be reverted as its version control)

some complications... which I think we will just have to 'work through'

  1. patches/objects not working.
    theres always a chance that certain objects don't work, or stop working after an update, and the contributor does not maintain. this ends up being 'dead objects' which is often frustrating for users.
    if it persists not working/complaints etc, then I think the maintainers will just remove it.
    (we cannot start passing the maintenance obligation on... with 100's of objects this will just be impossible)

  2. samples (and other 'media')
    we cannot have large samples included (size limit?) , it will just take too long for an end user to synchronise, and we have to remember they may be completely uninterested in your samples/demo!
    no copyrighted material can be uploaded ... including anything from commercial sound libraries.
    (fortunately, uploads contain username uploading... so enforcement will come after you :smile: )
    I was thinking we would 'outlaw' samples, and only allow axo/axp/axh/axs BUT this would mean many demos just would not work/be useful
    thoughts?

  3. demo uploads (axp)
    it would be contributors responsibility to ensure all dependent objects/samples are uploaded correctly.

  4. organisation.
    ok, hardest topic for last :smile:
    there are two possibilities i see:
    contrib/midi/ctrl/launchpad.axo
    contrib/thetechnobear/midi/ctrl/lauchpad.axo
    i.e. every contributors objects get put together, perhaps most logical for users, or we split by contributor.
    I think I favour the second, as it kind of makes the contributors recognised, and also if there are issues, end-users will be know who is looking after it....it also means if two contributors could use the same name without issue e.g. drumseq
    (remember: the object search window does matching, so in both cases if you type launchpad , matches will be found regardless of hierarchy and also duplicates will also be listed)

some of this has been raised in a thread here, but I wanted to bring together a 'full proposal' and thoughts in one place.

I'll repeat, I think a database/web front end would be better BUT I think (now!) waiting for that to be developed might be a mistake, as we all have lots to get on with, and perhaps this is not a big priority BUT at the same time, I think it would be great for users to have easy access to user contributions.

so.... what are your thoughts? a good first step? have i missed something?

one parting thought, I know there are better ways to do some of the above (e.g. packaging demos)
BUT can I please ask
- we keep 'expectations' in check. i.e. lets not have suggestions where its expected someone else has to develop it. this is aimed as a starting point, with relatively little dev effort required not necessarily the end game :smile:
- bare in mind, whilst you can ask people to do x,y,z e.g. tagging, if there are too many 'rules', they will either i) get it wrong ii) not do it iii) decide its too much effort to contribute. ... so we need to keep it easy/simple

#2

I've been using Node-Red a lot lately and they have a nice web frontend that gathers all user content from github/npm automatically:
http://flows.nodered.org
Might be good for inspiration !

#3

way too complicated, imho.

I think many people don't know how to use github or even what it is, since many of us are musicians and not programmers

There is a lot to learn from the Reaktor userlibrary and how it evolved over the years. What I noticed is that the most important is:
- search functionality. not just search by category or date, but also votes (as a indication of quality) and user.
- images. A picture tells a thousand words.
- usability. It must be easy to add stuff. having to read a manual to contribute means that almost nobody will contribute. Of course, a nice interface helps.

#4

I think you have misunderstood...

users will NOT see github, the Axoloti GUI will automatically be syncing and submitting things, most users will just continue to use the axoloti UI as they do today, with a couple more options:
as far as the user sees it, they will just say:
- share this new patch/axo etc
- upload changes patches/axo to library
- sync contribution library , at which point they will see new objects in the object search window.

no mention of git etc...
(only thing collaborators will have to do is signup to git, non-collaborators/normal users wont need to do this)

my point above about advanced operations, is I'm not going to start implementing things like delete, rename, seeing version histories etc. as most users don't need... so therefore 'for admin' purposes we can merely use github directly.
(but these operations are reasonably rare, and could be done my maintainers who know how to use git)

I thought I was pretty clear in my closing paragraph,
I don't think this is a perfect solution, more a practical one which I probably have enough time to implement, and perhaps a starting point.
Of course, Im sure we would all like a pretty web interface, with integration into the UI, searching, tagging.... and if you have the time/skills to implement it thats fantastic news for us all.

anyway, if its not what people want thats fine by me too... hence why I proposed it, rather than started implementing it.

#5

@thetechnobear
ah ok, I'll read it carefully again tomorrow :wink:

#6

This clearly overlaps with the 'thing' we where discussing on the other topics. I have to read evreything slowly again and think how can I contribute to this :grin: :stuck_out_tongue:

#7

I have read everything (specially)

and to my understanding its a good way to implement this "something" that must be daunting to organize and get right with low resources (time).

I have signup and am around Git for some time now, nothing special about.
@alex evolution is just that, like Reaktor user lib. I'm sure most musicians have many more skills to their day by day "trade" nowadays :wink:.

#8

@thetechnobear
I think it's a great idea to start off with.
Having everything accessible directly from the client app seems an easy way for us non-techs to get new access to new modules and examples without any hassle.
I'm sure in the future, having keywords or categories for the patches (or subpatches) would be useful, especially if we get flooded with new material, as well as a "What's new" page, with all the latest additions. But using the forum for that seems fine at the moment

#9

One thought:

Why make it so complicated? It is a file archieve system, which you can find 1000s and 1000s of on the internet all ready... It is not something that really need to be reinvented. Isnt it like reinventing the wheel?

There are as mentioned above loads of great patch archive system all ready, like Reaktor library, Electro-music.com patch library. Why ot just take inspiration from them? :smile:

#10

Git is a file archive, that is already being used by everyone here. I actually like Technobears approach quite a bit. (and Im glad you necroed the thread). Having it be part of the default patcher UI is pretty brilliant in that for most folks its just another menu called "library" that then reads from GIT. The Reaktor library is pretty slick (user voting, , but it would require pretty constant maintenance and a fair amount of front end web work, the elctro-music nord patch library is a bit of a mess though and I wouldn't use that as a good example.

For organization I would 100% go the tagging route (which GIT supports). The directory structure on the back end doesn't matter as much as long as the front end UI uses the Tags to organize stuff. You can have a preset set of tags (these are stolen straight from the Reaktor library) The bold ones are their categories the rest are tags.

Blocks
Patches
User Blocks
Building Templates

Instrument
Synthesizer
Sampler
Sequencer
Other

Effect
Multi-Effect
Distortion
Dynamics
Filter
Delay based
Other

Other
Macros and Tools
Sounds
Test Area
Documentation

I am not tied to GIT but it has a lot of the usable functionality for something like this built in. @thetechnobear have you done anymore work or thinking about this in the last few months?

#11

I am also not against it. I am only little bit familiar with Git(what I learned from using Axoloti), but not totally compatible yet :smile: Of course if it is what most people use anyway and are familar with, it makes sense to use Git :smile:

#12

After reading it carefully again, I can see why Git is a good choice.

But I am not sure I get this part of it:

Does that mean you have to download all content of Git Axoloti library or you only download the list of objects and the you can select which patches you want to download from within Axoloti editor?

This is an issue we cannot go around. Since many people use custom objects in there patches, this will always be an issue, especially when working with open source programs like Axoloti. Everyone is making their own versions of the objects. There are all ready loads of them in the community. It must be the uploader task to make sure that the patches uploaded also includes custom objects for that patch. We just need to have faith in each other :smile: If patches are not updated, after a friendly reminder or two, they should be deleted.

I think outlawing samples is not so good idea. But I also understand why it is a good idea. It would take up a lot of Github space. But I think the idea of a patch library kind of looses its weight, if you cannot upload everything like samples, custom objects along with the patches. If you have to link to another place, why not just put the patch there too? It is going to be too confusing shifting between pages to get samples/patches,/etc. Better to have all collected in one place.

I have many patches that are "specialized" for one specific sample. And I think most patches using samples are specialized cause of how the tables work. You cannot make patches that work well with all samples, a patch is usually dedicated to one specific sample to do one specific thing(I know you can make patches that work with hall samples, but take in mind table/alloc table size and other variables in patches). Also if you build a patch around one sample it would be weird not sharing the sample along with the patch. That way people might not understand what the patch is actually about.

But I think a lot of your proposals are good and some of the questions you have raised are very valid. And thanks for taking the time to do it :wink: I have no programming skills, so cannot contribute in development, but I will be happy to contribute to the patch library when it is "released".

I am thinking cant the "open URL" feature of Axoloti be used in this context? If you are browsing Git and find the patch you like, you copy the URL to Axoloti and paste it and open the patch like that? That is pretty simple. But yeah that might cause problems if custom objects and samples are included in the patches.

#13

No, Ive not done anything more... since there was no consensus, which meant I couldn't just start adding code into the main axoloti code base....

its a shame, because as i clearly stated, my belief is the existence of a shared library is much more important to Axoloti, than if its a good implementation or not. (let alone a perfect one).
but the discussions here and elsewhere, did not seem to agree with this... so here we are nearly 3 months later with no progress.

Git was only one solution on how to do it quickly, I thought I made that really clear in the opening paragraph of the post.

similarly in the closing paragraph i stated:

frankly if we want to do this really well, its many man weeks of development.... and Ive not seen anyone willing to take this on as yet.

so i ended up feeling, perhaps I was the only one that felt this, and was perhaps just being too eager to get to the next stage, a place where 'newbies' could pick up axoloti and 'just use' - and thats cool by me, it was only a proposal

#14

And that is also why I change my mind about it. Since I cant contribute with coding, I will support your idea, cause it seems like you put a lot of thought into it :wink: Just had to read it a few times.

Anyway, yeah, I also think it is a shame, cause it would make Axoloti more easy to approach for modular "beginners" :smile:

#15

It was a very good proposal, and your goals are very strong. Once I have gotten more comfortable with the whole environment I would certainly be interested in looking at this.

#16

Yes. A shared library is the best feature we can have to continue growing our Axoloti community.
But if everybody have an opinion we will never end this discussion. Let's do it!!

#17

I apologize for leaving this tread open for too long. Many things required my full attention during the past months that appeared more urgent to me, with the idea that a user object/patch library needs deep and careful planning.

OK, let's give it a go! I created https://github.com/axoloti/axoloti-contrib
PM me (with your github username) if you want to be added as a collaborator.

I have one question: will users be able to push commits to the repo from within the GUI, and how is login handled? But even without, a GUI like github's own application makes it fairly accessible.

#18

It seems like a good possible solution to this problem would be to provide a fully fledged package (patch?) management utility within the axoloti patcher, one which can point to one or many repositories such as the proposed axoloti-contrib repo.

This approach absolves several of the current concerns regarding file-size restrictions for patches and patches containing samples (to prevent user from downloading a huge mass of data from the repo unnecessarily) because this approach would allow for the end user to browse for packages (whatever you want to call it) and install in an ad-hoc fashion, or even en-mass.

A formalized package manager would also allow ample room for seemingly transparent repo-integration. One such way would be to always download a manifest of all contrib packages upon patcher startup (think apt-cache, if you're familiar with debian), and this way a user could see contrib patches in the patch-insertion menu (when you press space in the patcher), but the ones that need to be downloaded are grayed out or have some visual indication that they need to be downloaded/updated upon insertion. Also, there could be dependency resolution and version history (for patches which use an older external package version). A dependency/version resolution also solves another concern you had in regards to broken patches. if all patches were being maintained, it would make keeping patches operable a fucking nightmare because everything would be breaking all the time...

However, to implement this kind of thing, we may need to seriously reconsider the file representation of an axoloti patch. Now, I am speaking without having really opened up a patch, but I believe axoloti patches/subpatches themselves should be something like an archive, with a manifest file (which has references to external contrib patches with versions) and then patch-specific files, sub-patches, documentation, etc. stored within it.

This may not be a feasible approach, but I'm sure there are package management libraries we could leverage from the open-source community to aid in this effort.

#19

Johannes has been doing some work in packaging area, and I in the user library for a future release.
it might not be as feature rich as you outline (I'm familiar with package management system), but I hope will act as a 'stepping stone'.
BUT more importantly it will provide users with a functioning system, which will make sharing much easier and organised than it is today. I really hope this will encourage the creation of even more content for sharing in the axoloti community.

#20

Great

I was over on Github a few times looking for patches. but gave up.. and also gave up uploading anyting to it because it is too complicated. Since Johanne made the repository not one patch have been uploaded. Must people still do it here.