OpenSoundControl WIFI OSC through esp8266 (esp12-E) via serial PA3 [BETA]

Gael · 2018-12-26 02:29:03 UTC

---codes edited---
hello,
I played with esp8266 since one year, make some OSC controllers. Now i began to implement an axoloti object to com over serial with esp.
it's based on modified Paul's object "euxo/1-2/midi/button_pot". I used a WeMos clone as esp board.
Basically ESP12-E get float OSC messages and convert to 27bit (positive K-rate) incoded in 4x7bit bytes (midi data style),
ESP send serial.write (128+outlet number) "start byte" then send 4 data bytes (0 -127). So each OSC message received by ESP send a 5 bytes packet to Axoloti RX PA3. then OSC Axoloti object parse the packet and feed the outlets.
This offtenly work, but some time freeze aquire data (esp still receive osc). I need to improve this with your help.
"EDIT", now it's much more stable, I guess the other loaded objets made redondant serial declarations or something like that.
Wiring diagram :

I use a dedicate offline wifi router and I send osc messages with Lemur from ipad3 (broadcast Ip 192. 168. 0. 255) : /axo1, /axo2 ... /axo16
XML code of the object :

<axoObject id="test OSC sÃ©rial receive" uuid="d7ee3bc0-7466-4dbe-8769-bb5e15631a44">
   <sDescription>enable serial2( PA2/PA3 = SD2, baudrate: 115200) to communicate with esp-12 (esp8266) to recieve 16 float Osc messages : /axo1 ... /axo16</sDescription>
   <author>Gael</author>
   <license>BSD</license>
   <inlets/>
   <outlets>
      <frac32.positive name="axo1"/>
      <frac32.positive name="axo2"/>
      <frac32.positive name="axo3"/>
      <frac32.positive name="axo4"/>
      <frac32.positive name="axo5"/>
      <frac32.positive name="axo6"/>
      <frac32.positive name="axo7"/>
      <frac32.positive name="axo8"/>
      <frac32.positive name="axo9"/>
      <frac32.positive name="axo10"/>
      <frac32.positive name="axo11"/>
      <frac32.positive name="axo12"/>
      <frac32.positive name="axo13"/>
      <frac32.positive name="axo14"/>
      <frac32.positive name="axo15"/>
      <frac32.positive name="axo16"/>
   </outlets>
   <displays/>
   <params/>
   <attribs/>
   <depends>
      <depend>SD2</depend>
   </depends>
   <code.declaration><![CDATA[uint32_t axo1,axo2,axo3,axo4,axo5,axo6,axo7,axo8,axo9,axo10,axo11,axo12,axo13,axo14,axo15,axo16;
uint8_t ch; // channel code for outlet 128 + channel number
uint8_t inData[4] = { 0 }; // incomming datas for 27bit value
msg_t ThreadX2(){
#if CH_USE_REGISTRY
  chRegSetThreadName("euxo button pot"); // define thread name
#endif
//sdPut(&SD2,0xFF);
while(!chThdShouldTerminate()){
	while(!sdGetWouldBlock(&SD2) ){
		ch = sdGet(&SD2);
		switch((uint8_t) ch) { // Parsing start bytes
  		 case 129 : // axo1 start code : 128 + 1 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo1 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           case 130 : 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo2 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           case 131 : 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo3 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           case 132 : 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo4 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           case 133 : 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo5 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           case 134 : 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo6 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           case 135 : 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo7 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           case 136 : 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo8 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           case 137 : 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo9 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           case 138 : 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo10 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           case 139 : 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo11 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           case 140 : 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo12 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           case 141 : 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo13 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           case 142 : 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo14 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           case 143 : 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo15 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           case 144 : 
             inData[0] =  sdGet(&SD2); inData[1] =  sdGet(&SD2); inData[2] =  sdGet(&SD2); inData[3] =  sdGet(&SD2);
             axo16 = (inData[0] << 21) | (inData[1]  << 14) | (inData[2]  << 7) | inData[3] ; //sum at 1<<27
             break; 
           default : 
        	   break; //printf("error");        
   		 }
	}
     chThdSleepMilliseconds(1);
  }
  chThdExit((msg_t)0);
}
static msg_t EuxoButPot(void *arg) {
((attr_parent *)arg)->ThreadX2();
}
WORKING_AREA(waEuxoButPot, 256);
Thread *Thd;]]></code.declaration>
   <code.init><![CDATA[palSetPadMode(GPIOA, 3, PAL_MODE_ALTERNATE(7)|PAL_MODE_INPUT);// RX
palSetPadMode(GPIOA, 2, PAL_MODE_ALTERNATE(7));// TX
static const SerialConfig sd2Cfg = {115200, 0, 0, 0}; // set to midi baud rate but works also with higher baud rates.
sdStart(&SD2, &sd2Cfg);
Thd = chThdCreateStatic(waEuxoButPot, sizeof(waEuxoButPot),NORMALPRIO, EuxoButPot, (void *)this);]]></code.init>
   <code.dispose><![CDATA[chThdTerminate(Thd);
chThdWait(Thd);
sdStop(&SD2);]]></code.dispose>
   <code.krate><![CDATA[outlet_axo1 = this->axo1;
outlet_axo2 = this->axo2;
outlet_axo3 = this->axo3;
outlet_axo4 = this->axo4;
outlet_axo5 = this->axo5;
outlet_axo6 = this->axo6;
outlet_axo7 = this->axo7;
outlet_axo8 = this->axo8;
outlet_axo9 = this->axo9;
outlet_axo10 = this->axo10;
outlet_axo11 = this->axo11;
outlet_axo12 = this->axo12;
outlet_axo13 = this->axo13;
outlet_axo14 = this->axo14;
outlet_axo15 = this->axo15;
outlet_axo16 = this->axo16;]]></code.krate>
</axoObject>

Should I need to process some kind of checksum, use a more efficient parsing or implement a call/response protocol ?
and is there some "if (Serial.available() >9) ..." ? any other idea?
I'm more habit to Arduino IDE than C compiller, I red all topics about serial com (gpio PA2/PA3), but there is still a lot of shadows in my mind.
Thank you.

Here my work in progress arduino schetch, if you want to try this at home (now all 16 channels axo1, axo2 ... axo16 implement) :

/*---------------------------------------------------------------------------------------------
  Based on Open Sound Control (OSC) library for the ESP8266/ESP32
  https://github.com/CNMAT/OSC
  Receiving open sound control (OSC) bundles on the ESP8266/ESP32
  and serial com to Axoloti DSP
  Gael Jaton 2018
  code is in the public domain.
  --------------------------------------------------------------------------------------------- */
#ifdef ESP8266
#include <ESP8266WiFi.h>
#else
#include <WiFi.h>
#endif
#include <WiFiUdp.h>
#include <OSCMessage.h>
#include <OSCBundle.h>
#include <OSCData.h>
char ssid[] = "NUMERICABLE-A7A6";            // your network SSID (name)
char pass[] = "DD0258CE42F43E8AE7EACD4D4D";  // your network password
byte serialData[4] = { 0 };
//byte inbyte = 0;
unsigned long nextmillis = 50;
// A UDP instance to let us send and receive packets over UDP
WiFiUDP Udp;
//const IPAddress outIp(192, 168, 0, 255);     // remote IP of your computer
//const unsigned int outPort = 8888;         // remote port to receive OSC
const unsigned int localPort = 8000;        // local port to listen for OSC packets (actually not used for sending)
OSCErrorCode error;
int ledState;              // LOW means led is *on*
#define BUILTIN_LED 2
void setup() {
  pinMode(BUILTIN_LED, OUTPUT);
  digitalWrite(BUILTIN_LED, ledState);    // turn *on* led
  Serial.begin(115200);
  // Connect to WiFi network
  Serial.println();
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.begin(ssid, pass);
  while (WiFi.status() != WL_CONNECTED) {
    digitalWrite(BUILTIN_LED, 1);
    delay(100);
    digitalWrite(BUILTIN_LED, 0);
    delay(100);
    Serial.print(".");
  }
  //    IPAddress ip(192, 168, 0, 11); ////////////////////////////////////////////
  //    IPAddress routeur(192, 168, 0, 1);
  //    IPAddress subnet(255, 255, 255, 0);
  //    WiFi.config(ip, routeur, subnet);
  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
  Serial.println("Starting UDP");
  Udp.begin(localPort);
  Serial.print("Local port: ");
#ifdef ESP32
  Serial.println(localPort);
#else
  Serial.println(Udp.localPort());
#endif
}
void serialSendToAxo(byte startByte, uint32_t msg32) { // send to axoloti start byte include message number, then send 4 messages of 7bit data
  serialData[0] = msg32 >> 26 & B01111111; // pack into buf string as 4 x 7bit bytes
  serialData[1] = msg32 >> 19 & B01111111;
  serialData[2] = msg32 >> 12 & B01111111;
  serialData[3] = msg32 >> 5 & B01111111;
  Serial.write((byte)startByte); Serial.write((byte)serialData[0]); Serial.write((byte)serialData[1]); Serial.write((byte)serialData[2]); Serial.write((byte)serialData[3]);
}
void axo1(OSCMessage &msg) {
  uint32_t msg32 = msg.getFloat(0) * 0xffffffff;
  int ledState = 1023 - 1023 * msg.getFloat(0);
  analogWrite(2, ledState);
  serialSendToAxo(129, msg32);
}
void axo2(OSCMessage &msg) {  uint32_t msg32 = msg.getFloat(0) * 0xffffffff; serialSendToAxo(130, msg32); }
void axo3(OSCMessage &msg) {  uint32_t msg32 = msg.getFloat(0) * 0xffffffff; serialSendToAxo(131, msg32); }
void axo4(OSCMessage &msg) {  uint32_t msg32 = msg.getFloat(0) * 0xffffffff; serialSendToAxo(132, msg32); }
void axo5(OSCMessage &msg) {  uint32_t msg32 = msg.getFloat(0) * 0xffffffff; serialSendToAxo(133, msg32); }
void axo6(OSCMessage &msg) {  uint32_t msg32 = msg.getFloat(0) * 0xffffffff; serialSendToAxo(134, msg32); }
void axo7(OSCMessage &msg) {  uint32_t msg32 = msg.getFloat(0) * 0xffffffff; serialSendToAxo(135, msg32); }
void axo8(OSCMessage &msg) {  uint32_t msg32 = msg.getFloat(0) * 0xffffffff; serialSendToAxo(136, msg32); }
void axo9(OSCMessage &msg) {  uint32_t msg32 = msg.getFloat(0) * 0xffffffff; serialSendToAxo(137, msg32); }
void axo10(OSCMessage &msg) { uint32_t msg32 = msg.getFloat(0) * 0xffffffff; serialSendToAxo(138, msg32); }
void axo11(OSCMessage &msg) { uint32_t msg32 = msg.getFloat(0) * 0xffffffff; serialSendToAxo(139, msg32); }
void axo12(OSCMessage &msg) { uint32_t msg32 = msg.getFloat(0) * 0xffffffff; serialSendToAxo(140, msg32); }
void axo13(OSCMessage &msg) { uint32_t msg32 = msg.getFloat(0) * 0xffffffff; serialSendToAxo(141, msg32); }
void axo14(OSCMessage &msg) { uint32_t msg32 = msg.getFloat(0) * 0xffffffff; serialSendToAxo(142, msg32); }
void axo15(OSCMessage &msg) { uint32_t msg32 = msg.getFloat(0) * 0xffffffff; serialSendToAxo(143, msg32); }
void axo16(OSCMessage &msg) { uint32_t msg32 = msg.getFloat(0) * 0xffffffff; serialSendToAxo(144, msg32); }
void loop() {
  OSCBundle bundle;
  int size = Udp.parsePacket();
  if (size > 0) {
    while (size--) {
      bundle.fill(Udp.read());
    }
    if (!bundle.hasError()) {
      bundle.dispatch("/axo1", axo1);
      bundle.dispatch("/axo2", axo2);
      bundle.dispatch("/axo3", axo3);
      bundle.dispatch("/axo4", axo4);
      bundle.dispatch("/axo5", axo5);
      bundle.dispatch("/axo6", axo6);
      bundle.dispatch("/axo7", axo7);
      bundle.dispatch("/axo8", axo8);
      bundle.dispatch("/axo9", axo9);
      bundle.dispatch("/axo10", axo10);
      bundle.dispatch("/axo11", axo11);
      bundle.dispatch("/axo12", axo12);
      bundle.dispatch("/axo13", axo13);
      bundle.dispatch("/axo14", axo14);
      bundle.dispatch("/axo15", axo15);
      bundle.dispatch("/axo16", axo16);
    }
    else {
      error = bundle.getError();
      //Serial.print("error: ");
      //Serial.println(error);
    }
  }
  if (WiFi.status() != 3 ) {
    analogWrite(2, 1023);
    delay(100);
    analogWrite(2, 0);
    delay(100);
  }
}

TODO :
*more tests with different controllers
*try access point "WiFi.softAPConfig(local_IP, gateway, subnet)" provide by ESP for more mobility...

natcl · 2018-12-26 16:21:44 UTC

OSC over serial usually use SLIP to encode/decode the serial stream, is that what you are using ?

Gael · 2018-12-26 16:44:45 UTC

Nope, it's a custom protocol, looks like midi CC message but data is on 4x7bit bytes.
It's too much work and knowledge for me to implement a full Slip protocol. But I will look at this option later.

natcl · 2018-12-26 16:58:05 UTC

Ah got it, the esp does the osc to midi conversion.
I might have a look at it at some point.
A good starting point is the official Arduino OSC library:

Gael · 2018-12-26 17:26:17 UTC

Yes, I use this library, very stable and no surprise.
Custom protocol have much more resolution than midi : OSC 32bit float => 27bit k-rate positive outlet (whereas midi CC provide only 7bit res and 14bit for HiRes CC). So I can use most of the precision of OSC messages for smooth controls.

Suburb_Animal · 2018-12-31 19:00:54 UTC

Great work Gael!
I'll give it a try once I'll have my nodeMCU up and running.
I'm thinking about making a kind of OSCQuery through Serial.
OSCQuery is a OSC message that ask the recipient to send a list of prefixes.
I want to make an object that generate those address for each available parameters and pass it to the arduino.
The list would be sent on request or when a new patch is loaded

The NodeMCU will make a table of those prefixes with their serial equivalent that would be an index instead of a prefix.
(in order to shorten the messages in serial)

The NodeMCU would receive OSC messages and would parse it to the right index corresponding to the prefix.
For example:
serial index 1 would be /osc1/pitch
Lemur would send an /osc1/pitch value (in float)
NodeMCU would send 1 value (in 27bits)

Since Lemur can also receive a whole lemur patch through OSC i'm also thinking about storing them in the NodeMCU and send the corresponding patch:

But that's another story

Gael · 2018-12-31 20:03:36 UTC

oh great! I also begin a Github folder for this project : https://github.com/gaeljaton/OSC-Esp12-Axoloti
I don't have the lemur editor , nude version was installed on my ipad for a theatre project...
I'll have a look on your git!

Suburb_Animal · 2019-01-02 09:04:00 UTC

@Gael
The lemur editor is a free application, you can download it here:
https://liine.net/en/downloads/lemur
That github above is not mine it's an example of a lemur loader that send lemur patch through OSC

Gael · 2019-01-03 12:14:32 UTC

'Update'
I tried various accessPoint settings for ESP receiver, I got much more lost UDP packets than using a router. So I 'll developp code as station ESP only. If you want to try this, grab a dedicate offline router, even without password (security is not critical for this application). Some of old internet boxes can be setup as simple standalone wifi router, recycling is life.

mantaraffu · 2020-01-07 11:27:39 UTC

maybe I'm missing something. I've just put the osc object in a simple patch, but it doesn't work any suggestion?test_osc.axp (3.9 KB)

Gael · 2020-02-01 14:43:56 UTC

yeah it's a bit more complicated. you have to set SSID and password of your wifi network (wifi router or wifi switch) in the code of the esp8266 (in arduino ide)... it require some skill in arduino programming, and osc protocol knowlegde.
have you follow the project page tips : https://github.com/gaeljaton/OSC-Esp12-Axoloti ?

weasel79 · 2020-02-01 21:38:02 UTC

hah i don't know how i missed this before. awesome work @Gael hoping to try this out one day..

mantaraffu · 2020-04-19 13:03:49 UTC

yes, of course I've change all the parameter on the arduino code. the esp is connected and I've tried to send data from max and from touchdesigner without success. this, 3 months ago. I'll try today and I'll let you know. thanks!

orginita · 2021-01-31 15:30:54 UTC

instead of letting a router communicate, can it be communicated to another wemos d1 mini so as to be able to use midi communication even through midi devices without a pc such as a keyboard to a sound module?