Load floats into table(SDRAM) from file with constant multiplier


#1

Hi, My first post here..
I am trying to modify a table load object to multiply floats by (1<<21) as they are copied into the SDRAM table. Looking at the factory table load object I can't figure out how to access the data as it seems to be dealing with pointers and memcpy(). I'm not a programmer.

I have looked for a way to do this with objects and haven't come across anything.
alternately I could preprocess all of my data but I am lazy.

What can I do?
I'm reasonably certain that there is an easy way to do this.


Frequency Table Read for Alternative Tuning
#2

Just a thougth:

Do you have to do the scaling when it goes in to the table?

Is it not enough to rescale the output of the table instead, so when you play it back, it converts the number to what you want on the fly. I think thats probably alot easier to do, than to recale wha you put into the table. but that also depends how you plan to get the numbers into the table, record them, write them in code...

Was just yesterday making an object where I wanted to use floats from an array. The array I wanted to work with was also floats, so i have to convert them. But the problem with these numbers was that they were decimal numbers and Axoloti could not process them in time and patch would not go live at all. It seemed like that floats with too many decimals, made Axoloti hang, cause it takes too long time to convert them to something meaningful.

So what I did was rescale the floats into integers. So the floats that looked like this:
1,567459
was converted into this:
1567459

I multiplied with a million so the number now looked like this, now it is represented as integers instead, which Axoloti likes better than floats.
These numbers I can work with and now the patch would load, but I stopped here last night, I need to figure out a few more things before I can make it work. Interpolation etc.

There are other ways to convert from float to int32_t. Look here:

Anyway, sorry for the non-conclusive post. Hope you can make something of it, even though it is not a straight answer. :slight_smile:


#3

@jaffasplaffa Thanks for getting back to me.. You're right I should be using integers, Seems I might be best to preprocess these tables. I need to spend more time coming to terms with fixed point stuff.


#4

what you need to do is copy the table/load object, and then edit it.

the line(s) you basically need to change is:

        memcpy((char *)(&attr_table.array[0]) + offset,(char *)fbuff,bytes_read);

so what this does is copy the character array (fbuff) of size bytes_read into the attr_table.array.
its done using memcpy, as this is the quickest way to do it,

this might make more sense to you if i wrote it like this (but would be much less efficient :wink: )
(untested/compiled, so indicative only :wink: )

for(int i = 0;i<bytes_read;i++) {
     attr_table.array[offset + i] = fbuff[i]
}

if your going to alter the data as it comes in , then you have to use this form.
you can see in the above, you could now easily alter the data when doing the assignment,
however, be careful as this is just copying across as 'binary' data, so depending on the format of your file, it may have to be done differently. (particular if the data is multi byte)

another alternative (perhaps simpler) would be to post process the data.

so after the lines

    err = f_close(&FileObject);
    if (err != FR_OK) { report_fatfs_error(err,inlet_filename); return;};

you know the file has been loaded into the array, and so you can now process
e.g.

for(int i =0 ; i<attr_table.LENGTH;i++) {
    attr_table.array[i] = attr_table.array[i] << 2; // or whatever it is you want to do !
}

(hmm, id need to check attr_table.LENGTH , but dont have time :wink: )

the disadvantage of this approach, is that you have to work within the constraints for the array ... (e.g. you cannot upsize from a int to float... well you could but its a bit more processing :wink: )

not quite sure what to make of this... having too many decimals will not make axoloti hang/crash.
if its done during startup, or exceeds the watchdog timeout - then it will disconnect the editor, which is likely what has happened.

I personally see this as a good warning... with dsp programming, you should try avoiding doing large chunks of work in one go, that could disturb audio processing. there are various programming techniques you can use to distribute this work over time to avoid this (including using background threads)
(it may seem more complicated initially, but once you've done it a few times, it becomes easy to do... as much as anything its a way of 'thinking about dsp programming' , a style if you wish)

generally, upfront processing is to be preferred in dsp programming.
as @bds implied, its actually best to do this even before the file is put on the axoloti board,
but failing that, then a longer 'startup' phase is preferable, to a continuous load on cpu for every time you 'read' the data.... esp. considering the limited resources of the axoloti


#5

Awesome thanks @thetechnobear I'll give that a shot.


#6

Yeah, now I am a bit stumbled here. I did some testing last night and I kept getting the same error from the log, saying somrthing like "taking to long time to convert from floats to int32_t, disconnecting.

And right now I tried to do the same thing as I did last night, to replicate the error.... But this time it worked.... So I might have done something wrong last night. But I did get the above error about "conversion taking too long" and disconnect, so I rescaled it to integers(I looked in TSG diode sat object, he also uses integers for look up table), so I just went for that and that simply just worked.

Ill try a couple of times more to see if I can replicate. Stupidly I did not save the patch which gave the error.


#7

Yeah I actually also did that, I created the array in Pure data, created a list of floats, that I just multiplied with one million, to get integers and then pasted those into Axoloti. That worked and the conversion was done before putting into Axoloti array. But still, i needed to do a bit of conversion from int to int32.


#8

@jaffasplaffa how do you load your list of floats from the SD card?

I am trying to get my data into the axo from files on the SD... I have been battling with this for a few days now.:confused: The problem I suspect is how I am formatting the files, there must be a right way?

I tested with data of increasing in numerical value (a monotonic function), and I'm not getting the monotonic function back out.

I tried formatting the data in Max with different scaling factors. That doesn't seem to help.

I have even tried formatting the SD card between different data sets.
Heres a the patch I am trying to use.


#9

@bds I changed it to use integers instead.

I used Pure data to make a list out of the content of a table. When I had the list, I copied it into the object, so I didnt actually play it form Sd-card. I just played it from the object itself.

I am not sure how to read floats form sd-card, I haven't tried yet.

But it should be possible. For example look in toneburst interpolation objects. In those objects, you will find some float > int conversion and int > float. maybe there is something you can use in there?


#10

if you are using table/load then it reads the file as binary data, where each float is 32 bits (so 4 chars)
so the float has to be in the same format as floats are in axoloti (i.e. fixed point )
see https://en.wikipedia.org/wiki/Q_(number_format)

( I think theres some post about it here on the forum)

the easiest way to create a file like this, would be with axoloti itself, and using the table/save object.

not sure, what else supports this (its not uncommon, but not often exposed as a 'user format')

the alternative is as i stated early to do the parsing yourself, then convert it into the require format for axoloti.


#11

I'm looking at SirSickSik's primecreate to get a few ideas, particularly the way the data is stored on the card. Very nice in Hex. So I can see what I need to achieve.
Maybe I am biting off too much..though I think it would be good to be able to parse 32uint data (binary) from software and hardware outside axoloti in a way that the axoloti and table/load expects.

I'll hunt around the forum some more..
I am thinking maybe a command line script could be the ticket.