scaled ranges vs acuracy

[tester]

What is the maximum scaled range, to achieve 3-digit accuracy in FS 32-bit system?

I mean something like this. Let say you rescale (0-1) to (0-value), where the value is something like 22050. Between 0<x<1000 you get 3 digit accuracy, between 1000<x<10000 you get 1 digit less, but what interests me is the proper rounding (maximum value) in this scheme: (0-value) --> (0-1) --> (0-value). I noticed, that above certain thresholds, values like x.xxx end up with x.xxy, and I'd like to know that threshold.

[tester]

Hm... I just found a problem.

When value goes directly through the wires, then calculations is fine.
But when it's saved and reloaded as a preset, then the value is inaccurate.

So what is going on with this rounding?
Should not the preset pass-through show the real value?
How to evaluate these inaccuracy ranges?

[MyCo]

You introduce the rounding in the schematic by using format "%.3f", which rounds the float to 3 decimals.

But when saving to file, there might also be an issue. Looks like FS only save 6 decimals in the text file... this could cause some problems, when you deal with big value ranges

[tester]

Yes, rouding is placed to clean the mess after back conversion. But even if I remove this rounding thing, the problem is the same.

When I push to the input (at 0-16000 range) a value = 1, then output = 1.
But when save the preset and reload the preset, then the output = 1.008.

Sure, I tested the bigger ranges in order to notice, that some rounding is being done float2float (If I understood correctly - it is 6-digit total rounding, i.e. 999.999 / 9999.99 and so on), but that one goes on-the-fly. My concern is, that the preset primitive does not do the conversion on-the-fly too.

I'm guessing, that the proper max ranges should be something like x100, x1000, x10000 in order to get clear results (decimal multiplication/division). x10000 will have 0.01 accuracy, x1000 will have 0.001 accuracy. Can anyone confirm that or add some other hints on ranges? I'm not sure whether/how/if this interferes with 32bit representation.

Making dual-preset (less/greater division) is not in the play, preset program is right now made of c.a. 2000 entries and it loads c.a. 25 seconds on 2.2GHz CPU core.

Imagine that you work on some research project, that requires certain degree of accuracy instead of detuning. PITA The original project does not converts the values into (0-1) ranges, and everything is fine, except - it will not work with automation. But automation adds cool timeline on the other hand. As usual - work with SM/FS is a jar of honey with a spoon of tar ("a fly in the ointment").

[trogluddite]

There is a further complication...

You are talking of accuracy in terms of decimal places - but in 32bit float, they are binary places not decimal.
For example, try this very quick experiment...
- Place a float box, and type in '0.01'.
- Now go to menu "Options->Advanced", and select the box for "Show floats as 32bit binary representation."
- Take a look at your float box!

By default, FS shiws you a "decimalised" version of the number, in order to make the values easier to read. But as the float32 representation shows, there are many decimal numbers, even with only few decimal places, that do not have a precise binary equivalent.

When parsing a text representation of a decimal value, this is problematic. Each digit in the string must be multiplied by the corresponding power of ten and then summed - but as the negative powers of ten cannot be exactly represented in binary, it may result in a different output than passing a float as a literal number value.

Even if more decimal places are used in the text representation, it may not make the output any more accurate - the error in the larger decimal places could mask the tiny contribution of the smaller ones.
I am not certain whether using the "readable" decimal text is a choice of the FS dev's or whether it is part of the VST specification itself.

[tester]

I'm worried about 32bit interference into decimal usage of things, but if there is no any, then it's fine. Right now it looks, that I need just to use "normalized" rescaling decimal ranges, like 1000 or 10000 and not 720 or 16000 - and I will get less that I want to, but 0.01 accuracy on freqs maybe will do.

[matti]

You are trying to fight against the float format definition. Use integers.



If that doesn't make sense to you then try this; Take your 0-1 value and multiply with the range you need, convert to integer and voila you have a per decimal precision for that range. The small deviations in float values isn't going to affect the end result. Same works the other way; Take your ranged value and convert it to float, then divide by your needed range and you'r back to a 0-1 value.



Edit: Read the thread again.. bit slow in the mornings..

If the method i described somehow fails because of preset file saving/loading you could also try using VstStrings for your preset data.

[tester]

I'm nto sure what are you talking @matti but thanks for effort

The issue is, that for automation - values must be in range 0-1. The host does not accepts other ranges.

Neither preset strings will work here, because that primitive is not combined with automation protocols.

The only thing that could work - is to value into dual preset. But at 2000 parameters on oard right now - it does not makes sense (loading times).

But i rearranged in different way. Checked which priorities where are important - limits vs accuracy, so generally I'm fine right now. Not the high tech I wanted, but good enough to work with.

[matti]

I'm nto sure what are you talking @matti but thanks for effort

The issue is, that for automation - values must be in range 0-1. The host does not accepts other ranges.

Neither preset strings will work here, because that primitive is not combined with automation protocols.

The only thing that could work - is to value into dual preset. But at 2000 parameters on oard right now - it does not makes sense (loading times).

But i rearranged in different way. Checked which priorities where are important - limits vs accuracy, so generally I'm fine right now. Not the high tech I wanted, but good enough to work with.

Ok so you'd need perfect precision, which the current preset text file format can't give you, and automation in which you need to use the VstParameter primitives and values in 0-1 range. Then the most logical thing you need to do is implement your own preset file format. That's actually what I've done too, for different reasons though.

My implementation is to do with synthesizers and the midi format. Synths can communicate their internal patch data as System Exclusive, or Sysex, messages that can be sent over Midi cords. The values are usually 3 digit integers, though there's really nothing preventing more precision, and i've dealt with precision's up to 16384. What i do is I receive this Sysex data and decode it to my VstParameter values. I also need to send the data to a synth so i encode the data from the VstParameters back to Sysex. I also do my preset saving in this Sysex format, so basicly i'm doing exactly the thing you would wan't to do, saving decimal values of infinite accuracy to a text file. All you need to do is catch the values you need and save them to a file. If you really need it, it is possible.

[tester]

It's beyond my knowledge and skills. But if you already made it - you could share your preset system here, I think other people might be interested to use it too.