Weird float problem

[User]

I'm trying to reduce input noise by using an inverted signal with a value of 0.99999994. Problem is that the float primitive is rounding it to 1. Any ideas how to solve it ? Or why is it doing that rounding ?

[juha_tp]

I'm trying to reduce input noise by using an inverted signal with a value of 0.99999994. Problem is that the float primitive is rounding it to 1. Any ideas how to solve it ? Or why is it doing that rounding ?

Hmm... where do you put that value and where do you see it rounded?

[HughBanton]

In Options/Advanced there's a tick-box - 'Display floats as 32-bit binary representations'.

If ticked you should see 0.9999994 in a float box; unticked it shows 1. Could this be your issue?

H

[User]

Probably that's the issue. I will have to try that. It looks like on a knob it works but it's hard to control it if you want to increase/decrease the value by one unit. I tried to increase the SHIFT precision but it still jumps by 6 units instead of 1. After scrolling for miles. How do you make the knob work in 0.00000001 increments ? There must be some trick. Maybe using one knob for making big adjustments and another one for fine adjustments ?

[User]

It was doing that in the Float component. It's not doing the rounding anymore after checking that box in the Options/Advanced. Thank you for your help.

[User]

It looks like it's not allowing anything higher than 0.99999994. It gets rounded either to 1 or back down to 0.99999994. Even after checking the "Display floats as 32bit" box in the Options/Advanced. I guess the problem is still there.

[User]

Either my Windows 10 is screwy or maybe it has some bugs. I don't know. Just take a float component from the toolbox and type in 0.99999994. Then try to change it into 0.99999996 or 0.99999998. Mine jumps back to 0.99999994 or it gets rounded to 1.

[HughBanton]

Afaik this is just a display thing, and internally Flowstone calculates the same either way.

Here's what happens when you pass floats through Ruby; the 'Watch' doesn't change whether the 32-bit box is ticked or not. Try ticking & un-ticking.

float_test.fsm

(315 Bytes) Downloaded 752 times

H

[User]

That's interesting but it's still not able to increment that number by 0.00000001. It still jumps in increments of 0.00000006. And if you change 0.99999994 into something like 0.99999990 it gets rounded to 1. The problem is still there even if the 32bit float view is checked.

[deraudrl]

That's interesting but it's still not able to increment that number by 0.00000001. It still jumps in increments of 0.00000006. And if you change 0.99999994 into something like 0.99999990 it gets rounded to 1. The problem is still there even if the 32bit float view is checked.

Even if display rounding in the text conversion is ignored, the real issue is that the LSB of the mantissa of anything between -0.5 and 0.5 is 2^-24: that's all the resolution you have.

And as it turns out, 2^-24 is roughly 0.00000006. You can represent 0.00000001 very accurately in 32-bit floating point; what you can't do is represent 1.0 minus 0.00000001 accurately: there simply aren't enough bits in the mantissa. This is about the time where Martin or tulamide jumps in to tell me I'm off by a factor of 2 somewhere, but you get my point. The thing is, you can multiply/divide arbitrary FP numbers with great accuracy. Adding them? Not so much, especially when their magnitudes are that far apart.

(I don't recall offhand what FS provides in the way of double-precision primitives...you might give that a shot. But don't be surprised if the displayed values behave oddly if there aren't enough fractional digits displayed.)