Inconsistent String Behavior

[tulamide]

I was trying to help out billv here.
During testing I found out that Flowstone's green strings simply delete any non-printable-characters, although they are valid string chars. Have a look at the schematic.

On the module, click "save states". Inside the module have a look at the output of the string length prim. Now in the ruby edit instance named "Managing the states" look for the event block. Uncomment the line "#raise ArgumentError, s.size.to_s". Head back to the front of the module and click "save states" again. The module will show candy on purpose. Go back inside the module and look at the info pane of the ruby edit instance named "Managing the states". The number you see there is the size of the string right before sending it out to green.

Any thoughts, work-arounds, tips?

[KG_is_back]

You can convert the bytes of the string to array of bytes using s=s.each_byte.to_a
and then convert this array into string of ones and zeros by s=s*""
This can easily be send via green strings. It should also be easy to convert back.

[tulamide]

That's exactly what I did. It can't. And the reason are the non-printable chars. For example &b00000010 just gets deleted. It only works for printable chars. For example &b01000001, which would be intepreted as "A".

EDIT: Maybe I should explain it a little more. The bits are set in an array of integers, which on save are packed to a string via "L"* array.length, and unpacked on load. This works reliably. Tested it thoroughly. The class works as intended. It's only Flowstone's green that can't cope with such a string. Again, if I encode the bits in such a way, that the resulting string contains printable chars only (e.g. A-Z, a-z, 0-9, a few of the special signs), green handles them. It's only non-printable chars that doesn't work.

[tulamide]

I might be getting on your nerves, but it is an important thing for me, so...

Imagine this: You have 1000 values to store, and they should change asap while changing to a new preset. The values are simple on/off states. Although they would only use one bit each, you write them to a float array. 1000 * 4 bytes = 4000 bytes for simple on/off states.

Instead you now pack the states to bits of Integers. You have to avoid null bytes, so 1000 states fit into 144 bytes. This is only ~ 3% of the amount of data. That is very useful, especially when dealing with presets. You now create a binary string (that's simply a non-human-readable string) and the bits now represent valid ASCII chars.

Flowstone's green strings just ignore those valid strings. But how to use preset string, if you can't use green to transfer them to the preset string prim? Converting each bit to a char doesn't make sense, since you'd then have 1000 bytes again, while the whole effort is to compress data to the allowed size of 255 bytes per preset string.

(I hope this time the issue is more clear)

[KG_is_back]

The problem might be, that ASCII character isn't 8bit, but 7bit (+ the most significant bit is zero = 8bits per character). If the most significant bit is nonzero, the character is not valid ASCII character. Maybe try to create the bitmask and interleave it with "0" every 8th bit (7bits bitmask and 8th bit always zero). This will surely create a string with perfectly valid characters.

[KG_is_back]

Maybe you should consider using hexadecimal representation. That way, each char represents 4 bits. 4*255=1020bits, which fits your range.

[tulamide]

Thanks KG. After your reminder of the original ASCII from 1963 being 7-bit, I was a little worried. A quick test revealed that Flowstone does not just support this oldest of standards, but 8-bit ASCII (which is old enough, too). Setting the msb to 1 in each byte returns a full string as well as setting the lsb of every byte.
Interestingly enough they both have a printable sign representation (although lsb set represents a non-printable char, "Start Of Heading")

Thanks for the hex reminder. It's my fallback if I can't solve this.

[KG_is_back]

I think I have found the problem... when you use "s.each_char.to_a" after .save method it will convert every character into hexadecimal representation. When I displayed the array, second value was "\x00" which is null byte. that would explain why the string is only 1 char long (because the second is null byte = end of string). There is obviously something wrong with the ".save" method or even the full Bitmask class definition.

[tulamide]

that would explain why the string is only 1 char long (because the second is null byte = end of string).

I had the very same assumption. I was already sacrificing one bit per byte (always set to 1) to avoid null bytes. But, and that's so typical again, I made a slip. Instead of setting the lsb of every byte in the four byte integers, I only set the lsb of the first byte, the remaining 3 per integer still at null.

So, I made progress. It's outputting now. Just have to test it with the preset manager now (saving already works, the string is correctly displayed in the .txt file, but I'm having difficulties loading, always an empty string after load).

[tulamide]

...aaaaaannnnnnnddddd working!

Once again proven that 4 eyes see more than 2. Thank you KG