Ruby: Can be done?

[jimicev]

Hi team,
I wonder if it's possible to dynamically stream bitmaps into a ruby module without the memory being saved inside it...

This is my problem:

- while ruby is more efficient than old SM primitives to make knobs per example, using way fewer components
there is still a memory problem related to ruby...

every time you use a button or a knob made in ruby, that uses a bitmap as display... the ruby code will retain in memory the size of the module and its information...

and is the direct result of this?

Regards,
Jimi Smith

[trogluddite]

Welcome to the forum, Jimi.

Storing bitmaps to RubyEdit inputs and outputs was added with FS 3.0.6, and you're quite right, it's a total memory hog and slows down loading. Later versions are supposed to have corrected this according to the release notes, but those have their own problems, so "upgrading" to a more broken version of FS probably isn't wise.

Can it be done some other way? Yes, but it isn't particularly easy to manage. Within Ruby, bitmaps are just objects like any other, and they can be assigned to global variables or constants. Once you have a global reference to a Bitmap object, it can be reached from any other RubyEdit by copying just the reference to it rather than copying all of the Bitmap's data. Here's a very simple example of the principle...

Ruby_shared_bitmap.fsm

(983.33 KiB) Downloaded 841 times

You can duplicate the 'Shared Bitmap Draw' module as many times as you like without racking up memory for copies of the bitmap data, and the bitmap file loader could be deleted once it's not needed any more (or set as 'purgable' to make this automatic).

However, this isn't quite as simple as the few lines of code might make it seem, because global variables and constants aren't confined to a single schematic. They can be seen from every schematic that you have open, and even from modules in the toolbox (which execute Ruby code if it's needed for creating their toolbox thumbnail). So you have to be very careful about naming things to ensure that they will be unique! You also have to be very careful about the order in which you build the schematic and initialize things - the RubyEdit which defines the global variable/constant has to be ready before the shared bitmaps are read anywhere else.

This is something that I've been working on as part of a wider Ruby GUI project - a kind of "bitmap properties" container which is only accessible from within a single schematic and lets you look up the bitmaps by more memorable names which can be chosen by a string property. However, this is some way off, I think - the way that Ruby is integrated into FS results in some rather hacky code that has to be tested to death!

[tulamide]

@trog

I'm not at my PC, so everything is right from my head. Which means I could make a mega derp here. But isn't it much safer to send out a Ruby value instead of using globals?
The main RubyEdit would send the bitmap via RubyValue, which to my knowledge is byRef. A Module Wireless Output with a good name would then serve as connector to all other modules where this bitmap is needed. Should only be stored once.

[trogluddite]

But isn't it much safer to send out a Ruby value instead of using globals?

It's certainly safer, but unfortunately it doesn't prevent the problem with too many copies being memorised (unless any other workarounds have been devised).

When the schematic is saved, whatever is at a RubyEdit input or output is serialised using Marshal.dump - generating a String description of the object's entire structure. At load time, copies of the objects are constructed by Marshal.load, and assigned to @ins, @outs etc. before the init method is called. Even if the bitmap references are nested within a container, the bitmap data becomes part of the container's Marshal data. The same goes for anything stored via the saveState and loadState methods.

Each time Marshal.dump is called, a new String is generated; enough to describe how to make a copy of the original object, but no longer connected to the object's unique identity. So, even if the reference is shared before saving, you still end up with copies of the Marshal data stored into the schematic file, and a bunch of clones when you reload. The same is true for any kind of data stored in this way, it's just particularly noticeable for huge things like Bitmaps!

Some of the FS Ruby classes have another related problem, as not all of them can be Marshalled. For example, you can't store Brushes in saveState or a Ruby Value connector, and it affects quite a few of the others which don't have their own dedicated connector types. The same goes for any user-defined classes; they need working marshal_load and marshal_dump methods for schematic storage to work (for simple classes defined in pure Ruby, the default definitions inherited from Object usually work OK - the FS classes are mostly C-extensions wrapping GDI+ objects, making it a lot harder to do).

BTW: There's a useful little trick that demonstrates how independent Marshal data is. If you want to make a "deep" copy of a container like an Array or Hash; i.e. duplicating all of the nested content as well as the outer container...

Code: Select all

deep_copy = Marshal.load(Marshal.dump(object))

[tulamide]

If you're working on development tools (I'm not sure, if I understood it correctly), maybe a trigger that propagates through all RubyEdits to inform of the pending schematic save would be a way? The bitmap related RubyEdits could then send a simple "nil" class to their outputs before saving. Of course, it is a fragile state then, as those RubyEdits would need to rely on the bitmaps being ready, when they request them, which can be done with careful ordering of the RubyEdits in question. But I think that's a workaround worth the effort. Or not?

[trogluddite]

But I think that's a workaround worth the effort. Or not?

Yes, I think that may be a viable, and simpler, alternative in many cases - e.g. bitmaps are rarely needed until GUI drawing is started, so they could even afford to wait for an 'AfterLoad' trigger. As you say, it's much trickier if values might be needed before the trigger/event system is up and running (indeed, a problem for some of the dev tools I'm working on).

A little experiment shows that the input/output clearing method is actually very easy - you just have to null out the corresponding entry in the @ins or @outs array (which doesn't affect the connector's named instance variable* if it has one, nor sends an output trigger). This can either be done for every change (i.e. in the 'event' method), or only when the schematic is saved (by doing it within the 'saveState' method - always called before connector state is Marshalled). I had forgotten that for a Ruby Value output, there's no possibility of "reverse triggers" which might pull an invalid value from the @outs array, as can happen for 'green' outputs - so it seems more robust than I was originally thinking.

[* EDIT: Correction: The connector-label variable does get cleared, but not until after the methods have returned]

[trogluddite]

Here's a quick example demonstrating the 'connector clearing' method of doing it - one part using 'event' and the other using 'saveState'...

Ruby_shared_bitmap_02.fsm

(444.62 KiB) Downloaded 835 times

There are a few minor drawbacks...

- Contrary to what I said before, clearing the @ins entry within 'event' or 'saveState' does clear the connector-label instance variable (I didn't see it in my initial experiment because it's done after the methods have returned!)

- There's no way for the modules to pick up the bitmap when dragging from the toolbox or copying/pasting - you always have to go back to the source and trigger an update from there (a consequence of no Ruby Value "reverse triggers").

- The drawing modules will have a blank thumbnail if they're put into the toolbox, as they no longer contain the bitmap (not surprising, and unlikely to work as normal whatever way the bitmap was shared).