Serious memory problem in 3.08.1 on all VST - even stock!

[MyCo]

The negative number turns of the Ruby Timeout changes that Malc added in 3.0.8
If this works, I have at least an idea where the problem is located. Problem is, on my system I only see a minor Memory creeping, it takes several minutes to increase by 1 MB, so don't expect a fix any time soon...because on any code change I've to run FLS for several minutes

[tulamide]

MyCo, you are are fantastic detective! I would have given you a bro-fist, but unfortunately the image size restriction doesn't allow for the typical meme size to show up

Since this issue is critical also for 3.0.9, I appreciate all the time you need to hunt down the cause!

[MyCo]

Well after a lot of digging and trying to find a solution, I'm running out of ideas now. The bug is actually a very old Ruby bug: https://bugs.ruby-lang.org/issues/11174

Looks like whenever you use Threads in Ruby, Ruby doesn't bother to cleanup... and its developers don't give a fuck either. This bug report is 2 years old and still open.

FS uses Ruby Threads since 3.0.8 to limit the execution time of RubyEdit method calls... eg. the "event" method gets actually called like this:

Code: Select all

x = Thread.new {
   send("event",data)
}
x.priority = 100
x.kill unless x.join(timeout)

I've already tried like 10 workarounds for this but none work properly. Only way we could avoid it, would be to only have one thread and re-use it... haven't found a way to do it yet

[Spogg]

As a Ruby-ignorant person I have to ask does this really matter if we can just type in -8000 and get the result we want?

I'm sure this is too simplistic an approach but maybe not...?

Cheers

Spogg

[Walter Sommerfeld]

btw:
There is a neat 'little' side effect with the minus sign...

my monster project used 12 % CPU max on a Quad Core i7-3770 16GB Ram; Win10
now it uses 6%...

[MyCo]

As a Ruby-ignorant person I have to ask does this really matter if we can just type in -8000 and get the result we want?

You don't want to have the new timeout system turned off when developing in Ruby. eg. if you type in "sleep" in a ruby edit, the whole Ruby interpreter will shut off (on all modules) and you cannot turn it back on again. You'll have to restart FS. The new timeout system will only shut down the Thread that timed out and therefor only the ruby edit in which the issue occured.

I haven't found a solution for the memory leak issue, yet. However I've a workaround that actually has the benefit that Walter mentions. I'll release it with the next alpha version for testing. Basically it'll use the new timeout system in FS, but in EXE/VST exports it'll use the old timeout system. So you can develop safely in FS, and your exports will actually run faster (and also avoid the memory leak unless you use Threads on your own)

[Walter Sommerfeld]

Perfect Idea - i second that solution

[Spogg]

Sounds like a good plan MyCo.

Stepping back a bit, since the bug is embedded in the more recent Ruby interpreter there are 2 options: one is for the ruby devs to fix it (seems unlikely from what you say) and the other is to work around the issue with the solution you suggest. The latter path is obviously the way to go!

Cheers

Spogg

[FlowStoner]

Well after a lot of digging and trying to find a solution, I'm running out of ideas now. The bug is actually a very old Ruby bug: https://bugs.ruby-lang.org/issues/11174

Looks like whenever you use Threads in Ruby, Ruby doesn't bother to cleanup... and its developers don't give a fuck either. This bug report is 2 years old and still open.

FS uses Ruby Threads since 3.0.8 to limit the execution time of RubyEdit method calls... eg. the "event" method gets actually called like this:

Code: Select all

x = Thread.new {
   send("event",data)
}
x.priority = 100
x.kill unless x.join(timeout)

I've already tried like 10 workarounds for this but none work properly. Only way we could avoid it, would be to only have one thread and re-use it... haven't found a way to do it yet

This explains many things about why Ruby in FS is so inefficient ....
Using thread in Ruby is not very efficient, always prefer to use Fiber.
In addition ruby in windows uses win32api CreateThread,
It has a well-known memory leak of 80Byte for every call.
I think replacing the system from Thread, with Fiber
should not only solve the problem,
but also improve the efficiency of the entire Ruby system in FS.

[MyCo]

Fiber wouldn't work either, the important part is:

Code: Select all

x.join(timeout)

this basically gives the code a maximum runtime, and is the only reason why threads are used in the first place.