best way to obtain global presets?

[wlangfor@uoguelph.ca]

I was curious if there were any tried and tested methods better than just CSV using .txt files for global preset data. In My own usage it covers size of plugin, help options, ad popup toggle and so on.

Let Me know please if anyone has made or found something that I could use for this please .

[deraudrl]

Some subset of XML maybe? I've seen some tiny parsers in C/C++, have to believe there's one in Ruby floating around somewhere... here's a place to start:
http://www.ohler.com/dev/xml_with_ruby/xml_with_ruby.html

[trogluddite]

The main problem with anything other than simple CSV is that it's not easy to write a fast parser in FS. The .ini file format isn't too bad (very much like the VST preset text format), but anything much more complex (e.g. needing regular expressions etc.) can get greedy very quickly.

I had a look at the Ruby XML parser libraries in the above link and, as far as I can tell, they all rely on compiled C++ extensions or .dlls. This makes them fast, but very likely to be incompatible with FS, as it insists on binaries compiled using Microsoft Visual Studio, which most Ruby extensions aren't (in the wild, Ruby is used mostly on servers running on UNIX based systems). They also couldn't be embedded within the schematic, so you'd have dependencies on external files which users would have to be careful to install correctly.

I have made an XML parser in native Ruby for FlowStone (link below), but the problem is that doing it that way is very slow for all but the smallest files, and it still involves a lot of Ruby code (though the example could be trimmed down to a much smaller sub-set of XML and have the search features removed).

XML Class V100 Beta001.fsm

(20.27 KiB) Downloaded 1521 times

[tulamide]

@trog

Could you maybe be more specific? What are we talking about if you say "very slow"? A second for a preset? A minute? And which part makes it slow, the parsing, the file access, etc? And at which point would you consider it being fast?

As you can see, I'm trying to find out on what the relatives slow/fast are based under these specific circumstances.

Also, presets are in fxp/fxb format in FS4, which is the standard for vst. So why introducing another format besides preset texts?

[trogluddite]

@tulamide
As rough guide to speed, I'd say it's similar to the Ruby parser in RubyEdits. So, for a 'config' type file of only a few dozen tags, probably not too noticeable. When I tried it on something longer, like some SVG files I was trying to extract data from for GraphicsPaths, with maybe a few hundred elements in several nesting levels, it was slow enough that I even had to increase the Ruby timeout sometimes (though that was on a rather slower machine than I have now!) Of course, it may not be the most efficient code possible, or even the best I could write now, after many years more experience writing Ruby and Regexps.

The slow part is the regular expression matching for elements with arbitrary names which could occur at any arbitrary position in the input string - i.e. lots of any-length, any-place wild-cards. So it could be speeded up quite a bit for a format where the tag names are known in advance and the source is guaranteed to be well formed XML, maybe with limited nesting levels and a specific layout (e.g. every tag on a new line, etc.) But, as you say, once you insist on a more specific format, you might as well choose one that's the simplest possible to parse or is already an accepted standard, like .fxp for presets or .ini for config files. So I agree that there's not much point in re-inventing the wheel!

[tulamide]

@tulamide
As rough guide to speed, I'd say it's similar to the Ruby parser in RubyEdits. So, for a 'config' type file of only a few dozen tags, probably not too noticeable. When I tried it on something longer, like some SVG files I was trying to extract data from for GraphicsPaths, with maybe a few hundred elements in several nesting levels, it was slow enough that I even had to increase the Ruby timeout sometimes (though that was on a rather slower machine than I have now!) Of course, it may not be the most efficient code possible, or even the best I could write now, after many years more experience writing Ruby and Regexps.

The slow part is the regular expression matching for elements with arbitrary names which could occur at any arbitrary position in the input string - i.e. lots of any-length, any-place wild-cards. So it could be speeded up quite a bit for a format where the tag names are known in advance and the source is guaranteed to be well formed XML, maybe with limited nesting levels and a specific layout (e.g. every tag on a new line, etc.) But, as you say, once you insist on a more specific format, you might as well choose one that's the simplest possible to parse or is already an accepted standard, like .fxp for presets or .ini for config files. So I agree that there's not much point in re-inventing the wheel!

Thanks!
I wonder why nowadays everything has to be human-readable, as it seems? My own fileformats often are binary, inspired by RIFF, and therefore don't need to save all the hr overhead, can rely on a defined structure (parser = minimum effort) and are quite fast in the meaning of this discussion.

As we both said, it doesn't really need another format, but I'm curious why people made the shift to hr files? I never saw any human actually trying to read an extensive vector graphic "manually".

[trogluddite]

@tulamide
Likewise; my attitude is that human-readable is only necessary if you also intend human-writeable. In many cases, I think it's a bit risky to reveal so much "implementation detail" in a support file - most users would be much less tempted to tinker with a binary, so they're less likely to break anything. If I need config files for an FS schematic, I usually take the lazy option and put data into a Ruby container that I can Marshal - no parser to write at all!

Now that I mentioned it, Ruby Marshal might be a good answer to LeAttol's original question for 'global' settings. So long as all the data can be sent to a RubyEdit and collected into an Array or Hash, using Marshal is a very easy way to handle it, and allows data to be nested in categories very easily:

Code: Select all

# To Save
File.write(file_path, Marshal.dump(data_container))

# To Load
data_container = Marshal.load(File.read(file_path))


[wlangfor@uoguelph.ca]

Mm, yes; I see the dillema. This subject is by all rights my forte as I used to work with php quite a bit and sql data furthermore.

Writing algorithms that do this sort of thing is pretty common.

I can't help to wonder though; the c parse object, it seems fast; I remember finding it in the mvdlee missing links and it screamed old gem when I first saw it. But with the lack of the old synthmaker forum it really comes down to a lot of guesswork when it comes down to second guessing it.

Has anyone had experience with the c parse primitive? It seems very useful dependent on its success.

[trogluddite]

I must admit, I haven't used the custom parser ever since we got Ruby - Regexes are just so convenient! And AFAIK, no-one has ever tested them against each other for efficiency. My guess is that Regexes would probably be faster, as the Ruby Regex implementation "compiles" expressions into a kind of byte-code, so that effectively, they always run in C++ (so long as you're re-using the same ones, of course - it would be a lot slower if you're creating them on-the-fly). However, that may not be the case - just my hunch.

One advantage that Ruby does have is that you can read a file line-by-line instead of loading the whole file, which would be useful for very big files. But for something "small" like a preset/ini file, that's probably not so much of a factor.

[wlangfor@uoguelph.ca]

I must admit, I haven't used the custom parser ever since we got Ruby - Regexes are just so convenient! And AFAIK, no-one has ever tested them against each other for efficiency. My guess is that Regexes would probably be faster, as the Ruby Regex implementation "compiles" expressions into a kind of byte-code, so that effectively, they always run in C++ (so long as you're re-using the same ones, of course - it would be a lot slower if you're creating them on-the-fly). However, that may not be the case - just my hunch.

One advantage that Ruby does have is that you can read a file line-by-line instead of loading the whole file, which would be useful for very big files. But for something "small" like a preset/ini file, that's probably not so much of a factor.

Yeah, lately lol.. I have been literally making like three versions of a project just so I can test which one is more efficient than the other.

It's very un-efficient lol. Maybe I'll try and figure out the c parser, hmm. I think I can so I will try and post here regarding results.