XML Document Rubies

[trogluddite]

Of course, I couldn't leave FS/Ruby alone completely while I had my little break from the forum...

As part of the work on my vector graphics classes, I thought it would be good to use SVG files rather than invent a whole new FS graphics file standard. As SVG files are, like so many, just a type of XML file, I've started by making a bunch of classes for dealing with XML files.

Here's the first Beta - I've followed the W3 rules for XML parsing as far as I can, so it should work with most XML based file types, except for those containing their own custom entity definitions (I'll try to add this later).
The parser should catch most parsing errors for 'badly formed' XML and give error messages with a reasonable indication of what the problem is and where in the file - so feel free to post any examples of files that it is struggling with so that I can refine the code.
I've also made a start on a basic XPath implementation - the files are stored as nested "trees" of XML elements, which you can search using a 'file-path' type notation to find named 'nodes' within the file and check their attribute values - though the syntax checking for search terms is still a little flimsy while I get some experience of the finer points of the XPath standards.

XML Class V100 Beta001.fsm

(20.3 KiB) Downloaded 1541 times

There are a couple of examples and some basic documentation inside the schematic for the important "publicly accessible" classes and methods. The code is also commented for the more "hidden" elements that you might need to look up in order to tailor the parser for more specific types of XML file - I'll hopefully have an example of "customising" soon when I get to work on the SVG file parser that I'm aiming for.

[chackl]

LOL...

I just wanted to ast flowstone for XML support

I'll take a look now

Edit:
Thanks that gives me a realy good example

[chackl]

I'll work out a Module-Based Struture to get this better handled for persons like me that do now want to go as deeb in ruby.
I have alway to remember, that i bought this software because of its grafikal GUI and not typing millions lines of code

[chackl]

I meant something like that

Big thanks for your work - i was waiing for this

As you can see i want to manipulate some data in the VirtualDJs Datbase - this is an huge XML file that contains the data of nearly 20.000 Songs.
I reduced it for the example to 14 Songs ^^

Regards

[trogluddite]

Wow - that's some really nice work there, chakl - and so fast too!
And you're quite right, some 'plug-and-play' modules are just what those Ruby classes needed - no matter how cleanly made a Ruby API is, it becomes too many new methods to learn even for the folks who are comfortable with Ruby - and almost impossible for the non-Rubyists.

Phew! - a pleasant surprise too that the code has worked nicely for you "out of the box" - I was a little worried that I might have posted too soon with not enough testing of different XML 'flavours'.

How are you finding the XML classes to use? - I have been very focused on what I need for my SVG parser, so I'd appreciate any insights you have about using them with different data. I would take too much time, I think, to implement every single Xpath feature etc., but it would be good to know if there are any "big holes" that could do with filling.

[chackl]

That is a realy nice work!

Well as here - my Problem is to load and parse XMLs and sync them (a dj has 2 Laptops and a nas and wants to sync the DB - is not possible)
So i wanted to do it wih flowstone and i needed 10 min for 22000 songs to get each song element in one Array line
You sripts was able to do this within 1 Minute - inklusive loading the txt file to flowstone.

You open file tells me after 10 sec that there is a overrun when i try to load files that are bigger than ~2MB

I'll work out a whole class of modules and do some optimations for the "flow" feeling for XMLs
I'm stiking to the reason of SynthMaker - Programming without write any code - and i hope wie can get other big classes also to flowstone with this "stick-Module" style (just stick them together)

If i find any Errors i'll Report here to you

[trogluddite]

Interesting stuff, thanks for the report.
The 2MB (or maybe time-out?) limit I haven't seen before, I never tried anything that big! Would be interesting to find out exactly which part has the limit.

Do you use the Ruby "Trog::XML::Document.load(path)" method for the loading, or FS 'Load String' primitive? Loading and saving direct through the Ruby will be best, as there will be no "external" FS copy of the file, only the one living inside Ruby - so that should use much less memory.

Also, at the moment the parser must have the whole text before beginning parsing, plus the memory to hold the Ruby objects that it is building. But Ruby is able internally to load/save files in smaller chunks - even one byte at a time - so it may be possible to increase the limit with some changes to the parser.

[Tronic]

Wow!!!
juicy

[chackl]

Well i have troubles here with some problems:

Load XML from file:
Disenables all ruby module because there is something like Excesive claculation -- it may take too long for flowstone

Change Something in XML
Well parsing and viewing data works verry well!
Adding data also seems to work fine.
But if i want to change data i have several problems (maybe just understanding problems)

So for example i want to pick the xml record where FilePath attribute has a spoecific value and change some attributes and child elements.
How could that be done the best way?

Regards

[trogluddite]

Disenables all ruby module because there is something like Excesive claculation -- it may take too long for flowstone

Aah, I was worried that this might be a problem with very big files - very difficult to get around too, because it seems to be just a "time-out" kind of protection; no error in the code is necessary. Possibly some of the code could be made faster, but a way to turn of the protection for "special cases" is the only sure way.
I think it IS right that there is protection against possible bad crashes/file damage, but several of us have asked the dev's for an "expert" no-protection mode - so we will have to see what the next update brings!

How could that be done the best way?

When I add the new code to allow it!
Yes, this is the biggest thing that I still have to do with the XML module. At the moment only parse/add-new can be done, but I am working on the next version that will allow editing/deletion - it is trickier than I first thought, because it must always check that the edits resolve to correct nesting of tags and namespaces.
But editing will be essential for my SVG tools to work well too - so it is definitely the next thing on my "to do" list, and I hope to have the next Beta ready in the next week, with at least some of those features working.