SAVE/LOAD multi-size arrays - must b better way

[RJHollins]

I very much need advice on a very fundamental aspect to most projects I'll be working on.

I've put together an example schematic that outlines a common scenario, and am hoping to get some guidance to get a design that works best.

In my example, I stayed with FS primitives ... I'm certainly not opposed to using some RUBY ... but, I'm still novice with FS ... let alone RUBY

I've try to comment section of the schematic, as I have presented 2 concepts that I've tried to use. The 2 are NOT compatible [in present state] with each other. Each has +'s & -'s to the way they work.

What I hope for is a system that I can implement that handles whatever number of arrays that I may need for a project. The 'size' of these arrays could also vary ... in fact, I have situation where a User loads in an XML file that I'm parsing [as configuration data] ... I won't know the number of entries, so I need to have flexibility.

I'm saving this data to the MY DOCUMENTS folder, and then auto-loading it back in to separated arrays that can then distribute this data throughout the schematic [I've included a typical pull-down menu that I'm feeding with this data.

I could really use some advice and guidance to, once and for all, get this core function resolved in a clean, solid way.

Pre THANKS for all that look at this

Save-Load multi-sized arrays [example]-1.jpg (62.88 KiB) Viewed 20971 times

Save-Load multi-sized arrays [test] - 1.fsm

(12.91 KiB) Downloaded 991 times

[RJHollins]

I have finally been able to reproduce the issue my bud was having ... this is a step forward.

I'm thinking my text arrays are just to bland

When the file gets reloaded, the number of entries for each of the arrays [using the SPLIT prim], can easily reset to ZERO [because the original XML file was not loaded to establish the variable]. This makes sense, and is really a bad way for me to program this logic. Must fix.

I'm wondering if including some type of HEADER in the config file that would contain each of the array sizes could be used to set all the other array Splits ???

just stinkin' out loud

[trogluddite]

Some of this is hinted at in my other post in Chackl's thread about Ruby save/load...

Using the 'Marshal' methods, you can nest all of your arrays into one Ruby "mothership" array, and then save/load the whole thing in a single operation. Not at my FS machine at the moment, but it might look something like this...

Code: Select all

# Assuming you have ruby inputs 'array1', 'array2', 'array3' for your arrays.
# + a string input 'filename' for the path
# + two trigger inputs 'save', and 'load'

def save_arrays
  container = [ @array1, @array2, @array3 ]
  File.open( @filename, "w" ){ |file|  Marshal.dump( container, file) }
end

def load_arrays
  container = File.open( @filename, "r"){ |file| Marshal.load(file) }
  array1, array2, array3 = container
  output 0, array1
  output 1, array2
  output 2, array3
end

def event(in_id)
  case in_id
  when 'load' then load_arrays
  when 'save' then save_arrays
  end
end

The great thing about this is that "Marshal" can handle arrays nested to any depth, and it treats Integer, Float and String arrays exactly the same way. So long as you keep arrays 1,2,3 in the same order going in as coming out, they will be perfectly restored, regardless of length or content.
The only down-side is that the Marshal class turns everything into a String of binary bytes, so the resulting file is not "human readable", and almost impossible to edit. However, that could be done by defining a custom 'Container' class, with its own text formatted 'Marshal' methods - I am working on just such a thing to store MIDI assignments for GUI controls.

[RJHollins]

Hello TROG !

Yes ... I was just reading your post with C.Hackl, and trying to grasp the concept.

When the talk of special classes came up, I figured this was going to deep for me at this point in time. [although I can fully respect the power of Ruby .... in the right hands].

I will definitely look at the code you just posted here, as I really need a solid solution [that importantly I can understand to implement].

To reiterate ... this type of data structure, with the save&load is something that I'd like to have as a 'core' operation that can be modularized and used in all projects [with this design need].

Thank-you for looking in on any of my threads ... I very much appreciate your insights and comments ... always !
Thank-you!

[trogluddite]

No worries, glad to help.

a 'core' operation that can be modularized and used in all projects

That's a big part of the reason I suggest the 'Marshal' way of doing it - so long as you don't mind "unreadable" text output, of course!
You could add/remove/change the input connectors to take any number of Strings, Integers, Floats, Booleans etc. and it would work just as well as for the Arrays in the example.
In that sense 'Marshal' makes everything really generic - the code syntax sure is a bit cryptic, but those two lines of code can handle just about anything you fancy stuffing into the 'container'.
So 'universal' that It's exactly this method that FS uses internally so that Ruby primitives remember their input/output values when you save/load schematic files - and also used for the 'saveState' and 'loadState' methods that are mentioned in the user guide.

[RJHollins]

mmmm .... first test using the Marshal 'plan' ... works !

I only used the 3 arrays from the example I posted above ... hooked up everything.

The output was sent to 3 arrays, and viewed with the TEXT prim on each. MAGIC

ok ... got to play with this some more ... if working like this, then test integration into my current project.

I'll post up the schematic with the Marshal Ruby.

Save-Load multi-sized arrays [test] - 1.fsm

updated

(27.9 KiB) Downloaded 960 times

[RJHollins]

OK ... I couldn't wait to try this in my current project

I made the required mod, and expanded the RUBY code to handle 16 arrays.

I fed in each array, hit SAVE, the LOAD ... it output all 16 arrays, in the correct format and size, to the exact location .... perfectly

I also think the speed on the transfer is just short of instantaneous

As long as I haven't an error in this Marshal integration ... as everything looks excellent ... I've been able to remove a bunch of stuff that is now no longer needed. Man, did that cleanup this section of the schematic

If all good ... there IS one function I would like to implement

The ability to AUTO-SAVE the data [after reading in and parsing the Users XML file] . ... then, execute a AUTO-LOAD that will then re-fill all the destination arrays with the users data, which I have feeding all the special pull-down menus.

I have this all working as stated ... except for triggering the AUTO SAVE and LOAD.

Tonite when I get back from a gig, this will be the next function to add ... then I can send this out for beta-testing.

TROG ... I can't tell you how thankful I am ! This is absolutely HUGE ! An the simplicity is just elegant.

Sincerely.

[RJHollins]

And He's back from the gig ! a bit of Thursday nite JAZZ

The project ... Marshal routine has been implimented in my project and is working ... at least on my system !

Since this entire routine is handling configuration data, the AutoSave and LOAD would make this a background operation to the User ... exactly what I want to do.

Searching this forum ... thought I saw a RUBY trigger example The 'search' feature was giving me grief about my selection of terms ... how rude

So I ventured off [without a net], and am testing this idea ...

To start I added to TROG's code:

Code: Select all

def init
   @status = false
end


then, near the end of the Marshal code I added a status output:

Code: Select all

def event(in_id)
  case in_id
  when 'load' then load_arrays
  when 'save' then save_arrays
  output 17,  @status = true
  output 17, @status = false
  end
end

Hanging off the 17 output, I went GREEN, and added a 'BOOL2TRUE', and sent that to a 'Trigger Delay' ...
From there I thought to link back into the LOAD trigger of this main Marshal module.

The stinkin' ... after the User loads his XML file, and the parser does its' thing ... I would send a trigger to the Marshal SAVE trigger, which upon saving would trigger a delayed LOAD and fill the destination arrays for distribution.

Is this crazy ? am I out of my mind ... .......... yet ?

please be gentle ...

[RJHollins]

testing continues ...

hmm ... not out of the woods yet. All my testing of SAVE & LOAD where to test locations.

For the actual app routine, I had previously been using TROG's 'Save Load Application Data settings file' module.

Beside the AUTO features, there was directory management also taking place.

Things like checking for the existence of a named file, and .... the ability to create a FOLDER in a defined location along with filename. This was all done in GREEN, and I think comes from the SM forum. This is a very handy module.

But now we are in RUBY-land . ... I may need some help with this part It be my luck that something I'd try would end up re-formatting all my drives, and maybe go after my 'Echa-Sketch'

back to searching the forums

[trogluddite]

The Ruby File and Dir classes let you do all of those things with ease.
The most commonly used methods would be...

Code: Select all

File.exists?(file_path)  # returns true/false whether a path exists or not.
File.file?(file_path)  # Return true if the path exists AND is a file.
File.directory?(file_path)  # Returns true if the path exists AND is a folder.
File.delete(file_path1, file_path2, file_path3...)  #  delete files
File.join(string1, string2, string3...)  # Join strings, inserting the backslashes to make a path.
Dir.mkdir(folder_path)  #  Create a new folder
Dir.entries(folder_path)  # Gets an array of all the file names in a given folder
Dir.delete(folder_path)  #  Delete a folder (but only if it is empty!)

Is this crazy ? am I out of my mind ... .......... yet ?

Your logic seems sound enough - verify the contents of the save by re-loading; useful as you might otherwise not see it if the save fails for some reason. Windows 7/8 can be buggers for this as they'll sometimes not save a file as expected because of file permission, but without displaying any error!

To simplify the schematic, I'd suggest doing the delay completely inside Ruby - you can schedule an input or output event to happen some time later very easily.
To do this you use the 'input' or 'output' methods with a third value, which is the time that the event will be sent

Code: Select all

def event(in_id)
  case in_id
  when 'load' then load_arrays
  when 'save'
    save_arrays
    input 'load', nil, time + 1.0  # 1.0 is the delay in seconds
  end
end

It's pretty cool that a Ruby prim can send events to its own inputs like this - but beware of doing it for inputs that actually take a value, as it might clash with whatever value is coming from outside. But it is always safe for trigger inputs.
The 'nil' in this case is just a dummy value, because trigger inputs and outputs will ignore the value anyway. 'nil' is built into Ruby, and is just a completely empty object - so is handy for this as it uses no memory.
The same thing works with the 'output' method, e.g....

Code: Select all

output 0, "Hello", time + 1.0  # Send "Hello" to output 0 after one second.

In this case it is safe to send an actual value - but if all you need is a trigger output, just set the output to be a green trigger type, and you'll get a trigger every time you send ANY output value.

You must remember to use the built in "time" variable when calculating the event time for delays, because the time is absolute (seconds from startup) - so to delay an event, you add the delay to the current time.
The timing will also be much more accurate than a green trigger delay too - if you have audio running in your schematic, right down to a single sample of delay!