Flowstone and data logging/viewing

[tester]

I wonder if someone played with data logging via Flowstone. Practical use of arrays is limited to 16000 items per array/file. Larger array will slow down the app.

But maybe in a "wavefile timeline" style (only instead waves - ascii data)? Something that allows to load larger datasets, zooming and scrolling/paging timeline view, extracting/showing selected data points. Anyone could post something on that?

[trogluddite]

There was a start made with data-logging to HDD file in agc's GPS logger, HERE.

Whether or not the method is suitable will depend on the speed of data events - I have tried it with blue data (i.e. basic HDD audio streaming), but could not make it fast enough to work without glitches, despite trying several different buffering techniques.

For slower events it may be a good method though. Ruby has the ability to read/write any section of data without loading the whole file - which might help to keep the 'internal' array sizes smaller by loading only what is needed. There are also methods to convert float/integer etc. into ASCII data, or even to use raw binary data.

[tester]

Thanks. I guess the target resolution might be around 250sps per channel, so even at 32 channels it would be equivalent of 8kHz mono in total. Recently found interesting chip (TI ADS1298) to gather data of my interest, and I'm starting to consider to build something with it.

What was the limit speed in your app?

Have you played with display routines to process such logs in FS?

[trogluddite]

What was the limit speed in your app?

Don't know for sure - I was just attempting to make a simple audio recorder to grab the audio output of an .exe. So other data-rates and analysis were not part of my goal at the time.

No doubt the results would vary very much depending on the PC used (5yr old Q6700 system is my main machine). I was unable to make anything usable, even for a simple mono/44100Hz stream. The recording was pretty much OK, but terrible interruptions to the ASIO stream made it unbearable to monitor the audio - and that was only with a simple test oscillator, so very little CPU load from DSP/GUI etc.
Probably not that surprising - the Ruby 'frames' system is not very efficient - but my knowledge of buffering algorithms etc. is very basic, so maybe better results are possible with the right code.

Possible that I still have the .fsm somewhere, I will take a look - if so, I'll post it up, and maybe we can see if it could be adapted to generate some more useful tests.