CSV Text file to Wave Table Read

[aronb]

Hi,

I am trying to use excel to generate a text file that has 3 comma separated values (CSV) when exported to a text file and then bring them into Flowstone and use the" Wave Read Table" module to output the file data / sound... I am using a Ruby code module to do the parse / splitting of the input file - or at least trying too !

Think of it like this - The 3 CSV's are like channels, each column is a channel, 2 columns > 2 channels or "stereo", 3 columns > 3 channels, etc. I hope to do up to 8 channels if I can get this working.

So, I can generate the txt file from excel (no issue), but I am having some type of issue parsing the file in Ruby and getting 3 separate arrays to put into the "Wave Read Table" module. The file may have say 10 to as many as 2,000 or more lines or rows.

The Ruby code seems very close, but then again I am not a Ruby programmer

Can someone here please look over this code and give me some pointers - I need help PLEASE?

Thanks for any help you can give me !

Aron

[KG_is_back]

The fix is easy:

Code: Select all

x << fields[0].to_f
y << fields[1].to_f
z<< fields[2].to_f


You see, when the line is split, the 3 values remain in string format. When they are put out, ruby array(which contains strings) is converted into text. The " are added to denote the values are string - they are not part of the actual string - that's why your code does not work. Also, change the output connectors to float array (right click and select from the dropdown menu).

[aronb]

KG_is_back - Thank You !!!

I thought I was close, and what you said makes sense now that I take a step back. I get too focused and wind up hunting phantoms sometimes !

Now I am looking into the next issue... the output is shorter than it should be, and so I only get a portion of the sample

It has something to do with using "Wave Table" instead of "Float Array to Mem"... I may build something up using the DSP module.

Well back to work !

Aron

[tulamide]

Actually there's something important missing: You don't close the file properly. Use file.close after use, or use the block-form of file.open to automatically close the file after use.

[aronb]

Tulamide Thanks...

I added that (f.close in my case) to the end of my code.

The next problem is reading a binary files, in byte chunks, into an array, that I can then go back to and pull out useful information.

I have some old files that have an old format I/we used in the mid 90's and I would like to get at that data.

Any pointers on reading hex data from a file and storing the array so I can parse through it ? I've looked up bunches of examples but have yet to get any working in Flowstone's Ruby module.

Thanks,

Arom

[KG_is_back]

read about .pack and .unpack methods. They can be used to convert data formats very easily

[aronb]

KG_is_back thanks again,

I used unpack('H*') on the input like this:

# store incoming file into f, input
f = File.open(v,"rb")

# put file f data into xyz array
xyz = f.read.unpack('H*')

But now xyz holds one big huge hex string... and I was trying to use .scan(/../) to break up xyz into "bytes", but Flowstone says it is an "undefined method"... I thought the .scan was a built in method ?

Would seem easy to input a binary file of several hundred hex values "ed8a629f..etc." and put it in an array like "ed,8a,62,9f..etc." Then I could parse thru the data... This would be very easy at least in C++ or even BASIC.

In my file, each of the bytes are either used as bytes or words and I need to find data within this, and about a thousand other binary files so I can get some of my old samples back. I know the format, I just can't see to get Ruby to do my bidding ! It is not as easy as other languages at least for me.

I am in way over my head... certainly did not think Ruby was so difficult I've got to be missing part of the big picture, user appeal, or something in Ruby programming. Did some online tutorials fine, but parsing is very cryptic indeed !

Thanks for any help,

Aron

[tulamide]

I wouldn't blame Ruby. It's more of some concepts that you should read about and some basics you have to learn regarding Ruby. In fact, I never had a language so easy to comprehend as Ruby. At first I was stumbling over some special semantics unique to Ruby, but that's just learning in minutes. The rest is very easy then, because Ruby is consequent. So it's just a matter of learn once, use everywhere.

You want to read binary files byte by byte.

open(...'rb') is one way. Even easier is using .binread, it even closes the file after reading.

Code: Select all

#assuming f is of either File or underlying IO class
f.binread("filename") #reads filename, so provide a full path or set a working directory before
f.binread("filename", 48) #reads 48 bytes of filename
f.binread("filename", 48, 16) #reads 48 bytes of filename starting at byte 16 (0-based index, 16 is an offset)

Now you have a string filled with bytes read from a file. To have an array of numbers you just need to access the bytes.

Code: Select all

#assuming s is the bytestring and a the array
s.each_byte {|b| a << b}

And voila: An array filled with bytes read from a file.

[aronb]

Ok,

I am making progress...

But I am trying to decode bytes (low & high) and it works but not inside my while loop ?

I am at loss again... this would be how I normally would do it, but Ruby and I are at odds again

What is wrong with using a multiply or add here (ruby throws an error undefined method ' * '... why it is valid elsewhere in the same program and it works... ?

Code: Select all

def event i,v,t

# Convert incoming file into 7 seperate
# arrays so they can be turned into
# waveforms for output

   # create arrays to hold everything
   f = ""   # incoming data file
   d = []   # converted data array
   
   # main image data
   x = []   # x axis data points   
   y = []   # y axis data points
   z = []   # z axis data points
   i = []   # int data points
   r = []   # red data points
   g = []   # grn data points
   b = []   # blu data points
   
   # alternate test variables
   xl = 0   # x data low byte
   xh = 0   # x data high byte
   yl = 0   # y data low byte
   yh = 0   # y data high byte
   zl = 0   # z data low byte
   zh = 0   # z data high byte
   x1 = 0   # x dummy value
   y1 = 0   # y dummy value
   z1 = 0   # z dummy value   
   
   # clr trigger output
   output 0, 0   # trigger out low
      
   # store incoming file vslue(s) into f
   # note binread auto closes file !
   # Some binread Examples:
   # f.binread("filename", 48) reads 48 bytes
   # f.binread("filename", 48, 16) reads 48 bytes
   # of filename starting at byte 16
   # (0-based index, 16 is an offset)
   f = IO.binread(v)

   
   # put file f data into d array
   # d should hold entire binary file contents
   f.each_byte {|f| d << f}

   # get position and rate data from header
   # valid positions 0 <--> 42 inclusive
   points = d[39] + (d[40] * 256)
   rate = d[41] + (d[42] * 256)
      
   # get data points for image, starts at
   # position 43 and contains 10 bytes
   # xl, xh, yl, yh, zl, zh, i, r, g, b
   
   p = 0   # start data arrays at 0
         # and will consist of points
         # from 0 to points
         
   k = 43   # start at byte position 43
         # and increment p until
         # p <= points
         
   while p <= points do
      x[p] = d[k+0] + (d[k+1] * 256)
      y[p] = d[k+2] + (d[k+3] * 256)
      z[p] = d[k+4] + (d[k+5] * 256)
      i[p] = d[k+6]
      r[p] = d[k+7]
      g[p] = d[k+8]
      b[p] = d[k+9]
      p = p + 1
      k = k + 10
   end

   # output data array so we can read
   # them into memory for streaming at
   # the current sampling rate   
   output 0,1   # set trigger output
   output 1,x   # x output
   output 2,y   # y output
   output 3,z   # x output
   output 4,i   # i output
   output 5,r   # r output
   output 6,g   # g output
   output 7,b   # b output
   
   # watch list
   watch "filename", v
   watch "points", points
   watch "rate", rate
   watch "file size", f.size
   watch "p",p-1
   watch "k",k-10
end

Thanks again for any help,

Aron

[aronb]

OK...

I believe the issue was my point count went over limit... and the error becomes the first arithmetic operation it cannot do... multiply is the highest, so it errors on the ' * ' / multiply.

Thanks for the help... however if everything goes like I think I will be posting in another few minutes

Aron