Peak (multiple) frequency detection/measurement

[trogluddite]

(typically?) they are expression of 2^N

They absolutely have to be a power of two - the algorithm depends upon this factor. The other other information confirms mostly what I knew, though the overlap feature is somewhat more complex than I understand.

Anhow, some good news and some bad news...

Good News.
I have managed to get the FFT code working...

Ruby FFT 001.fsm

(4.68 KiB) Downloaded 976 times

Comparing against the FFT primitive at low window sizes, confirm that all is working correctly.
Note that, in order to generate freq/phase plots, the inverse FFT doesn't work at the moment - that can easily be rectified once other matters are dealt with...

Bad News.
My worries about the Ruby processing are confirmed. I can get a 32,768 point FFT to work sometimes. But other times, and for larger sizes, the "excess processing" kicks in and freezes the Ruby primitive.
And a warning - be sure to click the 'calculate' trigger only once - if you click again while the Ruby is already running, FS can choke up, and you might have to close and re-open.

It is very likely that the upper limit will depend on available CPU power, so I would be interested to know some comparative tests using different computers - I have been testing on a 4yr old Q6700 machine, maybe a more modern 'i' processor will perform better.

I will also attempt some optimisations, as I think the current program may not be totally efficient...
1 - Sine and Cosine are put into a lookup table for greater speed. If all files are to be analysed with the same settings, there is no need to re-calculate this table every time - it could even be loaded from a file, possibly.
2 - Currently generates frequency and phase outputs. Phase may not be required for your analysis, in which case it could be removed, saving further trig functions.

If I can squeeze better performance out of it, I will let you know ASAP.

[tester]

No phase measurements.
All normalized to mono, 44.1kHz, 16bit.
Gain per pack can be done in separate thread so frequencies only.

The overlap seems to be something rather simple. From the help file - it looks like this (attachement).

I wonder of there are other ways to calculate the FFT itself (math and steps behind). Maybe this is simpler than we think?

[trogluddite]

Ah yes - much simpler with a diagram, got mixed up somewhere understanding the text, I think.

No phase measurements.
All normalized to mono, 44.1kHz, 16bit.

Great, that will simplify things - I'll take the phase maths out of the code and see what improvement it makes.

I wonder of there are other ways to calculate the FFT itself (math and steps behind)

Well, the FFT algorithm itself is pretty standard, optimised for the minimum number of maths operations. But there may well be room for improvement in the Ruby implementation of it.

EDIT) Plus I'm an idiot!!
I have the code open on my other PC, and I just noticed that I left in a "watch" command. He he, the whole array is being converted to text so that I could de-bug - that really isn't going to help!!

I'll get those mod's done and post up again before I go to bed, shouldn't take too long.

[tester]

Plus - Martin suggests that this whole FFT "1-shot" thing for high resolution (with no windowing restrictions) - could be done via standard "code". Do you think it would be complicated, i.e. do you know how to convert such FFT formula into codebox? That would probably solve the ruby vs offline processing part.

[trogluddite]

Do you think it would be complicated, i.e. do you know how to convert such FFT formula into codebox?

In standard DSP code?
The FFT algorithm would be impossible - it requires loops of constantly shifting length that DSP code does not have.
However FFT (Fast Fourier Transform) is only one of a number of Fourier transform methods, and there are also other kinds of analysis such as wavelet transforms or correlation.
Because FFT is so commonly used, it is easy to find good code examples - the other kinds I don't understand well enough to comment on whether DSP code is capable of them, and they are too rarely used to find code examples that I find readable.
Getting a large array of values into a code box (or assembly) is also rather tricky - they are designed for reading incoming samples one at a time, so there is no input/output type for bulk data. That has been on my "wish list" for a long time.
I have no doubt that some form of unlimited anaylsis is possible - and Martin has successfully used the correlation methods in the past for pitch detection - but it is getting into very advanced code that is fighting against limitations every step of the way, just as we are here with the Ruby version.

Here's an update to the Ruby FFT. Some improvement - I can get up to 2^17 samples reliably on my system (takes about 5sec). There's some small room for further optimisation, I think, but I doubt it would be enough to make the step to the next power of two.

Ruby FFT 002.fsm

(2.52 KiB) Downloaded 1006 times

[tester]

2^17 works here too (C2D 2.2gHz at 1gHz save mode), at higher - ruby turns off.
I need some time to prepare an input with file loader.

Do you have some array parser, that will point N indexes at which the value V is:
V at N-1 < V at N > V at N+1
(peaks) ?

Meanwhile maybe we see what Martin finds out.
I only heard that wavelet is useful, but this one is also beyond my head.

[tester]

Meanwhile finishing other projects, here are my thoughts how to approach my idea.

1. Load the audio file
2. Add sharp bandpass butterworth filter to expose "first expected range" and everything else
3. Crappy FFT 16k and general peak detection (Martin, Trog) will find the general position of the pack of peaks.
4. Martin's accurate detection, extended to c.a. 50Hz range - will expose accurate peak frequencies (fragment of file can be lopped), which can be extracted then.
5. According to general position of first pack (pt 3) and designed ratio factor - sharp bandpass filter will be added to other places, where packs of peaks are expected to be. Thus - other areas will be exposed, while surroudings (in their own threads) will be damped.
6. Crappy FFT 16k and peak/max detector (pt 3) will find where are the packs exactly (per paralel thread).
7. Martin's accurate detection will determine peak frequencies in these packs.
8. Meanwhile - other detector will tell what was the maximum (initial) gain per band.
9. And yet another detector will damp everything that is 6-10dB the peak/pack threshold, thus - somehow - lenght detection per pack will be performed.

10. Other automatic calculations and associations can be made according to measurements.

Puff... A lot of work.

But even if it will analyze 1 sound sample per minute, then after two days (48h) it will be around 3000 of samples. Or - 3000 of samples within a week - gives c.a. 7 hours per day of automated work. But i guess it will be faster.

[trogluddite]

Yes, a lot of work - but I think your idea of a "refinement" process, beginning with a coarse measurement is a very good idea - let each analysis technique handle what it does best, and it should be faster than a detailed analysis of the whole spectrum.