Java Question Pitch Detection

[techknight]

I saw that a small section of the TarsosDSP library was wrapped for pitch detection and released here on the forums. I just now found it and it is the answer to one of my long standing issues I have had for a while now.

However, Could someone wrap the rest of the pitch detection support/algorithms? I need the Goertzel detection so I can pick a range of frequencies. I am looking for 3 specific frequencies at the same time among the rest of the "noise"

Kinda like DTMF but not.

Thanks

 

[techknight]

This is my halphazard attempt at editing the existing library to make it work:

package smm.dspp;

import anywheresoftware.b4a.AbsObjectWrapper;
import anywheresoftware.b4a.BA;
import anywheresoftware.b4a.BA.ActivityObject;
import anywheresoftware.b4a.BA.Author;
import anywheresoftware.b4a.BA.Hide;
import anywheresoftware.b4a.BA.ShortName;
import anywheresoftware.b4a.BA.Version;
import anywheresoftware.b4a.BA.Permissions;
import anywheresoftware.b4a.BA.DependsOn;
import anywheresoftware.b4a.BA.Events;
import be.tarsos.dsp.AudioDispatcher;
import be.tarsos.dsp.AudioEvent;
import be.tarsos.dsp.io.android.AudioDispatcherFactory;
import be.tarsos.dsp.pitch.PitchDetectionHandler;
import be.tarsos.dsp.pitch.PitchDetectionResult;
import be.tarsos.dsp.pitch.PitchProcessor;
import be.tarsos.dsp.pitch.PitchProcessor.PitchEstimationAlgorithm;
import be.tarsos.dsp.pitch.Goertzel;
import be.tarsos.dsp.pitch.Goertzel.FrequenciesDetectedHandler;

@Author("SMM")
@Version(0.02f)
@Permissions(values={"android.permission.RECORD_AUDIO"})
@ActivityObject
@ShortName("DSP")
@Events(values={"pitchdata(data as float)", "GoertzelData(DetectedFrequencies() as Double, DetectedLevels() as Double"})
@DependsOn(values={"TarsosDSP-Android-2.0-bin"})

public class dspba extends AbsObjectWrapper<AudioDispatcher> {
    private BA ba;
    private String eventName;
    private AudioDispatcher obj;

    public void Initialize(BA ba, String EventName) {
  
    }
  
    /**
    * Initializes the DSP Object as a Single Pitch Frequency Detector
    *EventName = Name of Object/Event that will be raised with the frequency value
    *SampleRate = Sampling frequency of the audio
    *AudioBufferSize = Size of the buffer where the sampled audio will be stored
    *bufferOverlap = Amount of space where the buffers are allowed to overlap
    */  
    public void StartPitchDetect(BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap) {
        _initializepitch(ba, EventName, sampleRate, audioBufferSize, bufferOverlap);
    }

    /**
    * Initializes the DSP Object as a Goertzel Frequency Detector
    *EventName = Name of Object/Event that will be raised with the frequency value
    *SampleRate = Sampling frequency of the audio
    *AudioBufferSize = Size of the buffer where the sampled audio will be stored
    *bufferOverlap = Amount of space where the buffer is allowed to overlap
    *DetectFrequencies = Array of frequencies at which to be detected
    */
    public void StartGoertzelDetect(BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap, double[] DetectFrequencies) {
        _initializegoertzel(ba, EventName, sampleRate, audioBufferSize, bufferOverlap, DetectFrequencies);
    }

    @Hide
    public void _initializepitch(final BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap) {
        this.ba = ba;
        this.eventName = EventName.toLowerCase(BA.cul);
        this.obj = AudioDispatcherFactory.fromDefaultMicrophone(sampleRate, audioBufferSize, bufferOverlap);
        setObject(this.obj);
        ((AudioDispatcher) getObject()).addAudioProcessor(new PitchProcessor(PitchEstimationAlgorithm.FFT_YIN, (float) sampleRate, audioBufferSize, new PitchDetectionHandler() {
            public void handlePitch(PitchDetectionResult pitchDetectionResult, AudioEvent audioEvent) {
                final float pitch = pitchDetectionResult.getPitch();
                ba.activity.runOnUiThread(new Runnable() {
                    public void run() {
                        if (ba.subExists(dspba.this.eventName + "_pitchdata")) {
                            ba.raiseEvent2(ba, false, dspba.this.eventName + "_pitchdata", true, new Object[]{Float.valueOf(pitch)});
                        }
                    }
                });
            }
        }));
        new Thread((Runnable) getObject(), "Audio Dispatcher").start();
        if (this.eventName.length() <= 0) {
        }
    }
  
    @Hide
    public void _initializegoertzel(final BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap, double[] DetectFrequencies) {
        this.ba = ba;
        this.eventName = EventName.toLowerCase(BA.cul);
        this.obj = AudioDispatcherFactory.fromDefaultMicrophone(sampleRate, audioBufferSize, bufferOverlap);
        setObject(this.obj);
        ((AudioDispatcher) getObject()).addAudioProcessor(new Goertzel((float) sampleRate, audioBufferSize, DetectFrequencies, new FrequenciesDetectedHandler() {
            @Override
            public void handleDetectedFrequencies(final double[] frequencies, final double[] powers, final double[] allFrequencies, final double allPowers[]) {
                //final float pitch = pitchDetectionResult.getPitch();
                ba.activity.runOnUiThread(new Runnable() {
                    public void run() {
                        if (ba.subExists(dspba.this.eventName + "_goertzeldata")) {
                            ba.raiseEvent2(ba, false, dspba.this.eventName + "_goertzeldata", true, frequencies);
                        }
                    }
                });
            }
        }));
        new Thread((Runnable) getObject(), "Audio Dispatcher").start();
        if (this.eventName.length() <= 0) {
        }
    }
}

However, I keep getting "Object should first be initialized" when I call my added Goertzel routine.

If I call the Pitch routine, it works fine. I dont get it.

any ideas?

 

[techknight]

bump.

Tried everything I can think of including deleting the pitch code to get it out of the way. Same problem.

I tried to simplify things the best I could, and now I have this:

package smm.dspp;

import anywheresoftware.b4a.AbsObjectWrapper;
import anywheresoftware.b4a.BA;
import anywheresoftware.b4a.BA.ActivityObject;
import anywheresoftware.b4a.BA.Author;
import anywheresoftware.b4a.BA.Hide;
import anywheresoftware.b4a.BA.ShortName;
import anywheresoftware.b4a.BA.Version;
import anywheresoftware.b4a.BA.Permissions;
import anywheresoftware.b4a.BA.DependsOn;
import anywheresoftware.b4a.BA.Events;
import be.tarsos.dsp.AudioDispatcher;
import be.tarsos.dsp.AudioProcessor;
import be.tarsos.dsp.AudioEvent;
import be.tarsos.dsp.io.android.AudioDispatcherFactory;
import be.tarsos.dsp.pitch.PitchDetectionHandler;
import be.tarsos.dsp.pitch.PitchDetectionResult;
import be.tarsos.dsp.pitch.PitchProcessor;
import be.tarsos.dsp.pitch.PitchProcessor.PitchEstimationAlgorithm;
import be.tarsos.dsp.pitch.Goertzel;
import be.tarsos.dsp.pitch.Goertzel.FrequenciesDetectedHandler;

@Author("SMM")
@Version(0.02f)
@Permissions(values={"android.permission.RECORD_AUDIO"})
@ActivityObject
@ShortName("DSP")
@Events(values={"pitchdata(data as float)",
        "GoertzelData(DetectedFrequencies() as Double)"})
@DependsOn(values={"TarsosDSP-Android-2.0-bin"})

public class dspba extends AbsObjectWrapper<AudioDispatcher> {
    private BA ba;
    private String eventName;
    private AudioDispatcher obj;

    public void Initialize(BA ba, String EventName) {
  
    }
  
    /**
    * Initializes the DSP Object as a Single Pitch Frequency Detector
    *EventName = Name of Object/Event that will be raised with the frequency value
    *SampleRate = Sampling frequency of the audio
    *AudioBufferSize = Size of the buffer where the sampled audio will be stored
    *bufferOverlap = Amount of space where the buffers are allowed to overlap
    */  
 //   public void StartPitchDetect(BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap) {
 //       _initializepitch(ba, EventName, sampleRate, audioBufferSize, bufferOverlap);
 //   }

    /**
    * Initializes the DSP Object as a Goertzel Frequency Detector
    *EventName = Name of Object/Event that will be raised with the frequency value
    *SampleRate = Sampling frequency of the audio
    *AudioBufferSize = Size of the buffer where the sampled audio will be stored
    *bufferOverlap = Amount of space where the buffer is allowed to overlap
    *DetectFrequencies = Array of frequencies at which to be detected
    */
    public void StartGoertzelDetect(BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap, double[] DetectFrequencies) {
        _initializegoertzel(ba, EventName, sampleRate, audioBufferSize, bufferOverlap, DetectFrequencies);
    }
/*
    @Hide
    public void _initializepitch(final BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap) {
        this.ba = ba;
        this.eventName = EventName.toLowerCase(BA.cul);
        this.obj = AudioDispatcherFactory.fromDefaultMicrophone(sampleRate, audioBufferSize, bufferOverlap);
        setObject(this.obj);
        ((AudioDispatcher) getObject()).addAudioProcessor(new PitchProcessor(PitchEstimationAlgorithm.FFT_YIN, (float) sampleRate, audioBufferSize, new PitchDetectionHandler() {
            public void handlePitch(PitchDetectionResult pitchDetectionResult, AudioEvent audioEvent) {
                final float pitch = pitchDetectionResult.getPitch();
                ba.activity.runOnUiThread(new Runnable() {
                    public void run() {
                        if (ba.subExists(dspba.this.eventName + "_pitchdata")) {
                            ba.raiseEvent2(ba, false, dspba.this.eventName + "_pitchdata", true, new Object[]{Float.valueOf(pitch)});
                        }
                    }
                });
            }
        }));
        new Thread((Runnable) getObject(), "Audio Dispatcher").start();
        if (this.eventName.length() <= 0) {
        }
    }
*/  
    @Hide
    public void _initializegoertzel(final BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap, double [] DetectFrequencies) {
        this.ba = ba;
        this.eventName = EventName.toLowerCase(BA.cul);
        this.obj = AudioDispatcherFactory.fromDefaultMicrophone(sampleRate, audioBufferSize, bufferOverlap);
        setObject(this.obj);
        ((AudioDispatcher) getObject()).addAudioProcessor(goertzelAudioProcessor);
        new Thread((Runnable) getObject(), "Audio Dispatcher").start();
        if (this.eventName.length() <= 0) {
        }
    }
  
    private final AudioProcessor goertzelAudioProcessor = new Goertzel(44100, 256, new double[]{ 1,2,3,4,5,6,7,8,9,10 }, new FrequenciesDetectedHandler() {
        @Override
        public void handleDetectedFrequencies(final double[] frequencies, final double[] powers, final double[] allFrequencies, final double allPowers[]) {
            //ba.activity.runOnUiThread(new Runnable() {
                //public void run() {
                    if (ba.subExists(dspba.this.eventName + "_goertzeldata")) {
                        ba.raiseEvent2(ba, false, dspba.this.eventName + "_goertzeldata", true, frequencies.length);
                    }
                //}
            //});
        }
    });
}

But I am still getting this:

** Activity (main) Pause, UserClosed = true **
*** Service (starter) Create ***
** Service (starter) Start **
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
Error occurred on line: 59 (Main)
java.lang.RuntimeException: Object should first be initialized (DSP).
    at anywheresoftware.b4a.AbsObjectWrapper.getObject(AbsObjectWrapper.java:50)
    at smm.dspp.dspba._initializegoertzel(dspba.java:94)
    at smm.dspp.dspba.StartGoertzelDetect(dspba.java:62)
    at b4a.example.main._activity_permissionresult(main.java:418)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.shell.Shell.runMethod(Shell.java:732)
    at anywheresoftware.b4a.shell.Shell.raiseEventImpl(Shell.java:348)
    at anywheresoftware.b4a.shell.Shell.raiseEvent(Shell.java:255)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.ShellBA.raiseEvent2(ShellBA.java:144)
    at anywheresoftware.b4a.BA$2.run(BA.java:370)
    at android.os.Handler.handleCallback(Handler.java:873)
    at android.os.Handler.dispatchMessage(Handler.java:99)
    at android.os.Looper.loop(Looper.java:193)
    at android.app.ActivityThread.main(ActivityThread.java:6718)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)

The line is this: dp.StartGoertzelDetect("dp", 44100, 256, 0, Array As Double(1024))

I think ive met my match here considering I am trying to work with a language I have maybe 2% familiarity with.

Whether I call dp.initialize or not makes no difference. I wish I knew what B4A looks for to know something is "initialized" because I think its hiding the real error.

 

[techknight]

If I change the setObject(This.obj); to simply setObject(obj);

the problem goes away. Why is that?

Also even though it doesnt crash anymore, the event never gets raised/nothing happens. So theres still that to figure out.

 

[Erel]

There is no difference between this.obj and obj unless there is also a local variable named obj in the same method.

 

[techknight]

I didnt think so either. but it runs one way, and not the other.

Also I discovered if I pass an unsupported sample rate setting, the same thing happens. the object never gets initialized.

 

[Jim-WI]

The TarsosDSP library was wrapped for pitch detection

Can you send me a link to the pitch detection wrapper? A working example would really help. I'm looking to determine musical notes from a mandolin. There is also a crepe algorithm the is in an ml5 javascript library that appears to give good results.