B4A Class Android Speech Recognition API Wrapper

stevel05 · Jan 30, 2016

Here is a wrapper for the Android speech recognition API using JavaObject complete with Source code.

It's based on the code found here: http://www.truiton.com/2014/06/android-speech-recognition-without-dialog-custom-activity/

There is not much to it, if you create a new project, you just need to add the Record Audio permission to your Manifest:

B4X:

AddPermission(android.permission.RECORD_AUDIO)

It supposedly works offline if you download a language pack, but I haven't tried that yet.

A very basic Gui is provided, but you can soon get rid of that if you prefer.

You can see in the code where to change the Language Preference. I'm afraid the best list I could find is on this page: http://developer.android.com/reference/java/util/Locale.html

Full documentation is here: http://developer.android.com/reference/android/speech/SpeechRecognizer.html

Let me know how you get on with it.

canalrun · Jan 21, 2017

JohnCody said:
Is there a way to use an audio file (of recorded speech) as the input audio/speech (to be translated) instead of using the microphone with this lib?

Take a look at this post and thread:
https://www.b4x.com/android/forum/threads/include-so-library-in-b4a-library.74250/#post-472419

It uses Microsoft Cognitive Services rather than Google Speech Recognition, but it does perform recognition of wave files (maybe other types of audio files, also).

Barry.

xpectmore · Jan 21, 2017

JackKirk said:
I have taken stevel05's class, totally tidied it up, added full beep management, made it able to handle offline speech recognition and documented the bejesus out of it - see attached zip containing class and ultra simple example of usage.

Some notes on beep management:

This is done via the XSpeechRecognizer_Beep_Mute_Time parameter of the class's Initialize method.

If set to 0 you will understand the need for beep muting.

If set to 800 (as suggested by Rick Harris in this post:
https://www.b4x.com/android/forum/t...chrecognizer-library.27409/page-3#post-220692
then this will typically eliminate all but a miniscule beep every so often.

If set to a value of 4000-6000 or higher you can totally eliminate beeps - at this level restoration of music volume is occurring at a slower rate than occurrence of error number 6 meaning music volume is never restored until the final Flush in Activity_Pause.

In my view this is not responsible behaviour - if something else wants to use the music channel then the user won't hear it - not sure what that could be (any comment here?)

If you were OK with this behaviour then you could simplify the class by taking out all the music muting/restoration and just mute in the Start method and restore in the Flush method.

Some notes on offline speech recognition:

This was the whole reason for my embarking on this exercise.

It is simply achieved via the following statement in the classes Start method:
SpeechRecognizerIntent.PutExtra("android.speech.extra.PREFER_OFFLINE", True)

But to take advantage of it you have to make sure the phone is configured for offline speech recognition, the user notes in my app on this subject are as follows:

Happy coding...

good job and Steve's too...
for the problem of increase volume/sensitivity of microphone i suggest
http://stackoverflow.com/questions/...ivity-while-recording-audio-in-android-solved
with my 0 java limited experience and almost 5% b4a i am not able to do it

or

B4X:

void     setStreamVolume(int streamType, int index, int flags)

Sets the volume index for a particular stream.

in
https://developer.android.com/reference/android/media/AudioManager.html
or
https://developer.android.com/reference/android/media/AudioManager.html#setStreamVolume(int, int, int)

Rusty · Apr 10, 2017

Is there a way to turn off the microphone?
I am able to initialize; start; receive results; issue a StopListening, but the RMS input keeps firing... To me, this implies the microphone is still listening...
Thanks in advance,
Rusty

Rusty · May 22, 2017

Is there a way to find out the supported languages both online and offline?
Thanks
Rusty

stevel05 · May 22, 2017

There is a link to a list of supported languages in the first post, I don't know if this has been updated since.

Rusty · May 22, 2017

Thanks Steve,
I missed that...
But even with that, I'm not sure how to create a "request" and receive a list of installed language packs...
Any ideas?
Rusty

stevel05 · May 22, 2017

No I couldn't find the answer quickly. Search google and see if you can find some Java code that will do what you want.

Multiverse app · Mar 5, 2018

I am aware that this thread is rather old but I just wanted to thank jackkirk and stevel05 for the above effort

Multiverse app · Mar 6, 2018

This is a small (my first ever) wrap of Jackkirk's work.
Added:

onReadyForSpeech,
onPartialResults,
onBufferReceived,
Now, The Voice recognition automatically detects language and gives results accordingly. Works both offline and online.
More accurate results

MODERN TALKING · Dec 2, 2018

THANK YOU SO MUCH Steve for creating this Library. TOP NOTCH KUDOS to Jack Kirk too

Praharsh, you took it to the stratosphere man. WUNDERBAR. Shukriya bhaisaab

xpectmore · Dec 13, 2018

It's a very nice work and i tested several Speech recognizer and all XSpeechRecognizer versions for 2 weeks in several combinations ...
suppressing that beep is useless,it's also nice work,it's working!

I made some tests for a BASIC b4a AI (artificial intelligence) project and one version ( i made using all resources i found in this forum ) had : the speech recognizer on ,then there is a class strCPU with parsing grabbed text and is returning the expected text for a expected phrase using REGEX ,then is there a tts (a GOOGLE one with the answer) put on a MediaPlayerStream mp.Play and in the same time i found a way (not a headset) while is tts playing i still use the recognizer seccond time to detect words as : "stop" ,"stay" ,"excuse me" and a name : "Cora" (to stop current mp.Play).
At this point if i use that music suppress to suppress beeps i think i close tts as well , tts i need.
SO in this stupid case - my case- your beep suppress is useless

Cesaral · Jan 20, 2023

Multiverse app said:
This is a small (my first ever) wrap of Jackkirk's work.
Added:

onReadyForSpeech,

onPartialResults,

onBufferReceived,

Now, The Voice recognition automatically detects language and gives results accordingly. Works both offline and online.

More accurate results

I have tested this App and it works but I get many times the "Client side error". Why? How can this be solved? Thanks!

Cesaral · Jan 20, 2023

Multiverse app said:
This is a small (my first ever) wrap of Jackkirk's work.
Added:

onReadyForSpeech,

onPartialResults,

onBufferReceived,

Now, The Voice recognition automatically detects language and gives results accordingly. Works both offline and online.

More accurate results

There is an important bug in this App. Where is says at the class:

Private Sub Received_Event (MethodName As String, Args() As Object) As Object

it should be:

Private Sub Received_Event (MethodName As String, Args() As Object)

Fixing this then you start getting the event messages: onPartialResults which are really useful. But fixing this function declaration issue you still get the Client side error at the event handler.

Mike1970 · Jan 24, 2023

Cesaral said:
There is an important bug in this App. Where is says at the class:

Private Sub Received_Event (MethodName As String, Args() As Object) As Object

it should be:

Private Sub Received_Event (MethodName As String, Args() As Object)

Fixing this then you start getting the event messages: onPartialResults which are really useful. But fixing this function declaration issue you still get the Client side error at the event handler.

Hi, I opened an old project where I was using this library and a noticed that now it does not work anymore because it crashes on some list error or shows up this message every X seconds...

How do you fix that error? which file did you change?

Cesaral · Jan 24, 2023

Mike1970 said:
Hi, I opened an old project where I was using this library and a noticed that now it does not work anymore because it crashes on some list error or shows up this message every X seconds...

View attachment 138470

How do you fix that error? which file did you change?

The documentation is so badly done, spread in many sources, and pour in general, that this is a trial-error and a big time-consuming for everybody programming Apps with voice recognition. I changed the file of the class of the example of this post.

Mike1970 · Jan 25, 2023

Cesaral said:
The documentation is so badly done, spread in many sources, and pour in general, that this is a trial-error and a big time-consuming for everybody programming Apps with voice recognition. I changed the file of the class of the example of this post.

Oh ok, you are referring to the original post?

Can you may share your fixed class with an example to help whoever is struggling with this? ?

Mike1970 · Jan 25, 2023

Cesaral said:
The documentation is so badly done, spread in many sources, and pour in general, that this is a trial-error and a big time-consuming for everybody programming Apps with voice recognition. I changed the file of the class of the example of this post.

I tried the code from post #6
with your edit:

Cesaral said:
Private Sub Received_Event (MethodName As String, Args() As Object) As Object

it should be:

Private Sub Received_Event (MethodName As String, Args() As Object)

This is the result after I say a word:

Error occurred on line: 35 (Starter)
java.lang.RuntimeException: Object should first be initialized (List).
at anywheresoftware.b4a.AbsObjectWrapper.getObject(AbsObjectWrapper.java:67)
at anywheresoftware.b4a.objects.collections.List.getSize(List.java:129)
at b4a.example.starter._sr_onresults(starter.java:237)
at java.lang.reflect.Method.invoke(Native Method)
at anywheresoftware.b4a.shell.Shell.runMethod(Shell.java:732)
at anywheresoftware.b4a.shell.Shell.raiseEventImpl(Shell.java:348)
at anywheresoftware.b4a.shell.Shell.raiseEvent(Shell.java:255)
at java.lang.reflect.Method.invoke(Native Method)
at anywheresoftware.b4a.ShellBA.raiseEvent2(ShellBA.java:157)
at anywheresoftware.b4a.BA$2.run(BA.java:395)
at android.os.Handler.handleCallback(Handler.java:907)
at android.os.Handler.dispatchMessage(Handler.java:105)
at android.os.Looper.loop(Looper.java:216)
at android.app.ActivityThread.main(ActivityThread.java:7625)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:524)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:987)

Cesaral · Jan 25, 2023

Mike1970 said:
Oh ok, you are referring to the original post?

Can you may share your fixed class with an example to help whoever is struggling with this? ?

Here is a big part of the code of my class. I just removed the part which is specific to my own App:

B4X:

Sub Class_Globals
    Private JO As JavaObject
    Private RecognizerIntent As Intent
    Private Initialized As Boolean=False
End sub
'
'
Public Sub Initialize(Act As Activity, centrar As Int, CadStart As String, CadEnd As String, sIdioma As String  )
    Act.LoadLayout("VR")

    SpeechRecognizer.InitializeStatic("android.speech.SpeechRecognizer")
    JO = SpeechRecognizer.RunMethod("createSpeechRecognizer",Array(JO.InitializeContext))
    
    If Not(IsRecognitionAvailable) Then
        Log("Speech Recognition Not Available")
        lbl_Error.Text = "Speech Recognition Not Available"
        Return
    End If
    
    RecognizerIntent.Initialize("android.speech.action.ACTION_RECOGNIZE_SPEECH",Null)
    RecognizerIntent.PutExtra("android.speech.extras.SPEECH_INPUT_POSSIBLY_COMPLETE_SILENCE_LENGTH_MILLIS", 1000)
    RecognizerIntent.PutExtra("android.speech.extra.LANGUAGE", sIdioma)
    RecognizerIntent.PutExtra("android.speech.extra.LANGUAGE_MODEL", sIdioma)
    RecognizerIntent.PutExtra("android.speech.extra.LANGUAGE_PREFERENCE" , sIdioma)
    RecognizerIntent.PutExtra("android.speech.RecognizerIntent.LANGUAGE_PREFERENCE", sIdioma)
    RecognizerIntent.PutExtra("android.speech.RecognizerIntent.EXTRA_LANGUAGE", sIdioma)
    RecognizerIntent.PutExtra("android.speech.RecognizerIntent.EXTRA_LANGUAGE_MODEL", sIdioma)
    RecognizerIntent.PutExtra("android.speech.RecognizerIntent.EXTRA_LANGUAGE_PREFERENCE", sIdioma)
    RecognizerIntent.PutExtra("android.speech.extra.MAX_RESULTS",1)
    RecognizerIntent.PutExtra("android.speech.extra.PREFER_OFFLINE", True)
    
    Dim Event As Object = JO.CreateEvent("android.speech.RecognitionListener","Received","")
    JO.RunMethod("setRecognitionListener",Array(Event))
    
End Sub
'
'
Public Sub IsInitialized As Boolean
    Return Initialized
End Sub
'
'
Public Sub IsRecognitionAvailable As Boolean
    Dim JO1 As JavaObject
    JO1.InitializeContext
    Return JO.RunMethod("isRecognitionAvailable",Array(JO1))
End Sub
'
'
Public Sub StartListening
    Log("Sub StartListening")

    JO.RunMethod("startListening",Array(RecognizerIntent))
End Sub
'
'
Public Sub StopListening
    Log("StopListening()")
    JO.RunMethod("stopListening",Null)
End Sub
'
'
Public Sub Destroy
    Log("Destroy()")
    JO.RunMethod("destroy",Null)
End Sub
'
'
Private Sub Received_Event (MethodName As String, Args() As Object)
    'Log("Received_Event=" & MethodName)

    Select MethodName
        Case "onBeginningOfSpeech"
            Log("Received_Event=" & MethodName)
        Case "onReadyForSpeech"
            Log("Received_Event=" & MethodName)
        Case "onError"
            Log("Received_Event=" & MethodName)
        
        Case "onEndOfSpeech"
            Log("Received_Event=" & MethodName)

        Case "onResults"
            Log("Received_Event=" & MethodName)
            Dim Results As JavaObject = Args(0) ' EL RESULTADO MAS PROBABLE VA AL ARRAY 0
            Dim Matches As List = Results.RunMethod("getStringArrayList",Array("results_recognition"))
            'Log("  Size=" & Matches.Size)
            
            Dim Text As String = ""
            For Each Result As String In Matches
                Text = Result
                Exit
            Next
            
            Log("R0=" & Text)
            
        Case "onRmsChanged"
            Log("Received_Event=" & MethodName & "=" &  Args(0))
        
        Case "onPartialResults"
            Dim s As String
            Dim Results As JavaObject = Args(0)
            Dim Matches As List = Results.RunMethod("getStringArrayList",Array("results_recognition"))
            s=Matches.Get(0)
            Log ("PR=" & s)
        Case Else
            Log("*** Received_Event=" & MethodName)
    End Select
    
End Sub
'
'
private Sub GetErrorText(ErrorCode As Int) As String

    Select ErrorCode
        Case SpeechRecognizer.GetField("ERROR_AUDIO")
            Return " Audio Recording error"
        Case SpeechRecognizer.GetField("ERROR_CLIENT")
            Return " cse" ' " Client side error"
        Case SpeechRecognizer.GetField("ERROR_INSUFFICIENT_PERMISSIONS")
            Return " Insufficient permissions"
        Case SpeechRecognizer.GetField("ERROR_NETWORK")
            Return " Network error"
        Case SpeechRecognizer.GetField("ERROR_NETWORK_TIMEOUT")
            Return " Network timeout"
        Case SpeechRecognizer.GetField("ERROR_NO_MATCH")
            Return " No match"
        Case SpeechRecognizer.GetField("ERROR_RECOGNIZER_BUSY")
            Return " RecognitionService busy"
        Case SpeechRecognizer.GetField("ERROR_SERVER")
            Return " error from server"
        Case SpeechRecognizer.GetField("ERROR_SPEECH_TIMEOUT")
            Return " No speech input"
        Case Else
            Return "Didn't understand, please try again."
    End Select

End Sub

B4A Class Android Speech Recognition API Wrapper

Attachments

canalrun

Well-Known Member

xpectmore

Member

Rusty

Well-Known Member

Rusty

Well-Known Member

stevel05

Expert

Rusty

Well-Known Member

stevel05

Expert

Multiverse app

Active Member

Multiverse app

Active Member

Attachments

MODERN TALKING

Active Member

xpectmore

Member

Cesaral

Member

Cesaral

Member

Mike1970

Well-Known Member

Cesaral

Member

Mike1970

Well-Known Member

Mike1970

Well-Known Member

Cesaral

Member

Similar Threads

B4A Class Android Speech Recognition API Wrapper

Attachments

Well-Known Member

Member

Well-Known Member

Well-Known Member

Expert

Well-Known Member

Expert

Active Member

Active Member

Attachments

Active Member

Member

Member

Member

Well-Known Member

Member

Well-Known Member

Well-Known Member

Member

Similar Threads

Privacy & Transparency

Privacy & Transparency