Android Example Offline Continuous Speech Recognition

This is a very old project which is unlikely to function correctly on newer versions of Android. The link to the project should still be active and if you want to use it for reference to update to a newer version feel free.

This is a proof of concept port of CMU PocketSphinx for Android demo app.

It's written in B4a using JavaObject to access the PocketSphinx library.

There is a wealth of information on the website : http://cmusphinx.sourceforge.net/ on tailoring the dictionaries, grammar and models.

If you want to control simple tasks or games with a few keywords, it doesn't look like it would be too difficult to implement. I'll let you know in a few weeks when I have integrated it into a current project.

I've used B4a 3.82, so if you're using an older version you will have to change some of the JavaObject calls relating to the array definitions i.e. change them to Array As Object(..) instead of Array(...)

In line with recommendations from CMU the setup copies the files from the assets directory to the apps default directory if it's not there, or the MD5 ID has changed. This is done in a thread in a separate class that has the #ExcludeFormDebugger: True directive, so you should be able to debug the rest of the app.



Download and unzip the library file and copy it to your additional libraries folder.

The project is located on my Google Drive as, with the dictionaries, it is too large (about 10 MB) to upload to the forum you can download it here : SpeechRecognizerContinuous


It's a bit confusing to download if you haven't done it before (as I just found out), click on the file, then there will be a download icon somewhere at the top middle of the screen.

The copyright.txt file contains the original copyright notice from the demo app.

Update to V1.1
  • Replaced threading with wait for
  • Replaced File.DirExternal with RuntimePermissions GetSafeDirDefaultExternal
  • Changed listener event handler (not quite sure why that needed changing)
  • Changed manifest to android:targetSdkVersion="29"
  • Recognizer object is passed back on configuration completion instead of being shared.

  • Updated link to project on Google drive due to their change in security.


Download the new project from SpeechRecognizerContinuous. The library zip is unchanged if you already have it.

Have fun, and let me know how you get on with it.
 

Attachments

  • copyright.txt
    1.5 KB · Views: 1,966
  • Library.zip
    403.1 KB · Views: 960
Last edited:

youjunjer

Member
Licensed User
Longtime User
dear stevel05
I have try your code by thread:
https://www.b4x.com/android/forum/threads/offline-continuous-speech-recognition.42898/
But the event alwayse do Case "onBeginningOfSpeech" and Case "onEndOfSpeech".No any recognition text output?
Is there any more information about this API?

That I want to make a machine like "echo", it will wake up on a special word like "computer" or "master".
And then I can use google Speech Recognition api to make some voice command.
 

stevel05

Expert
Licensed User
Longtime User
This was done 4 years ago, I'm sure Pocket sphinx has moved on since then, the latest info can be found here http://cmusphinx.sourceforge.net/. I'm afraid I haven't used it since the original post so haven't kept up to date with improvements.
 

Sargasso

New Member
Licensed User
Longtime User
Hi!
Im using the latest B4A (7.80) and Android 7+.
And I was searching for "exactly this". An offline (or online) Speech recognition for an app. For me Im going to use it as... Say a word or number and the app check if its correct! So I dont need to know so many words. (But hopefully in many languages. )

So the question: What do you think about this? Or do anyone have something else that's newer and better for 2018?
(I was thinking about starting a new thread but I saw that you have wrote in this thread today so..)

THanks for any idea!
 

DonManfred

Expert
Licensed User
Longtime User
I was thinking about starting a new thread
From the forum Guidelines...

Tips:
  • Post in the correct sub-forum.
  • Use
    B4X:
    code here...
    tags when posting code.
  • All questions should be open to all members.
  • Start a new thread for your question instead of posting in an existing thread.
  • Use the Like button as a gratitude for other members who try to help you.
  • The thread title should summarize the question.
  • Use the search tool.
 

petr4ppc

Well-Known Member
Licensed User
Longtime User
Dear stevel05,

After some time I saw your answer (your update of Cont.Speech.Rec.). Thank you so much for this.
I am trying your project and I get:
"An error has occured in Sub:setuprecognizer ResumableSub.ConfigureRecognizerresume(java line: 101) _keyphrase not found" Continue? Yes-no

If I click on YES, I see only white screen and then nothing happen (I am trying to say : computer, etc)..

I am trying it with Android 11, Manifest editor: targetsdkversion:29 (or 30)
android.jar, SDK=30

Please very much, do you know where I am doing mistake?
Best regards and thank you very much,
p4ppc
 

stevel05

Expert
Licensed User
Longtime User
Sorry, this is not an error I am familiar with, and don't have an android 11 device to test it on. Does anyone recognise the error?
 

petr4ppc

Well-Known Member
Licensed User
Longtime User
Thank you for answer stevel05. For certain I am sending full error (android sdk 30):
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
setuprecognizer$ResumableSub_ConfigureRecognizerresume (java line: 101)
java.lang.RuntimeException: Field: _keyphrase not found in: com.stevel05.speechrecognizer
at anywheresoftware.b4j.object.JavaObject$FieldCache.getField(JavaObject.java:307)
at anywheresoftware.b4j.object.JavaObject.GetField(JavaObject.java:182)
at com.stevel05.setuprecognizer$ResumableSub_ConfigureRecognizer.resume(setuprecognizer.java:101)
at com.stevel05.setuprecognizer._vvv2(setuprecognizer.java:60)
at com.stevel05.setuprecognizer$ResumableSub_DoConfiguration.resume(setuprecognizer.java:212)
at com.stevel05.setuprecognizer._doconfiguration(setuprecognizer.java:191)
at java.lang.reflect.Method.invoke(Native Method)
at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
at anywheresoftware.b4a.keywords.Common$11.run(Common.java:1178)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:246)
at android.app.ActivityThread.main(ActivityThread.java:8512)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:602)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1139)
java.lang.RuntimeException: Field: _keyphrase not found in: com.stevel05.speechrecognizer

and android sdk 28(here I have not installed all components I think):
ObfuscatorMap.txt file created in Objects folder.
Compiling layouts code. (0.00s)
Organizing libraries. (0.00s)
(AndroidX SDK)
Compiling resources (1.32s)
Linking resources Error
bin\extra\compiled_resources\androidx.core.zip@values-am_values-am.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-en-rCA_values-en-rCA.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-hu_values-hu.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-my_values-my.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-gu_values-gu.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-ky_values-ky.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-eu_values-eu.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-hy_values-hy.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@drawable-xhdpi-v4_notify_panel_notification_icon_bg.png.flat: error: failed to read compiled file header.
bin\extra\compiled_resources\androidx.core.zip@values-b+sr+Latn_values-b+sr+Latn.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-nb_values-nb.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@drawable-hdpi-v4_notification_bg_normal_pressed.9.png.flat: error: failed to read compiled file header.
bin\extra\compiled_resources\androidx.core.zip@drawable-mdpi-v4_notification_bg_normal_pressed.9.png.flat: error: failed to read compiled file header.
bin\extra\compiled_resources\androidx.core.zip@layout_notification_template_part_chronometer.xml.flat: error: failed to read compiled file header.
bin\extra\compiled_resources\androidx.core.zip@values-ur_values-ur.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-tr_values-tr.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-sr_values-sr.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-mn_values-mn.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-en-rAU_values-en-rAU.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@layout-v16_notification_template_custom_big.xml.flat: error: failed to read compiled file header.

All errors are detected here:
B4X:
    mRecognizer = Setup.RunMethodJO("defaultSetup",Null) _
    .RunMethodJO("setAcousticModel",Array(JavaFile(ModelsDir,"hmm/en-us-semi"))) _
    .RunMethodJO("setDictionary",Array(JavaFile(ModelsDir,"dict/cmu07a.dic"))) _    
    .RunMethodJO("setKeywordThreshold",Array(Val)) _
    .Runmethod("getRecognizer",Null)
    Log ("cr4")
    'Create Keyword-activation search
   
    Dim ModuleJO As JavaObject = mModule
    Log ("cr41 ***** here")
    mRecognizer.RunMethod("addKeyphraseSearch",Array(ModuleJO.GetField("_kws_search"), ModuleJO.GetField("_keyphrase")))
    Log ("cr5 *****")
    'Create Grammar based searches
    mRecognizer.RunMethod("addGrammarSearch",Array(ModuleJO.GetField("_menu_search"),JavaFile(ModelsDir,"grammar/menu.gram")))
    mRecognizer.RunMethod("addGrammarSearch",Array(ModuleJO.GetField("_digits_search"),JavaFile(ModelsDir,"grammar/digits.gram")))
    mRecognizer.RunMethod("addNgramSearch",Array(ModuleJO.GetField("_forecast_search"),JavaFile(ModelsDir,"lm/weather.dmp")))
I am using Android 11, tested with Android SDK 28 and 30, B4A 10.70
Best regards
p4ppc
 
Last edited:

stevel05

Expert
Licensed User
Longtime User
OK it looks like you are running in obsfuscated mode, you could try changing the references to KEYPHRASE to KEY_PHRASE, including the JavaObject call in the SetupRecognizer module:

B4X:
mRecognizer.RunMethod("addKeyphraseSearch",Array(ModuleJO.GetField("_kws_search"), ModuleJO.GetField("_key_phrase")))
 

petr4ppc

Well-Known Member
Licensed User
Longtime User
OK it looks like you are running in obsfuscated mode, you could try changing the references to KEYPHRASE to KEY_PHRASE, including the JavaObject call in the SetupRecognizer module:

B4X:
mRecognizer.RunMethod("addKeyphraseSearch",Array(ModuleJO.GetField("_kws_search"), ModuleJO.GetField("_key_phrase")))

Dear Stevel05, thank you very much for your advice. In release mode is everything ok, as you wrote.
Best regards
p4ppc
 

Beja

Expert
Licensed User
Longtime User
Hi Steve,

I tried the example but couldn't have it running.. (unfortunately stopped....)
This is the logs:

SpeechRecognizer - 49: Not all code paths return a value. (warning #2)
File 'cmu07a.dic' is not used. (warning #15)
File 'digits.gram' is not used. (warning #15)
File 'feat.params' is not used. (warning #15)
File 'mdef' is not used. (warning #15)
File 'means' is not used. (warning #15)
File 'menu.gram' is not used. (warning #15)
File 'noisedict' is not used. (warning #15)
File 'readme' is not used. (warning #15)
File 'sendump' is not used. (warning #15)
File 'transition_matrices' is not used. (warning #15)
File 'variances' is not used. (warning #15)
File 'weather.dmp' is not used. (warning #15)
Library 'Reflection' is not used. (warning #32)

I set the mic permission.
Any meaning appreciated
 

stevel05

Expert
Licensed User
Longtime User
Updated link to project on Google drive in first post due to their change in security.
 

stevel05

Expert
Licensed User
Longtime User
Sorry @Beja, I missed your post, did you get it running?
 

stevel05

Expert
Licensed User
Longtime User
I have updated the link to my google drive as it was blocked due to a security update and I had a request for it. Please note that this is a very old project if it works for you then great, but I will not be able to support it.
 

omo

Active Member
Licensed User
Longtime User
I have updated the link to my google drive as it was blocked due to a security update and I had a request for it. Please note that this is a very old project if it works for you then great, but I will not be able to support it.
Good day/night/afternoon,@stevel05, I tried to download this your pocketsphnix, especially your latest update but the file is no longer available. I have tried it long time before and looked good; I tried to check my achieve in my hardisk to retry it, but unfortunately; the two hard disks I put them couldn't give me right signal. I am crazy with myself right now because of unexpected misbehaviour of the two harddisks at the same time.

I want to try it again and your new update, but the file seems not available any longer on your Google drive link. Please, if you have the link again, kindly update it. I know there are other newer alternatives in the forum, but I wish I get this again especially your new updates. Thanks
 

stevel05

Expert
Licensed User
Longtime User
I'm sorry, all of my shared files on my google drive appear to have gone. No idea how or when. I will have a backup somewhere, but it's not going to be found quickly. This project is 10 years old and there is no guarantee that it will still work if I can find it, you would be better off looking for a more up to date solution.
 

omo

Active Member
Licensed User
Longtime User
I'm sorry, all of my shared files on my google drive appear to have gone. No idea how or when. I will have a backup somewhere, but it's not going to be found quickly. This project is 10 years old and there is no guarantee that it will still work if I can find it, you would be better off looking for a more up to date solution.
Ok, no problem; I pray you find it. Is still working because I used it at 31 SDK after setting necessary permissions. I prefer it for some reasons and I was able to remove some components to 10MB compare to existing one of 40MB. It also picks voice better if called behind locked phone

Although, one of my crashed hard disks is giving me hope of recovery, but I have never gotten time to attend to it yet. Although, it is not your recent changes, it was older one, but I was able to get what I wanted from it till SDK 31 I last tried. Despite changes in Google policies and android complexities, is still very much working after tweaking a bit.

If you accidentally find it, don't hesitate to put the link back. Is still working under different newer androids if tweaked a bit. Don't discard it yet. Thank you and remain blessed.
 

stevel05

Expert
Licensed User
Longtime User
I pray you find it.
I think you might have got lucky, I don't have time to try it. I'm not set up for running Android apps at the moment, but I came across a full backup of my google drive I did 3 years ago. And there it was. I've updated the link on the first page, I hope it's the one you want.

Steve
 

omo

Active Member
Licensed User
Longtime User
I think you might have got lucky, I don't have time to try it. I'm not set up for running Android apps at the moment, but I came across a full backup of my google drive I did 3 years ago. And there it was. I've updated the link on the first page, I hope it's the one you want.

Steve
Whaoooooooooo, Amazing! @Steve05,thank you so so so....much. My standing ovation for you. I am so grateful, I will download it soon and create time pieces it again. Regards
 
Top