wrapper for Snowboy: a hotword detection engine

Multiverse app · Aug 20, 2017

I have always used pocketsphinx for Hotword recognition in my app. The problem was it was too heavy on resources- especially battery. https://snowboy.kitt.ai/ is a great alternative.

Documentation: http://docs.kitt.ai/snowboy/
Github: https://github.com/kitt-ai/snowboy
Working implementation (from GITHUB): https://github.com/Kitt-AI/snowboy/raw/master/resources/alexa/SnowboyAlexaDemo.apk

Wrapper needed for Android. My budget is 100$.

DonManfred · Aug 26, 2017

Multiverse app said:
Wrapper needed for Android. My budget is 100$.

https://www.dropbox.com/s/u281czkokpriac5/b4aSnowBoy.apk?dl=0

Say Alexa

As of now it is only the detection engine.
You need to create your own pmdl file using their HTTP Api to get multiple keywords recognized i guess. I ust tried existing files. Alexa

B4A Example-Code

B4X:

Sub Globals
    'These global variables will be redeclared each time the activity is created.
    'These variables can only be accessed from this module.
    Dim snowboy As SnowBoy
    Private btnRecord As Button
End Sub

Sub Activity_Create(FirstTime As Boolean)
    'Do not forget to load the layout file created with the visual designer. For example:
    Activity.LoadLayout("Layout1")
    snowboy.Initialize("Snowboy",File.Combine(File.DirRootExternal, "snowboy/common.res"),File.Combine(File.DirRootExternal, "snowboy/alexa.umdl"))
    Log(snowboy.NumHotwords)
End Sub

Sub Activity_Resume
End Sub

Sub Activity_Pause (UserClosed As Boolean)

End Sub
Sub Snowboy_start()
    ' To Capture the audio you should create the output file here and open an outputstream for the audio-data
End Sub
Sub Snowboy_onAudioDataReceived(audio() As Byte, length As Int)
    ' Write the audiobytes to the opened outputstream here
    'Log($"Snowboy_onAudioDataReceived(${i},${audio})"$)
End Sub
Sub Snowboy_stop()
    ' Close the outputstream to the captured audio.
    ' you know the destination of the captured audio.
    ' To train the audio you need to convert the audio-file
    ' to an base64 encoded string.
    '
    ' See http://docs.kitt.ai/snowboy/#api-v1-train
    ' for detailed description on how to train....
    'Log($"Snowboy_stop()"$)   
End Sub
Sub Snowboy_onHotword(hotword As Int)
    Log($"Snowboy_onHotword(${hotword})"$)   
End Sub
Sub btnRecord_Click
    If btnRecord.Tag = "record" Then
        btnRecord.Text = "Start Record"
        btnRecord.Tag = "stopped"
        snowboy.stopRecording
    Else
        btnRecord.Text = "Stop"
        btnRecord.Tag = "record"
        snowboy.startRecording
    End If   
   
End Sub

A sample log-output (logging the Hotword-Event

Logger verbunden mit: 5784193308e57d55
--------- beginning of system--------- beginning of main
*** Service (starter) Create ***
** Service (starter) Start **
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
** Activity (main) Resume **
--G-> recording started ...
Startrecording
--G-> Detected 1 times
Snowboy_onHotword(1)
--G-> recording stopped

Multiverse app · Aug 27, 2017

The app stops on my s8

DonManfred · Aug 27, 2017

Multiverse app said:
The app stops on my s8

Hmm. Can you connect it with usb-debuging and see if you get any logs in the IDE?

Multiverse app · Aug 27, 2017

Got nothing

B4X:

--------- beginning of main
--------- beginning of system
*** Service (starter) Create ***
** Service (starter) Start **
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
*** Service (starter) Create ***
** Service (starter) Start **
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
** Service (starter) Start **
*** Service (starter) Create ***
** Service (starter) Start **
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
** Service (starter) Start **
*** Service (starter) Create ***
** Service (starter) Start **
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
** Service (starter) Start **

DonManfred · Aug 27, 2017

Multiverse app said:
Got nothing

Even in the unfiltered log i cannot find anything useful :-/

Johan Schoeman · Aug 27, 2017

Here is the APK of my "shortcut" wrap. It recognises "alexa" and "snowboy" (and more can be added)
When saying "alexa" it will show a toast with the number "1". For "snowboy" it will show "2"

https://drive.google.com/file/d/0B4g9tud5lvjgS2NTSl9CMEVIVFE/view?usp=sharing

Multiverse app · Aug 27, 2017

Johan Schoeman said:
Here is the APK of my "shortcut" wrap. It recognises "alexa" and "snowboy" (and more can be added)
When saying "alexa" it will show a toast with the number "1". For "snowboy" it will show "2"

https://drive.google.com/file/d/0B4g9tud5lvjgS2NTSl9CMEVIVFE/view?usp=sharing

Doesn't work either

DonManfred · Aug 27, 2017

Multiverse app said:
stops on my s8

I have an S7 EDGE
Do you have a chance to check it on another device?

Which CPU architecture does the S8 have?

Johan Schoeman · Aug 27, 2017

DonManfred said:
I have an S7 EDGE
Do you have a chance to check it on another device?

Which CPU architecture does the S8 have?

It is running on my S4 mini (KitKat) with ARMv7

inakigarm · Aug 27, 2017

Johan Schoeman said:
It is running on my S4 mini (KitKat) with ARMv7

Doesn't work (the APK) on Bq Aquaris M5 Android 6.0.1 (app stops when clicking Start)

Johan Schoeman · Aug 27, 2017

inakigarm said:
Doesn't work (the APK) on Bq Aquaris M5 Android 6.0.1 (app stops when clicking Start)

Can you see anything in the unfiltered logs when you connect your device via USB cable to your PC/Laptop and B4A is active? You need to clear the logs and wait for the log to "stabilize" and then click on the START button and immediately go to the top of the unfiltered log to see what it reports. There must be something....

Does it crash when the very first start button in pressed?

inakigarm · Aug 27, 2017

Johan Schoeman said:
Can you see anything in the unfiltered logs when you connect your device via USB cable to your PC/Laptop and B4A is active? You need to clear the logs and wait for the log to "stabilize" and then click on the START button and immediately go to the top of the unfiltered log to see what it reports. There must be something....

Does it crash when the very first start button in pressed?

Was a permissions question (Settings -> Aplications -> b4asnowboy -> Permissions (activate Magazement and microphone)

Otherwise, how works the app ? (When clicking start, another activity is open with two buttons to record and play the recorded sound but I've not found the recognisement function)

Johan Schoeman · Aug 27, 2017

inakigarm said:
Doesn't work (the APK) on Bq Aquaris M5 Android 6.0.1 (app stops when clicking Start)

As what @DonManfred has asked above - what is the CPU architecture? The lib only provides for ARMv7 at present

Johan Schoeman · Aug 27, 2017

inakigarm said:
Was a permissions question (Settings -> Aplications -> b4asnowboy -> Permissions (activate Magazement and microphone)

Otherwise, how works the app ? (When clicking start, another activity is open with two buttons to record and play the recorded sound but I've not found the recognisement function)

When the activity with the two buttons open:
1. press on start button (left of the Play button).
2. then say "alexa" or "snowboy" - be close to the microphone of your device when saying it. The toast should show.
3. If you press "Stop" (the start button displays "Stop" when you click on it) and then press on "Play" it will play what you have said last.

inakigarm · Aug 27, 2017

Works ! (the app doesn't like my "Alexa" pronunciation, "snowboy" does the job)

DonManfred · Aug 27, 2017

inakigarm said:
Doesn't work (the APK)

Can you also try my version from #2? Does it work for you? The only hotword is "Alexa" at this time.
It should PLING if it detects Alexa said after you start recording

Johan Schoeman · Aug 27, 2017

inakigarm said:
Works ! (the app doesn't like my "Alexa" pronunciation, "snowboy" does the job)

Alexa should also work if you pronounce it close to the way that it was recorded....

inakigarm · Aug 27, 2017

DonManfred said:
Can you also try my version from #2? Does it work for you? The only hotword is "Alexa" at this time.
It should PLING if it detects Alexa said after you start recording

Yes it works (same issue with Permissions--> need to aprove on Settings)

DonManfred · Aug 27, 2017

inakigarm said:
(same issue with Permissions--> need to aprove on Settings

Hmm. What do you mean with you need to aprove?
The targetsdk is set to 22 here to not need runtimepermissions.

wrapper for Snowboy: a hotword detection engine

Active Member

Expert

Active Member

Attachments

Expert

Active Member

Attachments

Expert

Expert

Active Member

Expert

Expert

Well-Known Member

Expert

Well-Known Member

Expert

Expert

Well-Known Member

Expert

Expert

Well-Known Member

Expert

Privacy & Transparency

Privacy & Transparency