B4A Library Accessibility Services (assisting users with disabilities, automation etc)

This is a partial wrap/library for Android's Accessibility Services which you can read about here:
https://developer.android.com/reference/android/accessibilityservice/AccessibilityService

Android's Accessibility Services is meant to assisting users with disabilities in using Android devices and apps but can also be used for automation.
Some automation tools, such as Tasker, are using Accessibility Services. In this post, I am attaching a sample app which permits you to automate WhatsApp so messages can be sent to contacts and groups without any user-interaction.

It should be noted that an app such as the attached WhatsApp-sample might not be permitted on Google Play since it could be used as a bot unless you are able to motivate that it is purely used for users with disabilities.

I started to write this wrapper for a personal app I am using when riding my motorbike. I have seen many requests for a wrap of the Accessibility Services and initially, I meant to write a complete and generic wrapper but I have realized that it is a huge task and I simply don't have enough time to do so. It is much easier to write a customized wrapper which covers your requirements. If you would like a customized library and you want me to help you, you can contact me by PM.

That said, the attached library contains the most important functionality and should be usable for many projects you might have in mind. Open the sample-app to better understand how it works. You also need to add some things in the Manifest and there are also some configuration-files which need to be set up properly. You can find these files in the Resource-library which is attached.

Read also the docs. Unfortunately, there isn't much documentation to be found on the internet except for the official documentation provided by Google (see the link earlier in this post) although you can find some information/examples on StackOverflow.

22/02/2020 - updated library to version 0.11 (bug fix)
22/02/2020 - updated sample app to use targetSDK = 28


Here is a YouTube video to see the WhatsApp sample app in action:


Please remember that creating libraries and maintaining them takes time and so does supporting them. Please consider a donation if you use my free libraries as this will surely help keeping me motivated. Thank you!

Good luck!

 

[Erel]

You can find the Accessibility library documentation by searching for Accessibility:
https://www.b4x.com/android/forum/pages/results/?query=accessibility

The very first link will take you to the format documentation. Unlike this library it doesn't implement an accessibility service.

 

[Scotter]

You can find the Accessibility library documentation by searching for Accessibility:
https://www.b4x.com/android/forum/pages/results/?query=accessibility

The very first link will take you to the format documentation. Unlike this library it doesn't implement an accessibility service.

Did I miss something? When I did that search before (yesterday) and now, I see nothing on Accessibility. The closest match it found was "access". I'd scroll down through the pages of the search results, except... wouldn't posts on accessibility be above posts on access if I searched for accessibility?

EDIT: OK I went ahead and scrolled down to page two and am finding some posts on accessibility (but Moster67's service). I'm leaving this post up so you can hopefully help me understand why the search results arrived sorted this way? I guess it is prioritizing date? But I have "any time" set in the date dropdown. I'm confused, heh.

 

[Scotter]

@Erel

(1) I'm finding no instruction or other info on the included Accessibility library. Why is this? If the info (tutorials or anything) exists, can you please share a direct link?
Thank you!

(2) Do I need it to be a "service" in order to be able to add text to any object that will get read when a blind person has turned on the accessibility features in their settings?

 

[moster67]

Docs: https://www.b4x.com/android/help/accessibility.html

It shows up fine for me:

Anyway, if you want to discuss the B4A Accessibility library, then you should open a separate thread so this thread is not cluttered.

 

[Scotter]

Docs: https://www.b4x.com/android/help/accessibility.html

It shows up fine for me:

View attachment 75402

Anyway, if you want to discuss the B4A Accessibility library, then you should open a separate thread so this thread is not cluttered.

YES! Thank you thank you thank you!
It's all clear now, hehe. Meaning I see I only need the basic accessibility features of focus order and text description.
Woohoo! Maybe some day I'll need the more feature-rich service you provide but for now, I need super simple.

 

[Jmu5667]

Hi Mike

Got this error this morning:

java.lang.NullPointerException: Attempt to invoke interface method 'java.lang.String java.lang.CharSequence.toString()' on a null object reference
    at com.tillekesoft.accessibilityservices.AccessibilityEventsListenerWrapper.onAccessibilityEvent(AccessibilityEventsListenerWrapper.java:161)
    at android.accessibilityservice.AccessibilityService$2.onAccessibilityEvent(AccessibilityService.java:1538)
    at android.accessibilityservice.AccessibilityService$IAccessibilityServiceClientWrapper.executeMessage(AccessibilityService.java:1724)
    at com.android.internal.os.HandlerCaller$MyHandler.handleMessage(HandlerCaller.java:37)
    at android.os.Handler.dispatchMessage(Handler.java:106)
    at android.os.Looper.loop(Looper.java:193)
    at android.app.ActivityThread.main(ActivityThread.java:6863)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:537)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)

This is the first time we have got this, and having been using the lib for a couple of months and it has been quite stable. Any ideas why this would occur ?

Regards

John.

 

[moster67]

Yep,
Was already fixed some time ago in a separate project. I will try to update the lib here in the forum in the next days.

 

[Jmu5667]

Yep,
Was already fixed some time ago in a separate project. I will try to update the lib here in the forum in the next days.

Great, thank you.

 

[moster67]

I will try to update the lib here in the forum in the next days.

Updated library (see first post in this thread):
22/02/2020 - updated library to version 0.11 (bug fix - see post 26)
22/02/2020 - updated sample app to use targetSDK = 28

 

[Jmu5667]

@OliverA

 

[samperizal]

Greetings.
I tried running the example on a tablet with Android 4.4.2 and it doesn't work. Is there any limitation of operating system version?
Thank you

 

[moster67]

I tried running the example on a tablet with Android 4.4.2 and it doesn't work

Well, the sample project was set to require SDK 21 as a minimum (which you can see if you open the manifest editor). It might work if you reduce the minimum SDK to perhaps 16. I think the methods used in the sample project might go as low as that.

Some of the text related methods in the library itself might require at least SDK 21 if I recall correctly.
In a future update, I may make that clear in the description in the IDE or handle it by code but that really depends on if there is really an interest in this library.

 

[sirjo66]

Otherwise, the java sources are there if anyone else would like to have a look.

Yes, I want to take a look at source code, where I can find it ??

 

[moster67]

Yes, I want to take a look at source code, where I can find it ??

Sorry, but I decided to unpublish it a long time ago since I received lots of questions from users asking me to help them with Java issues and how to implement this and that. I simply did not have time to help all considering I also published it for free after a lot of time studying how to get it working properly with B4A. The very reason I wrote the wrapper was so there was no need for Java and to make a generic version available suitable for most tasks.
My wrapper can be customized to be more specific for certain tasks but then you need to contact me as I wrote in the first post.

 

[Jannete]

Hello, my name is Jannete and i work in a school of people with problems of vision, and i like to use this libary to my app.

Actually i use a tts and voice recognition, but i like add a function called safe number, if the person be lost, they can push a bt button to activate a camera, the camera take a photo from the place and send that in whatsapp to the safe number (with your example i can send only the location, but i like a photo option for more precision).

Thank you, and excuse me for my bad english.

 

[moster67]

Hmm, the WhatsApp sample app is just to show what can be achieved using Accessibility Services and automation. The library itself is not focused on WhatsApp - that was just an example.
To resolve your problem I guess you can use the camera, save the picture and send it as an attachment in a WhatsApp message using intents (if available) or maybe you can use the library to simulate a tap on the photo-button in WhatsApp and then send.

 

[Jannete]

Hmm, the WhatsApp sample app is just to show what can be achieved using Accessibility Services and automation. The library itself is not focused on WhatsApp - that was just an example.
To resolve your problem I guess you can use the camera, save the picture and send it as an attachment in a WhatsApp message using intents (if available) or maybe you can use the library to simulate a tap on the photo-button in WhatsApp and then send.

First, thank you for your answer. I think of something similar:
:

 Dim out As OutputStream

        out = File.OpenOutput(File.DirRootExternal & "/app/", "image.jpg", False)
        b.WriteToStream(out, 100, "JPEG")
        out.Close
        
        Dim FileName As String = "imge.jpg"
        File.Copy(File.DirRootExternal & "/app/", "image.jpg", Starter.Provider.SharedFolder, FileName)
        Dim in As Intent
        in.Initialize(in.ACTION_SEND, "")
        in.SetType("image/jpg")
        in.PutExtra("android.intent.extra.STREAM", Starter.Provider.GetFileUri(FileName))
        in.Flags = 1 'FLAG_GRANT_READ_URI_PERMISSION
        in.WrapAsIntentChooser("Select")
        StartActivity(in)

My app takes a photo with camera libary, but when i try to send, Whatsapp asks to choice a contact. I cant found the Intent to open a share with contact image dialog.

Maybe as you say, i can simulate the touches, but i cant find the way, i see the SendInput library, but it cannot work within a service.

When i see your example, i cant find how emulate the tap events (i only see the tap for send).

Any help would be greatly appreciated.

 

[Jannete]

I find this:

Action: Click
UI Element Class: android.widget.TextView
Ui Element Text: John Doe
UI Element ID: com.whatsapp:id/conversations_row_contact_name

With intent of share image in whatsapp, i can add some similar to select a contact, but i dont know how to add to the code.

 

[JohnC]

Got a question....

In the example, it "clicks" the "Send" button using:

success = acs.PerformNodeActionOnViewWithArgs(True, "com.whatsapp:id/send",AccServConstants.ACTION_CLICK, Null)

But how did you figure out the criteria name of "com.whatsapp:id/send" was the proper criteria to use?

 

[americo Montes]

Funciona perfectamente.
pero trate de utilizarla para enviar mensajes multiples. ahi lastimosamente no funciona ya que envia el mensaje al primer usuario y nada mas..