Android Question [SOLVED] Problem getting geofences to work in later versions of Android

[Sandman]

UPDATE: I added Test 4, 5, 6, 7, 8, 9, 10 and 11 to this post. I've also posted a new thread with a follow-up.

This post is entirely based based on the code in https://www.b4x.com/android/forum/threads/geofence-monitoring-a-region-in-the-background.84767/

I'm trying to get geofences to work on Android, and I'm having big problems with that.

To understand the situation better, and narrow down the list of issues, I did a number of tests using the example code linked above. Tests and results are listed below. Unfortunately I haven't been able to figure out what the problem is. I've looked at several non-B4X projects from GitHub, where they target latest version of Android and as far as I can tell we're doing what they are doing, with same stuff in the manifest etc.

I know that Erel mentioned foreground service in a recent thread about geofences, but I haven't investigated that road at all. Both because that none of the other projects I looked at mentioned using that, and because it just seems strange that the geofence functionality would require it. (After all, it's mostly a process of 1. App X informing OS about geofences. 2. User does whatever in whatever apps. 3. When OS notice enter/exit, it launches App X and informs about relevant geofence id.)

So for my testing I had four devices, and I only made small code changes to see if/when something would start working, especially for newer versions of Android.


Methodology

• I manually uninstalled any previous version of the app first
• I restarted each phone before installing new version (probably not necessary)
• I then cleaned the project in the IDE before installing the test apps on each of my four devices
• I made sure to manually check that they all had the required permissions to get access in the background
  • For older versions of Android, I just had to verify that app had access to location
  • For newer versions of Android, I verified that app used the "Always allow" option
• I made sure all devices had wifi enabled
• All tests contained three places I moved between to see if the app would react:
  • Start in geoence: I started the app with me and the devices in the geofence zone, put the app in the background and turned off the screen
  • Out of geofence: I left by car to a safe distance, waited for about an hour or until all devices informed me I left the geofence
  • Back in geofence: I went back to the mothership in the geofence zone and waited another 10-15 minutes to see if they reacted
  • All locations had access to open skies

Test 1 - Baseline

I took the exact code example from the top linked thread, and did only these changes:

1. Changed geofence location to my current location
2. Changed AppLabel to VanillaFence (to make app easier to find in the device)
3. Changed package name to b4a.geofence (to make sure we don't clash with any previous projects)

As-is, the project targets API 26. I left that without change.

Device	OS version	Start in geofence (*)	Out of geofence	Back in geofence
Galaxy S III	4.4.4	Instantly triggered	Instantly triggered	Instantly triggered
Alcatel Pixi	6.0	Instantly triggered	Instantly triggered	Instantly triggered
Nexus 5X	8.1.0	Instantly triggered	No reaction	No reaction
Pixel 3a	11	Instantly triggered	No reaction	No reaction

Note: I was out of geofence for about an hour.



Test 2 - Bump API to 29

Exact same situation as Test 1, but added this change:

1. Target API to 29 (which also the IDE wants me to do)

Device	OS version	Start in geofence (*)	Out of geofence	Back in geofence
Galaxy S III	4.4.4	Instantly triggered	Instantly triggered	No reaction (**)
Alcatel Pixi	6.0	Instantly triggered	Instantly triggered	Instantly triggered
Nexus 5X	8.1.0	Instantly triggered	No reaction	No reaction
Pixel 3a	11	Not tested	Not tested	Not tested

Note: I was out of geofence for about an hour. Also, it wasn't possible to add geofences to Pixel 3a in this test, it didn't have an option for "Allow always"



Test 3 - Bump API to 30 and adjust manifest

Exact same situation as Test 1, but added these changes:

1. Target API to 30
   1. Which we all need to do come August 2021
2. Added AddPermission(android.permission.ACCESS_BACKGROUND_LOCATION) to manifest
   1. Which enables the "Allow always" option for Android 11

Device	OS version	Start in geofence (*)	Out of geofence	Back in geofence
Galaxy S III	4.4.4	Instantly triggered	Instantly triggered	No reaction (**)
Alcatel Pixi	6.0	Instantly triggered	Instantly triggered	Instantly triggered
Nexus 5X	8.1.0	Instantly triggered	No reaction (****)	No reaction
Pixel 3a	11	Instantly triggered (***)	No reaction (****)	No reaction

Note: I was out of geofence for about an hour.



Test 4 - App in foreground

This test is a bit of a sanity check, to see if the issue is related to the app going into the background.

Exact same situation as Test 3, but added these changes:

1. App is in foreground at all times
2. Screen is on at all times

Device	OS version	Start in geofence (*)	Out of geofence	Back in geofence
Galaxy S III	4.4.4	Instantly triggered	Instantly triggered	Triggered after 15 min
Alcatel Pixi	6.0	Instantly triggered	Instantly triggered	Instantly triggered
Nexus 5X	8.1.0	Instantly triggered	No reaction	Triggered after 20 min
Pixel 3a	11	Instantly triggered	Triggered	Triggered

Note: I was out of geofence for about 15 minutes.



Test 5 - Added check for ACCESS_BACKGROUND_LOCATION with API 29

This test is based on the suggestion from OliverA, in the post below. With the exception of API 29, it is identical to Test 6, below.

Exact same situation as Test 1, but added these changes:

1. Target API to 29
2. Added AddPermission(android.permission.ACCESS_BACKGROUND_LOCATION) to manifest
3. Added code snippet for runtimepermissions:
   
   ' START: Code added for extra permission
   rp.CheckAndRequest("android.permission.ACCESS_BACKGROUND_LOCATION")
   Wait For Activity_PermissionResult (Permission As String, Result As Boolean)
   Log("Background permission result: "&Result)
   Log("Note: A false result is not a problem for lower versions of Android, the permission just doesn't exist")
   ' STOP: Code added for extra permission
4. This project is attached as Test5.zip - remember to change to a sane location if you decide to test this

Device	OS version	Start in geofence (*)	Out of geofence	Back in geofence
Galaxy S III	4.4.4	Instantly triggered	Instantly triggered	No reaction (**)
Alcatel Pixi	6.0	Instantly triggered	Instantly triggered	Instantly triggered
Nexus 5X	8.1.0	No reaction	No reaction (****)	No reaction (****)
Pixel 3a	11	Instantly triggered	No reaction (****)	No reaction (****)

Note: I was out of geofence for about 30 minutes.



Test 6 - Added check for ACCESS_BACKGROUND_LOCATION with API 30

This test is based on the suggestion from OliverA, in the post below. With the exception of API 30, it is identical to Test 5, above.

Exact same situation as Test 1, but added these changes:

1. Target API to 30
2. Added AddPermission(android.permission.ACCESS_BACKGROUND_LOCATION) to manifest
3. Added code snippet for runtimepermissions:
   
   ' START: Code added for extra permission
   rp.CheckAndRequest("android.permission.ACCESS_BACKGROUND_LOCATION")
   Wait For Activity_PermissionResult (Permission As String, Result As Boolean)
   Log("Background permission result: "&Result)
   Log("Note: A false result is not a problem for lower versions of Android, the permission just doesn't exist")
   ' STOP: Code added for extra permission
4. This project is attached as Test6.zip - remember to change to a sane location if you decide to test this

Device	OS version	Start in geofence (*)	Out of geofence	Back in geofence
Galaxy S III	4.4.4	Instantly triggered	Instantly triggered	No reaction (**)
Alcatel Pixi	6.0	Instantly triggered	Instantly triggered	Instantly triggered
Nexus 5X	8.1.0	Instantly triggered	Instantly triggered	Instantly triggered
Pixel 3a	11	Instantly triggered	No reaction (****)	No reaction (****)

Note: I was out of geofence for about an hour.



Test 7 - Added long pause between requesting permissions

This test is based on the suggestion from JordiCP. He had the idea that the OS might dislike if we ask for the background permission too quickly after been given the fine permission. I only tested with Android 11, it's the most problematic.

Exact same situation as Test 6, but added these changes:

1. Split into three buttons:
   1. Request permission for FINE
   2. Request permission for BACKGROUND
   3. Add geofence

I waited five minutes between pressing buttons for FINE and BACKGROUND.

Device	OS version	Start in geofence (*)	Out of geofence	Back in geofence
Pixel 3a	11	Instantly triggered	No reaction (****)	No reaction (****)

Note: I was out of geofence for about 30 minutes.



Test 8 - Using foreground service

This test is based on desperation and willingness to try anything. It seems reasonable that one shouldn't have to use a foreground service for this. After all, we give the OS the task of keeping an eye open for our geofences. When it detects us crossing a barrier, it should just start the app and inform what happened. But as that doesn't work, I tried with a foreground service.

Interesting to note was that I tried making the GeofenceService the foreground service, but that made it so no geofence events at all appeared. Instead I created a separate service with a notification containing a basic counter. (To verify that the app was actually alive.)

This worked like a charm, using a foreground notification like this, made Android 11 receive geofence events. As it is, I'm sort of willing to accept this solution, but it still doesn't sit well with me, so I also made Test 9, below.

Exact same situation as Test 6, but added these changes:

1. Foreground service (not the geofence service)

Device	OS version	Start in geofence (*)	Out of geofence	Back in geofence
Galaxy S III	4.4.4	Instantly triggered	Instantly triggered	No reaction (**)
Alcatel Pixi	6.0	Instantly triggered	Instantly triggered	Instantly triggered
Nexus 5X	8.1.0	Instantly triggered	Instantly triggered	Instantly triggered
Pixel 3a	11	Instantly triggered	Instantly triggered	Instantly triggered

Note: I was out of geofence for about 10 minutes.



Test 9 - Disabling battery optimization

The idea this might be battery related came up. And with that, the usual suspect is Battery Optimization. I checked, and the app was indeed battery optimized by default. So I manually changed that so the app wasn't, and did the test. Only with Android 11 this time.

This worked like a charm.

Exact same situation as Test 6, but added these changes:

1. Manually disable Battery Optimization for app before running it

Device	OS version	Start in geofence (*)	Out of geofence	Back in geofence
Pixel 3a	11	Instantly triggered	Instantly triggered	Instantly triggered

Note: I was out of geofence for about 10 minutes.



Test 10 - Using updated example by Erel

This test is based on the updated code Erel posted here. It's pretty much identical to the code in Test 1, with some notable changes. (I haven't done any testing, but I guess that the most important change was to modify the Intent.) It's worth noting this solution doesn't disable battery optimization, and doesn't need a foreground service.

This worked like a charm!

Device	OS version	Start in geofence (*)	Out of geofence	Back in geofence
Galaxy S III	4.4.4	Instantly triggered	Instantly triggered	Triggered after 2 minutes
Alcatel Pixi	6.0	Instantly triggered	Instantly triggered	Instantly triggered
Nexus 5X	8.1.0	Instantly triggered	Triggered after 5 minutes	Triggered after 3 minutes
Pixel 3a	11	Instantly triggered	Triggered after 2 minutes	Instantly triggered

Note: I was out of geofence for about 25 minutes.



Test 11 - Using updated example by Erel - UNFAIR BATTERY SETTINGS

I consider this solved with the solution in Test 10, above. But I was curious what happened if I made things really unfair for the app, from a battery perspective. This means that I verified that all apps were battery optimized (where the OS had the feature), and they were. I also enabled battery saver on all phones, and the Pixel 3a also had the option to restrict the app so it was forbidden to use the battery if in background. I enabled that too.

So, basically, I tried to create a scenario where the user had made things as hard as they could, from a battery perspective. And I also made sure to force-stop all apps before starting the testing.

The result was a mixed bag. The Alcatel Pixi sort of worked, but mostly nobody reacted at all. The Nexus 5X reported out of geofence, but also that it all of a sudden was back in geofence when it was quite far from it. So that's the first completely wrong geofence event I've seen here.

Seems things will get weird if battery saver is enabled, which isn't much of a surprise really. I doubt it's possible to work our way round it from a technical perspective, better to somehow detect that battery saver is enabled and then inform the user that geofences are offline for that reason.



Conclusion

I'm unable to figure out why this doesn't work. Are geofences on Android a crap technology that we shouldn't really use? Or does the example code miss something important for geofences in 2021? (I know it's missing the code for getting to "Always allow", but I handled that step manually, as I wrote above.) I know it's not an exact science with positioning, but surely it's meant to be so good that we would prefer them to a service to start every 15 minutes and use the GPS to get a fix?



Conclusion version 2 (after doing Test 8 and 9)

It seems that it's possible to get geofences to work on Android 11 (API 30). But I'm still not entirely sure this completely correct. As I've written a couple of times, it seems wrong to use a foreground service. And it seems wrong to have to disable battery optimization. If anything, the phone should automatically, temporarily do that if it positions the phone near a registered geofence. Plus the fact that none of the examples I've seen on GitHub (modern, recently updated) mention foreground services or battery optimizations.



Conclusion version 3 (after doing test 10)

I consider this solved. Check the updated code Erel posted for the solution (link in section for Test 10).


(*) As it was related to app launch, the app was always in front at this time, so no background triggering
(**) This result is unexpected - perhaps a bug in Android 4.4.4?
(***) The example code isn't really written for Android 11 so I had to force-quit the app, manually change the permission to "Always allow" and then manually launch the app again. At this point the app succeeded in adding a geofence, which instantly triggered
(****) I even tried to provoke the OS to trigger the geofence by launching Google Maps and getting my current location

 

[OliverA]

In your app (using SDK 29, don't use SDK30 yet), log

rp.Check("android.permission.ACCESS_BACKGROUND_LOCATION ")

If it logs false, add

rp.CheckAndRequest("android.permission.ACCESS_BACKGROUND_LOCATION")

and the appropriate Wait For to your code. For code that does multiple runtime requests, see https://www.b4x.com/android/forum/threads/handle-multiple-permission-request.94611/post-598788
The reason I'm excluding SDK30 is that it may require an additional step (explaining to the user why BACKGROUND LOCATION is needed).

 

[Sandman]

The reason I'm excluding SDK30 is that it may require an additional step (explaining to the user why BACKGROUND LOCATION is needed).

That's relevant for a real-world scenario, but for our testing purposes where it's just me, surely I could target API 30 and just manually set the permission for the app?

As for the rest of your post, I'll adjust the code example and add a fifth test to my first post.

 

[OliverA]

surely I could target API 30

I would not yet. I would try 29 first. If 29 works, then I would go for 30, even though it looks like B4A may just not be there yet 100% (going by this: https://www.b4x.com/android/forum/threads/is-it-possible-to-use-targetsdkversion-30.128192/). With 30, if it does not work, the mechanism to inform the user as to why the permission is needed would need to be sorted out (plus B4A needs to support 30 all the way).

 

[Erel]

plus B4A needs to support 30 all the way

The signing issue related to targetSdkVersion = 30 is resolved in B4A v10.7.

 

[Sandman]

I just updated my first post with three more tests, in the hope of making things clearer for the forum. (I'm still lost on why I can't get it to work.)

 

[Biswajit]

I had the same problem. The device wasn't informing the app about the geofence events. It was working sometimes.
I end up using a sticky service and a custom class as a workaround.

[B4X] GPSPolyMap - check if GPS location is inside a polygon

With this class, you can check if the GPS location coordinate is inside a polygon. You can use this class to check whether the user is inside a building, shopping mall or any custom shape. This class is only for B4A and B4I. Here is how can you get the coordinates of an establishment (school...

www.b4x.com

 

[Sandman]

I had the same problem. The device wasn't informing the app about the geofence events. It was working sometimes.

Thanks for adding to the thread.

Were you able to see differences between the Android versions? And did you try with Android 11?

 

[Biswajit]

Thanks for adding to the thread.

Were you able to see differences between the Android versions? And did you try with Android 11?

No. I tested on Android 8 and 9 two years back when I was working on a project to track user movements inside a shopping mall.

 

[Sandman]

I've gotten it to work now, but I'm not sure it's the intended way. See Test 8 and 9 for more information.

 

[OliverA]

GitHub (modern, recently updated)

Which ones? The ones I'm finding (on the quick) are 6+ months old. One uses BIND_JOB_SERVICE, a permission that is not allowed for us mere mortals.

And it seems wrong to have to disable battery optimization

Not really. Samsung comes to mind. Without disabling battery optimization, Samsung is very aggressive in disabling apps that need to run for a long time/need to service system call backs. And without actually installing them, we don't quite know if these examples would pass your tests.

 

[Sandman]

Not really. Samsung comes to mind. Without disabling battery optimization, Samsung is very aggressive in disabling apps that need to run for a long time/need to service system call backs.

Yes, really. Note that I have no problem at all with my app getting killed, that's fair. But I do have a problem with the OS not waking it up again when a geofence event happens. That seems wrong to me. And disabling battery optimization seems like a bad workaround for that situation. But you don't agree with that view, right?

 

[OliverA]

But you don't agree with that view, right?

What I'm trying to say is that reality sucks when it comes to Android and manufacturers mucking with stock Android functionality. Samsung is not the only manufacturer that throws curveballs, but that is the one I have experience with. Many other manufacturers get mentioned on this site. And since Google does not seem to enforce their standards with the manufacturers/cell service providers, we are stuck with dealing with a mess. Heck, often it is Google that changes it's mind and we are left with dealing with the repercussions. So yes, if a manufacturer forces me to use disable battery optimization in order to keep my app running, then I do that and I do not think it is wrong (it is the proper solution for the given situation).

Edit: BTW, great work in listing the issues and the solutions you derived at. Many people do not go into such great detail.

 

[Sandman]

if a manufacturer forces me to use disable battery optimization in order to keep my app running, then I do that and I do not think it is wrong (it is the proper solution for the given situation)

I agree, I'm also quite pragmatic. I've however come to try to not outsmart Google (or Apple) and instead use their preferred way of solving things. That's a less stressful way forward, and also reduce the risk that I all of a sudden need to come up with another "great" workaround.

In this specific case, I'm trying to update my app, currently it's using gps in the background every five minutes or so. That's the solution I used way back because it worked, and I didn't know better. The app works fine, but I do have a number of users complaining about battery use. So I thought I'd switch to geofences and get rid of excessive battery usage (and complaining users). Plus it's simple to see that I represent one of the bad apps out there that motivate Google to tighten the screws for everybody. :-/

Edit: BTW, great work in listing the issues and the solutions you derived at. Many people do not go into such great detail.

Thank you, it's nice knowing it's appreciated!

 

[Johan Hormaza]

it doesn't work for me. It only works when the Geofence is started and added when it is inside the Geofence but the Output does not register it "transtion = -1". Even when I am out of the Geofence and then I enter it does not register that I entered as I said before, it only happens when the Geofence is initialized. Thanks