Android Question Removing httpjobs pending due to no connection

kostefar · Jan 7, 2017

Dear All,

In this app, in case the network is unreachable and/or the phone is being turned off, I´m building a list of messages pending to be sent to the server whenever there´s a connection. The list is removed as soon as the messages are succesfully sent, which is triggered via a timer event which checks every 5 seconds if there´s anything in the list, and attempts to send the items.
This works well, except for one thing:
Each unsuccesful attempt to send the messages is added to a buffer that gets released once there´s a connection. So if I wait 3x5 = 15 seconds with the connection turned off, then when turning it on again I have the 3 post jobs firing off together with the 1st succesful post event taking place - 4 in total.
So I´d like to find a way to remove these pending events, which obviously are there for a good reason so that indeed offline messages automatically will fire when the connection becomes available. I do want to have control over this though, so that they´ll also fire when the device is turned off while offline, and on again, and this is where b4as own "protection" conflicts with what I´m doing.
I´ve already checked if it´s windows buffering the messages, but JobDone is triggered every time a buffered message is sent, so it has to be in b4a that this is happening.
Is there some flush command I could use for this? Been looking for it, but not able to find it.
Only workaround I can think of is if there´s an event like Jobdone which handles the outgoing data before it´s actually sent to the socket, where I could stop these "ghost" events from the buffer.

Help appreciated, thanks!

sorex · Jan 7, 2017

only send 1 request and wait for it's response.

if successful remove the task from the list/map and do the next one.

if it failed try again later.

don't fire them all at once.

kostefar · Jan 7, 2017

sorex said:
only send 1 request and wait for it's response.

if successful remove the task from the list/map and do the next one.

if it failed try again later.

don't fire them all at once.

Hi Sorex,

Thanks.

"only send 1 request", as opposed to sending one every 5 seconds you mean?
So, in order for that to work, I´d need to find another way to check for wether if the connection is up and running again. Which would be the best way to do this then?
In fact I think you ment sending each item from the list separately. Not sure why that would be a good idea.. the list is a multidimensional array being sent in one job, which I think is much less trouble than sending each item alone. I mean, if the connection is up again, why not send them all in one postjob?

EDIT: Ok, I think I got inspired here.. The timer also checks for new incoming events, so I can use something like: If job is not succesful, trigger flag which is set when connection is offline. When new incoming events is succesful again, it´ll send the array of messages. Still would like to do it the way I first described, by having control over b4as buffered events, as my control and overview would be much better then.

Erel · Jan 8, 2017

kostefar said:
which is triggered via a timer event which checks every 5 seconds if there´s anything in the list, and attempts to send the items.

No reason to use a timer here. You should remove the HttpJobs from the list in JobDone event.

You don't need to send the messages one by one. Send them all and add all the failed jobs (HttpJob.Success = False) to a list. Use a timer or CallSubPlus or whichever other way to later retry to send the failed message.

kostefar · Jan 8, 2017

Erel said:
No reason to use a timer here. You should remove the HttpJobs from the list in JobDone event.

You don't need to send the messages one by one. Send them all and add all the failed jobs (HttpJob.Success = False) to a list. Use a timer or CallSubPlus or whichever other way to later retry to send the failed message.

Thanks Erel, This is actually what I was doing. Although instead of adding them when adding them upon success = false, they´re already added to the list before that, and removed upon success = true. Not sure what you mean with not using a timer, if you later said I should?
The problem with later retrying is that b4a remembers old attempts to send, so duplicates from each timer tick with a failed attempt are fired once the connection is up again.

Anyway, what I came up with actually works. Here are the main parts:

B4X:

Sub JobDone (Job As HttpJob)
If Job.Success = True Then
If offlineflag = True Then
SendOffline
offlineflag = False
End If
Select Job.JobName
Case "postmessagejob"
File.delete (File.DirDefaultExternal & "/" & usernametmp & "/messages/" , "offline")
End Select
Else
offlineflag = True
End if
End Sub

Sub SendOffline
    If File.Exists (File.DirDefaultExternal & "/" & usernametmp & "/messages/" , "offline")    Then
        RAF.Initialize2(File.DirDefaultExternal & "/" & usernametmp & "/messages/" , "offline", False,False)      
            Dim tl As Fivelines
            Dim l As List
            Dim msgstring As String
            Dim jsongen As JSONGenerator
            l.Initialize
            RAF.CurrentPosition = 0
            Do While (RAF.CurrentPosition < RAF.Size )
                Dim m As Map
                m.initialize
                tl = (RAF.Readb4xObject(RAF.CurrentPosition))                 
                m.Put ("subject",tl.msg_subject)
                m.Put ("message",tl.msg_message)
                m.Put ("sender",tl.msg_Sender)
                m.Put ("threadid",tl.msg_threadid)
                m.Put ("msgtype",tl.msg_msgtype)      
                l.add (m)          
            Loop
            jsongen.Initialize2(l)
            msgstring = jsongen.Tostring
            postmessage (msgstring)
    End If
End Sub

kostefar · Jan 9, 2017

Ok, I´ve not solved this yet. Let´s recap:

I´m trying to send let´s say 3 messages separately while the network is down, but b4a apparently keeps a copy of each failed attempt and retries when the network comes back. This does not seem consistant though; sometimes they´re fired, sometimes they´re not.
So I built an array with the 3 messages in it which will fire when the network is back online, and this works great. The problem is the 3 retries that may or may not happen, how can I get rid of them?
Is there no way to either flush this buffer with pending poststrings, or is there an event before jobdone where I can monitor what´s going out and compare it to the array I have, and remove them from the array?
Of course I could also have a ping that triggers an offlineflag if timing out, not allowing the poststring to happen in the first place, but poststring could occur between a succesful and a timed out ping, and then the problem would be there again.
I´ve tried also with StopService(httputils2) and in Sub Service_Destroy I have StartService(httputils2), but the result is the same: Buffers are not emptied although the service is sent to destruction..

EDIT: So, I think I´ve come up with something.. the httpjob Tag value. I can probably add a value to it unique for each message, which will also be in my array. Then I can compare the outgoing tags when the network is up again, and only fire parts of the where there´s none of the individual tags being the same. Not a good solution though, cause I can´t know how long to wait for doing such a check before sending the array: Network could be slow, or the messages could be big. A suggestion to gain more control over this would be appreciated!

I hope I´m clear here.

KMatle · Jan 9, 2017

kostefar said:
but b4a apparently keeps a copy of each failed attempt and retries when the network comes back.

You HAVE full control over your jobs. HttpUtils does NOT try to send the jobs x times (that would be new to me). It just waits for max. 30 secs to connect. There is no retry or buffering of jobs.

So it's a simple wokflow:

1. Send your request (start a job) and wait
2. If it's successful: Start the next one
3. If it's not successful: Try again
4. If your connection is crap: What do you expect? It's like a Porsche in a traffic jam

Additionally:

Decrease the timeout of a HttpJob with

B4X:

    Dim LoginJob As HttpJob
    LoginJob.Initialize("Login", Me)
    LoginJob.PostString(ServerAddress & "/xxx/xxxx.php", JSONstring)
    LoginJob.GetRequest.Timeout=2000 '2 secs to respond

Here if the server isn't responding in 2 secs, the job timeouts (JobDone).

kostefar · Jan 9, 2017

KMatle said:
You HAVE full control over your jobs. HttpUtils does NOT try to send the jobs x times (that would be new to me). It just waits for max. 30 secs to connect. There is no retry or buffering of jobs.

So it's a simple wokflow:

1. Send your request (start a job) and wait
2. If it's successful: Start the next one
3. If it's not successful: Try again
4. If your connection is crap: What do you expect? It's like a Porsche in a traffic jam

Additionally:

Decrease the timeout of a HttpJob with

B4X:

Dim LoginJob As HttpJob LoginJob.Initialize("Login", Me) LoginJob.PostString(ServerAddress & "/xxx/xxxx.php", JSONstring) LoginJob.GetRequest.Timeout=2000 '2 secs to respond

Here if the server isn't responding in 2 secs, the job timeouts (JobDone).

Thank you SO much. This is exactly what I needed!! I guess it´s all a matter of what you call it then, but to me this is a retry:

B4X:

JobName = postmessagejob, Success = false
JobName = postmessagejob, Success = true

I did test it with more than 30 seconds, and it still happens but the timeout, which I was not aware of existed, gave me the full control I was looking for.

My connection is fine, I´m testing this with my firewall blocking all on/off. If there´d be no way to have control over this scenario in the app, people that´ll be using it having not so good connections would end up with a useless app where messages would sometimes arrive multiple times on the server. I could of course build a filter in php to look for message duplicates in the sqb db every time a message arrives, but that´d be alot of unnecessary cpu cycles compared to just setting the timeout so that I´m sure it´ll only be my array being sent. Alternatively, without the array, messages sometimes wouldn´t be sent at all. None of the scenarios are not acceptable.

Thanks again for making me aware, that´s the missing piece I was looking for

Android Question Removing httpjobs pending due to no connection

kostefar

Active Member

sorex

Expert

kostefar

Active Member

Erel

B4X founder

kostefar

Active Member

kostefar

Active Member

KMatle

Expert

kostefar

Active Member

Similar Threads

Android Question Removing httpjobs pending due to no connection

Active Member

Expert

Active Member

B4X founder

Active Member

Active Member

Expert

Active Member

Similar Threads

Privacy & Transparency

Privacy & Transparency