Android Tutorial Custom WebSocket Based Push Framework

The online example is currently disabled due to spam

Now that both B4A and B4J support WebSockets it is possible to implement a full push framework solution.
The solution is made of two components. The client app (B4A) and the server WebApp (B4J).

The client opens and maintains a WebSocket connection with the server.

In the browser you can see the number of active connections and the total number of users (which includes inactive users). You can send a message to all users.

The message is queued in a database and will be delivered to the devices when they connect.

A similar process happens on the device. If the activity is paused then the message is stored with the help of KeyValueStore and a notification is shown. Later when the activity becomes visible the messages are listed on the device.

Note that the device can also send messages to the server (it is not implemented in the demo interface).
It is also possible to send messages to specific ids.

Please give it a try. You need to first download WebSocket library (v1.01+): http://www.b4x.com/android/forum/threads/40221/#content
Run the program and then go to the online console to send a message: http://basic4ppc.com:51042/push/index.html

You can also download the compiled apk and install it.

The server code is available here: http://www.b4x.com/android/forum/threads/webapp-web-apps-overview.39811

B4J client implementation: http://www.b4x.com/android/forum/threads/jwebsocketclient-library.40985/

Edit: Both the server code and device code were updated.
This example can be the base for many types of solutions that require a persistent server connection.

 

[Erel]

Please start a new thread for these questions.

 

[marcick]

Luca vai sicuro, che con un poll ogni 30sec. (keep-alive) il tuo server dopo i primi 10K clienti scoppia... Io l'ho installato su un server cloud di Aruba e va molto bene, ma ogni cliente chiede un socket e i socket sono limitati....
Ciao
Mauro

Hi Mauro, where is exactely the limit ? I have rent a cloud server on Aruba running windows and I manage to write a VB.NET app that receive incoming TCP (or UDP, still have to understand what's better) connection from hundreds of devices (potentially thousands) and store data in a database. It will be a server based fleet tracking application. What do you exactely mean with "socket are limited" ?
Marco

 

[tigrot]

Depending on windows' version there are "commercial" limits in the number of opened connections. Aruba(my preferred provider) has limit only in memory usage. Use TCP, it's safer than UDP. Ask Aruba for the windows version which is better for your deployment.
Ciao
Mauro

 

[marcick]

Hi Mauro, thanks.
Meanwhile I made some progresses. My Aruba cloud server is Windows 2008 64 bit WAMP (just picked from the template list without knowing nothing about the better version). Everything is working fine in TCP now, but still have to discover what happens when there will be hundreds of requests in the same time ....
Ciao
Marco

 

[vfafou]

Hello!
Sometimes after a few hours working, I take the following errors:

java.lang.RuntimeException: org.eclipse.jetty.websocket.api.WebSocketException: RemoteEndpoint unavailable, current state [CLOSED], expecting [OPEN or CONNECTED]
    at anywheresoftware.b4a.keywords.Common.CallSub4(Common.java:455)
    at anywheresoftware.b4a.keywords.Common.access$0(Common.java:426)
    at anywheresoftware.b4a.keywords.Common$CallSubDelayedHelper.run(Common.java:500)
    at anywheresoftware.b4a.keywords.SimpleMessageLoop.runMessageLoop(SimpleMessageLoop.java:30)
    at anywheresoftware.b4a.StandardBA.startMessageLoop(StandardBA.java:26)
    at anywheresoftware.b4j.object.WebSocketModule$Adapter$ThreadHandler.run(WebSocketModule.java:195)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.eclipse.jetty.websocket.api.WebSocketException: RemoteEndpoint unavailable, current state [CLOSED], expecting [OPEN or CONNECTED]
    at org.eclipse.jetty.websocket.common.WebSocketSession.getRemote(WebSocketSession.java:245)
    at anywheresoftware.b4j.object.WebSocket.Flush(WebSocket.java:101)
    at rm.rmtunnel.pushb4a._sendmessages(pushb4a.java:1049)
    at rm.rmtunnel.pushb4a.callSub(pushb4a.java:1184)
    at anywheresoftware.b4a.keywords.Common.CallSub4(Common.java:430)
    ... 10 more


java.lang.RuntimeException: org.eclipse.jetty.websocket.api.WebSocketException: RemoteEndpoint unavailable, current state [CLOSED], expecting [OPEN or CONNECTED]
    at anywheresoftware.b4j.object.WebSocket.setEvents(WebSocket.java:378)
    at anywheresoftware.b4j.object.WebSocketModule$Adapter$ThreadHandler.run(WebSocketModule.java:190)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.eclipse.jetty.websocket.api.WebSocketException: RemoteEndpoint unavailable, current state [CLOSED], expecting [OPEN or CONNECTED]
    at org.eclipse.jetty.websocket.common.WebSocketSession.getRemote(WebSocketSession.java:245)
    at anywheresoftware.b4j.object.WebSocket.sendText(WebSocket.java:107)
    at anywheresoftware.b4j.object.WebSocket.setEvents(WebSocket.java:375)
    ... 6 more


org.eclipse.jetty.websocket.api.WebSocketException: RemoteEndpoint unavailable, current state [CLOSED], expecting [OPEN or CONNECTED]
    at org.eclipse.jetty.websocket.common.WebSocketSession.getRemote(WebSocketSession.java:245)
    at anywheresoftware.b4j.object.WebSocket.Flush(WebSocket.java:101)
    at anywheresoftware.b4j.object.WebSocketModule$Adapter$1.run(WebSocketModule.java:131)
    at anywheresoftware.b4a.keywords.SimpleMessageLoop.runMessageLoop(SimpleMessageLoop.java:30)
    at anywheresoftware.b4a.StandardBA.startMessageLoop(StandardBA.java:26)
    at anywheresoftware.b4j.object.WebSocketModule$Adapter$ThreadHandler.run(WebSocketModule.java:195)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

After these errors, the connections or the application hangs!
Is it possible to be needed DebugNetworkLatency greater than 200?

 

[Erel]

Does it happen in debug mode? DebugNetworkLatency only affects debug mode.

Are you sure that it hangs? The server should continue to run after errors.

 

[vfafou]

Hi Erel! Thank you for your reply!
After errors, it stops getting messages from tablets. It likes as closing the connections.
I have to tell you that I have implemented an AsyncStreams server into the same application in order to receive text messages from several VB6 clients of my LAN, and push them to tablets.
I haven't test it in debug mode with 200 devices needed. It's very difficult because of the infrastructure.
I'm sorry for any bother, but I'm very new to web applications and I'm trying very hard to create a system to talk an old VB6 app with tablets connected to Internet.
The websockets server is running under Linux Ubuntu with Java 1.7.75.
My machine is an HP Proliant 360 G6 with 4core XEON and 16GB RAM.
From time to time, I discover several settings that have to be done, like increasing the network connections of the OS, from 128 to 2048, start JAVA with Xms and Xmx modified e.t.c.
I don't know what else will be needed...
The errors happen when the number of connections is ~200.
I think that I haven't found all settings needed yet.
Any help is highly appreciated, because I have the system on production!

 

[billzhan]

I found a websocket testing mod:
https://github.com/maciejzaleski/JMeter-WebSocketSampler

Jar files built can be found:
http://stackoverflow.com/questions/16152648/websocket-plugin-for-jmeter

I've tested websockets with Jmeter (not b4x clients).
Linode VPS(1G Ram/1 cpu) + ubuntu server/JRE1.7.65 server
To keep the connection, I ping (send dateticks ) every 30seconds, client ->server->client (heartbeat)
When concurrent connection number is 500~800, things are fine in 4 hours.
When concurrent connection number is 1000, JVM crashed after about 1 hour.

Lessons learned:
1. All the resources should be closed when ws disconnected.(sql/timer/ any variable called by other threads)
2. ws disconnections can be found, especially when the package loss>40%.

 

[Erel]

I haven't test it in debug mode with 200 devices needed.

No need to test it in debug mode. I just verified that you test it in release mode.

I ran the example as is on a linux server and it ran for several months before I stopped it due to spam.

 

[vfafou]

Hello Erel!
Is this example multithreaded?
I mean does every PushB4A run on its own thread and the calls make run on is own thread?
I'm asking you because I see that you use CallSubDelayed.

 

[Erel]

Server solutions are always multithreaded. Each websocket connection is handles with its own thread.

I don't know what you did with AsyncStreams but it should be quite difficult to correctly combine AsyncStreams with a server solution as AsyncStreams is build for a different type of applications.

 

[vfafou]

@billzhan,
Thank you for your reply!

You write:

Lessons learned:
2. ws disconnections can be found, especially when the package loss>40%.

My question is: how do you find the package loss percent in B4J?

Thank you in advance!

 

[vfafou]

Server solutions are always multithreaded. Each websocket connection is handles with its own thread.

I don't know what you did with AsyncStreams but it should be quite difficult to correctly combine AsyncStreams with a server solution as AsyncStreams is build for a different type of applications.

AsyncStreams is used from socketserver in order to have managed client LAN connections into the same app that is running the websocket server, in order to push messages from my old VB6 app to the websocket server and vice versa (as real-time as allowed)!
May this is the problem.
Is there any other way to make this communication, with managed clients, into the same server application?

 

[billzhan]

@vfafou

Should be network packets loss ... windows cmd: ping 12.233.23.xx from clients
For linux, you can use mtr (My traceroute).

It should be possible to ping with b4j(java) . FYI https://www.b4x.com/android/forum/threads/jnettools.52723/#content

Mobile connections tend to have higher packets loss

 

[Erel]

Maybe create a small B4J program that will run together with the vb6 app on the client and will connect to the web socket server and the old client.

 

[vfafou]

Maybe create a small B4J program that will run together with the vb6 app on the client and will connect to the web socket server and the old client.

I was thinking of a little different solution: make a B4J separate SocketServer application, running on the server where is placed the web socket server and make one web socket connection. I think that your suggestion will be far better because the server will not take all this load.
Thank you!

 

[Erel]

It is also a possible solution, to break the server into two processes.

 

[vfafou]

Hello!
The following code portions are contained into the server side of this tutorial, inside the class PushB4A.
Please tell me:
Below is a Server Event and we need to call ws.flush.

Public Sub SendMessages(Messages As List)
    ws.RunFunction("NewMessage", Messages)
    ws.Flush
End Sub

Below is a Server Event and... why we don't need to call ws.flush?

Public Sub Device_Ping(Data As Map)
    lastPingTime = DateTime.Now
    CallSubDelayed(PushShared, "UpdateBrowsers")
    ws.RunFunction("Pong", Null)
End Sub

Below is an Event that doesn't count as Server Event, so we don't need to call ws.flush!

Public Sub Device_Id (Data As Map)
    id = Data.Get("id")
    Dim version As Double = Data.Get("version") 'ignore
    CallSubDelayed3(PushShared, "NewConnection", id, Me)
End Sub

Thank you in advance!

 

[Erel]

Device_Ping is an event that was started from the client. Flush is called automatically at the end of client events so you don't need to call flush in this case.

 

[vfafou]

OK, Erel! That's clearer now!