Android Tutorial [B4X] Simple audio call over internet with own TURN/relay server

Hi everyone,

I posted a thread long ago about the audio/video calling over the internet. Here is a simple example of how you can call almost (will discuss below) realtime.

Why only a simple audio calling example while I said that I will post both audio and video calling examples?
While working on this project I realized that a good quality audio/video calling application with all required functionalities needs huge time and effort. Because like whatsapp, google duo if you want faster calling app then you have to compress the audio/video buffer before sending and decompress the audio/video buffer after receiving. This may sound like a simple byte compression job but it isn't. Compressing/decompressing audio/video buffer in realtime and playing requires lots of research about audio/video processing, audio/video codes, how conversion works, and many other things. Thus I decided to post a simple example of how it works for free and will post a full example that will be chargeable. That will have lots of functionalities, check below to know more.

This will work,
  1. From Android to Android
  2. From Android to iOS
  3. From iOS to Android
  4. From iOS to iOS
Functionalities:
  1. Apps:
    1. List all the connected users and update in realtime
    2. Call any connected users
    3. Check if the user is busy or not
    4. Show custom in-app incoming call screen with accept/reject button
    5. Play ringtone for incoming call
    6. Automatic screen dimming while talking
    7. Call accepted/rejected/ended/busy/offline status
    8. Call timer
    9. Incoming call notification while the app is in the background (Android)
    10. Noise cancellation, Acoustic echo cancellation, Automatic gain control (Android)
  2. Server:
    1. Can handle multiple connections simultaneously.
    2. Deliver call requests and status
    3. Broadcast connected user list
    4. Check if the callee is busy or not
    5. Work as a relay server for delivering audio buffers to respective clients
Limitations:
  1. Few milliseconds delay in delivering audio buffer (sometimes). Because,
    1. The example apps are processing uncompressed audio buffer (24Kb-30Kb buffer per seconds)
    2. The test server (you can find the IP inside the project) has been configured to limit bandwidth usage for testing purposes.
  2. No notifications when the app is closed. I haven't included that because it's a simple example.
How to run:
  1. App:
    1. Download latest socket.io client library wrapper (B4A / B4I)
    2. Download, extract and run.
    3. For ios, you need a working developer certificate because it will not work on the simulator.
  2. Server: The test server is always up. If you want to test with the test server then just run the app. If you want to run it on your local server, then
    1. Install Node.js
    2. Extract the server code
    3. Open terminal/cmd and navigate to the folder where you have extracted the code
    4. Run npm install (it will take few minutes)
    5. Run node app.js
    6. That's it.
Functionalities that the full example will have,
  1. Audio/Video call
  2. End to end encryption
  3. Switching audio output between speakers/headphone
  4. Faster calling even with low internet speed (5-10kbps)
  5. Incoming calling screen that will show even when the app is not running.
  6. Register a name for your device. So the other users can see who is calling
  7. Example of STUN server as default server and fallback to TURN server
  8. Example of Socket.IO server along with WebSocket Server
  9. *Maybe group calling.
The full example will be chargeable. I haven't decided the amount yet. It will be between $50 - $100.

Though the test server has bandwidth limitations, it can handle more than 100-200 users.

Download the test apk for testing.
 

Attachments

  • [B4A] Socket.io Simple Call Example.zip
    116.8 KB · Views: 1,152
  • [B4i] Socket.io Simple Call Example.zip
    111.7 KB · Views: 874
  • [Server] Socket.io Simple Call Example.zip
    2.4 KB · Views: 1,096
Last edited:
Click to expand...
MarcoRome

MarcoRome

Expert
Licensed User
Longtime User
Super job. Waiting for an Audio / Video demo. If everything is ok, I will certainly make the purchase.
At the moment if you use a wi-fi it gives problems (the voice comes with a flickering after several seconds), on 4G sometimes it is very fluid, other times there is a delay of up to 3-4 seconds
 
Biswajit

Biswajit

Active Member
Licensed User
Longtime User
MarcoRome said:
Super job. Waiting for an Audio / Video demo. If everything is ok, I will certainly make the purchase.
At the moment if you use a wi-fi it gives problems (the voice comes with a flickering after several seconds), on 4G sometimes it is very fluid, other times there is a delay of up to 3-4 seconds
Click to expand...
Yes, a delay is there. Its because of the test server bandwidth limitation and the unprocessed raw PCM buffer. It will be smoother with compressed buffer.
 
M

mehdipass

Active Member
hi,
I will test your example on my server but it gives an error.
B4X: 
io.socket.engineio.client.EngineIOException: xhr poll error
why?
 
Ertan

Ertan

Active Member
Licensed User
Hi,

i testing your project, i run the application and the application crashes while connecting to the server.


B4X: 
Günlükçü şuna bağlandı: 11160b24ac4a3502
--------- beginning of crash
--------- beginning of main
*** Service (starter) Create ***
** Service (starter) Start **
Connecting...
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
Connection established
sending message to waiting queue (CallSubDelayed - update_user_list)
sending message to waiting queue (activity_permissionresult)
running waiting messages (2)
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
--------- beginning of system
*** Service (starter) Create ***
** Service (starter) Start **
Connecting...
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
io.socket.engineio.client.EngineIOException: xhr poll error
** Activity (main) Pause, UserClosed = false **
Tring to Reconnect
Reconnecting
io.socket.engineio.client.EngineIOException: xhr poll error
Reconnection Error
sending message to waiting queue (activity_permissionresult)
running waiting messages (1)
** Activity (main) Resume **
Tring to Reconnect
main_update_status (java line: 526)
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
    at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:7816)
    at android.view.ViewRootImpl.invalidateChildInParent(ViewRootImpl.java:1345)
    at android.view.ViewGroup.invalidateChild(ViewGroup.java:5446)
    at android.view.View.invalidateInternal(View.java:14755)
    at android.view.View.invalidate(View.java:14719)
    at android.view.View.invalidate(View.java:14703)
    at android.widget.TextView.checkForRelayout(TextView.java:8536)
    at android.widget.TextView.setText(TextView.java:5083)
    at android.widget.TextView.setText(TextView.java:4908)
    at android.widget.TextView.setText(TextView.java:4883)
    at anywheresoftware.b4a.objects.TextViewWrapper.setText(TextViewWrapper.java:39)
    at b4a.example.main._update_status(main.java:526)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.keywords.Common.CallSub4(Common.java:1082)
    at anywheresoftware.b4a.keywords.Common.CallSubNew2(Common.java:1037)
    at b4a.example.starter._socket_onreconnecting(starter.java:308)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.BA.raiseEvent(BA.java:193)
    at com.biswajit.socketio.SocketIO$8.call(SocketIO.java:151)
    at io.socket.emitter.Emitter.emit(Emitter.java:117)
    at io.socket.client.Socket.access$601(Socket.java:19)
    at io.socket.client.Socket$5.run(Socket.java:177)
    at io.socket.thread.EventThread.exec(EventThread.java:55)
    at io.socket.client.Socket.emit(Socket.java:173)
    at io.socket.client.Manager.emitAll(Manager.java:156)
    at io.socket.client.Manager.access$600(Manager.java:20)
    at io.socket.client.Manager$11$1.run(Manager.java:555)
    at io.socket.thread.EventThread$2.run(EventThread.java:80)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
    at java.lang.Thread.run(Thread.java:762)
** Activity (main) Pause, UserClosed = true **

Screenshot_3.png
 
Last edited:
Biswajit

Biswajit

Active Member
Licensed User
Longtime User
mehdipass said:
hi,
I will test your example on my server but it gives an error.
B4X: 
io.socket.engineio.client.EngineIOException: xhr poll error
why?
Click to expand...
This means your app wasn't able to connect to the server. Are you testing this with your own local/remote server? or the test server?
 
Biswajit

Biswajit

Active Member
Licensed User
Longtime User
Ertan said:
Hi,

i testing your project, i run the application and the application crashes while connecting to the server.


B4X: 
Günlükçü şuna bağlandı: 11160b24ac4a3502
--------- beginning of crash
--------- beginning of main
*** Service (starter) Create ***
** Service (starter) Start **
Connecting...
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
Connection established
sending message to waiting queue (CallSubDelayed - update_user_list)
sending message to waiting queue (activity_permissionresult)
running waiting messages (2)
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
--------- beginning of system
*** Service (starter) Create ***
** Service (starter) Start **
Connecting...
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
io.socket.engineio.client.EngineIOException: xhr poll error
** Activity (main) Pause, UserClosed = false **
Tring to Reconnect
Reconnecting
io.socket.engineio.client.EngineIOException: xhr poll error
Reconnection Error
sending message to waiting queue (activity_permissionresult)
running waiting messages (1)
** Activity (main) Resume **
Tring to Reconnect
main_update_status (java line: 526)
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
    at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:7816)
    at android.view.ViewRootImpl.invalidateChildInParent(ViewRootImpl.java:1345)
    at android.view.ViewGroup.invalidateChild(ViewGroup.java:5446)
    at android.view.View.invalidateInternal(View.java:14755)
    at android.view.View.invalidate(View.java:14719)
    at android.view.View.invalidate(View.java:14703)
    at android.widget.TextView.checkForRelayout(TextView.java:8536)
    at android.widget.TextView.setText(TextView.java:5083)
    at android.widget.TextView.setText(TextView.java:4908)
    at android.widget.TextView.setText(TextView.java:4883)
    at anywheresoftware.b4a.objects.TextViewWrapper.setText(TextViewWrapper.java:39)
    at b4a.example.main._update_status(main.java:526)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.keywords.Common.CallSub4(Common.java:1082)
    at anywheresoftware.b4a.keywords.Common.CallSubNew2(Common.java:1037)
    at b4a.example.starter._socket_onreconnecting(starter.java:308)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.BA.raiseEvent(BA.java:193)
    at com.biswajit.socketio.SocketIO$8.call(SocketIO.java:151)
    at io.socket.emitter.Emitter.emit(Emitter.java:117)
    at io.socket.client.Socket.access$601(Socket.java:19)
    at io.socket.client.Socket$5.run(Socket.java:177)
    at io.socket.thread.EventThread.exec(EventThread.java:55)
    at io.socket.client.Socket.emit(Socket.java:173)
    at io.socket.client.Manager.emitAll(Manager.java:156)
    at io.socket.client.Manager.access$600(Manager.java:20)
    at io.socket.client.Manager$11$1.run(Manager.java:555)
    at io.socket.thread.EventThread$2.run(EventThread.java:80)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
    at java.lang.Thread.run(Thread.java:762)
** Activity (main) Pause, UserClosed = true **

View attachment 112197
Click to expand...
This means the socket.io client process is running on different thread. Only the main thread of the application can control any view modification. I tested the code its working fine. Did you changed any code or running the code as it is?
 
Ertan

Ertan

Active Member
Licensed User
Biswajit said:
This means the socket.io client process is running on different thread. Only the main thread of the application can control any view modification. I tested the code its working fine. Did you changed any code or running the code as it is?
Click to expand...
I haven't touched any code. I just wrote the local ip address of my server in the mobile application.
  1. Install Node.js
  2. Extract the server code
  3. Open terminal/cmd and navigate to the folder where you have extracted the code
  4. Run npm install (it will take few minutes)
  5. Run node app.js
  6. That's it.
I performed these steps.
I updated the IP address in the application and uploaded it to my phone. I got the same error.
 
Biswajit

Biswajit

Active Member
Licensed User
Longtime User
Ertan said:
I haven't touched any code. I just wrote the local ip address of my server in the mobile application.
  1. Install Node.js
  2. Extract the server code
  3. Open terminal/cmd and navigate to the folder where you have extracted the code
  4. Run npm install (it will take few minutes)
  5. Run node app.js
  6. That's it.
I performed these steps.
I updated the IP address in the application and uploaded it to my phone. I got the same error.
Click to expand...
Ok. I will check the code again. What is your b4a version?
 
Addo

Addo

Well-Known Member
Licensed User
Longtime User
I have tested this project. The echo canceler that you have set within the source code is not effective on most Android devices tested on Android 7,8,11
The echo is a bottleneck for any duplex chat platform
 
Ertan

Ertan

Active Member
Licensed User
Biswajit said:
Ok. I will check the code again. What is your b4a version?
Click to expand...
I found the solution to the problem.

After saying "npm install" and installing it will give you a warning.

Screenshot_4.png


If you say "npm audit fix --force" as it said, it will make a few changes in your project. These changes will cause your project to not work properly. So I ignored this warning and started the project by typing "node app.js" directly. I was able to perform my test without any trouble.

But..

1) Two-sided audio departure is very delayed (3-4 seconds) sometimes it doesn't even go away. What is the reason of this? How can we avoid this situation?
2) On my Android 7.0 phone, the project is running in Debug mode, but in Release mode my project crashes and gives an error.

This is not a problem on my Android 10 phone.

Crash:

B4X: 
--------- beginning of system
*** Service (starter) Create ***
** Service (starter) Start **
Connecting...
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
io.socket.engineio.client.EngineIOException: xhr poll error
Tring to Reconnect
main_update_status (java line: 484)
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
    at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:7816)
    at android.view.ViewRootImpl.invalidateChildInParent(ViewRootImpl.java:1345)
    at android.view.ViewGroup.invalidateChild(ViewGroup.java:5446)
    at android.view.View.invalidateInternal(View.java:14755)
    at android.view.View.invalidate(View.java:14719)
    at android.view.View.invalidate(View.java:14703)
    at android.widget.TextView.checkForRelayout(TextView.java:8536)
    at android.widget.TextView.setText(TextView.java:5083)
    at android.widget.TextView.setText(TextView.java:4908)
    at android.widget.TextView.setText(TextView.java:4883)
    at anywheresoftware.b4a.objects.TextViewWrapper.setText(TextViewWrapper.java:39)
    at b4a.example.main._update_status(main.java:484)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.keywords.Common.CallSub4(Common.java:1082)
    at anywheresoftware.b4a.keywords.Common.CallSubNew2(Common.java:1037)
    at b4a.example.starter._socket_onreconnecting(starter.java:308)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.BA.raiseEvent(BA.java:193)
    at com.biswajit.socketio.SocketIO$8.call(SocketIO.java:151)
    at io.socket.emitter.Emitter.emit(Emitter.java:117)
    at io.socket.client.Socket.access$601(Socket.java:19)
    at io.socket.client.Socket$5.run(Socket.java:177)
    at io.socket.thread.EventThread.exec(EventThread.java:55)
    at io.socket.client.Socket.emit(Socket.java:173)
    at io.socket.client.Manager.emitAll(Manager.java:156)
    at io.socket.client.Manager.access$600(Manager.java:20)
    at io.socket.client.Manager$11$1.run(Manager.java:555)
    at io.socket.thread.EventThread$2.run(EventThread.java:80)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
    at java.lang.Thread.run(Thread.java:762)
** Activity (main) Pause, UserClosed = true **
 
Ertan

Ertan

Active Member
Licensed User
mehdipass said:
Does anyone have an idea to solve this problem?
Click to expand...
Ertan said:
I found the solution to the problem.

After saying "npm install" and installing it will give you a warning.

View attachment 112310

If you say "npm audit fix --force" as it said, it will make a few changes in your project. These changes will cause your project to not work properly. So I ignored this warning and started the project by typing "node app.js" directly. I was able to perform my test without any trouble.
Click to expand...
 
Biswajit

Biswajit

Active Member
Licensed User
Longtime User
PassionDEV said:
I have tested this project. The echo canceler that you have set within the source code is not effective on most Android devices tested on Android 7,8,11
The echo is a bottleneck for any duplex chat platform
Click to expand...
PassionDEV said:
You can lower your bitrate or even encode your audio for bandwidth saving before sending.
Click to expand...
All the possible fixes will be posted as a paid version. I am down due to covid19. Once I feel good I will continue that paid version.
 
J

José A.

Member
Licensed User
MarcoRome said:
Super job. Waiting for an Audio / Video demo. If everything is ok, I will certainly make the purchase.
At the moment if you use a wi-fi it gives problems (the voice comes with a flickering after several seconds), on 4G sometimes it is very fluid, other times there is a delay of up to 3-4 seconds
Click to expand...
Hi! Nice to meet you.
Do you know anything about the project Simple audio call over the Internet with your own TURN/relay server?
Was it just done? I'm interested, I've written to Thread Starter Biswajit but they don't reply.
Thank you so much for everything.
 
You must log in or register to reply here.

Similar Threads

LucaMs
  • Article
Android Code Snippet [B4X][Library] A solution to easily add your own code snippets?
Replies
11
Views
5K
LucaMs
LucaMs
Erel
  • Article
Share My Creation [B4X] B4X Pleroma - open source client for Mastodon / Pleroma social networks
Replies
13
Views
13K
Erel
Erel
K
  • Article
B4J Code Snippet [B4X]Raise Your Own Exceptions
Replies
1
Views
2K
Diceman
Diceman
DonManfred
  • Article
Android Tutorial [B4X] Manage your own Let´s Encrypt Certificates (ACME Client)
Replies
2
Views
4K
DonManfred
DonManfred
andymc
Games Who writes their own functions?
Replies
18
Views
3K
JordiCP
JordiCP
Top