B4J Video Conference / Video Streaming (RTSP H.264) / and webRTC

[Star-Dust]

I took another step towards video conferencing, using webRTC. So something different from the h.264 broadcast.

I used jitsi servers and here is a first result

this is the code i used

Sub Globals
    'These global variables will be redeclared each time the activity is created.
    Private rtcView1 As rtcView
End Sub

Sub Activity_Create(FirstTime As Boolean)
    Activity.LoadLayout("Layout")
End Sub

Sub Activity_Resume
    For Each permission As String In Array(rp.PERMISSION_CAMERA, rp.PERMISSION_RECORD_AUDIO)
        rp.CheckAndRequest(permission)
        Wait For Activity_PermissionResult (permission As String, result As Boolean)
        If result = False Then
            ToastMessageShow("No permission!", True)
            Activity.Finish
        End If
    Next
End Sub

A simple layout with only one view

 

[Star-Dust]

Why?

see here

MQTT vs Websockets

I hope some of you have already worked on both and have a general idea of the differences between them. Currently (for centuries) I'm trying to develop an online multiplayer turn-based game. I know MQTT not so much, but I can not see great advantages in using it instead of WebSockets. The...

www.b4x.com

[IoT] MQTT Protocol

What is MQTT? "MQTT is a Client Server publish/subscribe messaging transport protocol. It is light weight, open, simple, and designed so as to be easy to implement. These characteristics make it ideal for use in many situations, including constrained environments such as for communication in...

www.b4x.com

 

[Star-Dust]

I made my own library by building it around an existing library by @moster67 (this one)
This library allows you to access the webcam and see it on screen, stream and save the video. You can also capture live screen with video, or a part of the screen or a secondary screen, view it, stream it or save the video to a file
To create the necessary layouts for the view edit the existing layouts in runtime to fit them

SD_VideoStreaming

Author: Star-Dust
Version: 1.03

• SD_VideoStream
  • Events:
    • ChangeState (state As Int)
    • Error
    • Prepared
    • ReadyToPlay
    • ReadyToSend
    • SpeedRateChange (Success As Boolean)
  • Functions:
    • CanPause As Boolean
    • Class_Globals As String
    • GetAspectRatio As String
    • getDurate As Long
      millisecond
    • getPlayRate As Float
    • getPosition As Float
      current play position: 0.0-1.0
    • getTime As Long
      current play time
    • getVolume As Int
      Volume = 0-200
    • getZoom As Float
    • Initialize (CallBack As Object, EventName As String) As String
      Initializes the object. You can add parameters to this method if needed.
    • IsInitialized As Boolean
      Verifica se l'oggetto sia stato inizializzato.
    • IsPlayable As Boolean
    • IsPlaying As Boolean
    • Mute As String
    • Pause As String
    • Play (source As String) As String
      Play("rtsp://192.168.1.1:8554/test.mp4")
    • Player As JavaObject
      return Player Object
    • PlayExtra (Options As String(), Source As String) As String
    • Release As String
      release resource
    • SaveCam (filename As String, DegreeRotate As Int) As String
    • SaveDesktop (Savefilename As String) As String
      file .flv
    • SaveDesktopPrimary (Savefilename As String) As String
      file .flv
    • SaveDesktopRect (Savefilename As String, Left As Int, Top As Int, Width As Int, Height As Int) As String
    • setPlaySpeedRate (speedRate As Float) As String
      Raise event SpeedRateChange
    • SetPosition (position As Float) As String
    • setRepeat (Repeat As Boolean) As String
    • setTime (time As Long) As String
    • setVolume (Volume As Int) As String
    • setZoom (factor As Float) As String
      0 for adapter;
    • Skip (delta As Long) As String
      Slip position time long
    • SkipPosition (delta As Float) As String
      Slip position float 0-1
    • Stop As String
    • StreamCam (LocalIP As String, Port As Int, DegreeRotate As Int, external_FileNameMp4 As String) As String
      StreamCam("192.168.1.100","test.mpa",8554,0) or StreamCam("192.168.1.100","test.mpa",8554,180)
      for Play: Play("rtsp://192.168.1.1:8554/test.mp4")
    • StreamDesktop (LocalIP As String, Port As Int, external_FileNameMp4 As String) As String
    • StreamDesktopPrimary (LocalIP As String, Port As Int, external_FileNameMp4 As String) As String
    • StreamDesktopRect (LocalIP As String, Port As Int, external_FileNameMp4 As String, Left As Int, Top As Int, Width As Int, Height As Int) As String
    • StreamFile (LocalFileVideo As String, LocalIP As String, Port As Int, external_FileNameMp4 As String) As String
    • unMute (VolumeLevel As Int) As String
      Volume: 0-200
    • VideoHeight As Double
    • VideoWidth As Double
  • Properties:
    • Durate As Long [read only]
      millisecond
    • PlayRate As Float [read only]
    • PlaySpeedRate
      Raise event SpeedRateChange
    • Repeat
    • Time As Long
      current play time
    • Volume As Int
      Volume = 0-200
    • Zoom As Float
      0 for adapter;
• StreamSetup
  • Functions:
    • HookPlayer (pNode As Node, VideoPlay As JavaObject) As String
    • LoadLayoput (base As Pane, pNodeName As String, PakageName As String) As Node
    • Process_Globals As String
    • VlcInstalled As Boolean

---

In this example the webcam is captured and you will see a preview on a panel and it will be sent in streaming (H.264)

Sub Process_Globals
    Private fx As JFX
    Private MainForm As Form
    Private xui As XUI
 
    ' Streaming Class (Send and Play)
    Private VideoStream As SD_VideoStream
    Private VideoRemotePlay As SD_VideoStream
    Private VideoLocalPlay As SD_VideoStream
    ' Pane Base to Play - Add on Design
    Private BaseRemoteVideo As Pane
    Private BaseLocalVideo As Pane
    ' Node Base (into pane) - don't add by design
    Private NodeRemotePlay As Node
    Private NodeLocalPlay As Node
End Sub


Sub AppStart (Form1 As Form, Args() As String)
    MainForm = Form1
 
    If StreamSetup.vlcInstalled = False Then
        Log("VLC must be installed on the computer to run this program.")
        ExitApplication
    End If
     
    VideoStream.Initialize(Me,"VideoStream")
    VideoLocalPlay.Initialize(Me,"VideoLocalPlay")
    VideoRemotePlay.Initialize(Me,"VideoRemotePlay")
 
    ' load Layout with pane for VideoPlay
    MainForm.RootPane.LoadLayout("Layout1")
    ' create layout for node by code
    NodeLocalPlay=StreamSetup.LoadLayoput(BaseLocalVideo,"NodeLocalPlay","it.tecnomedia.stream")
    NodeRemotePlay=StreamSetup.LoadLayoput(BaseRemoteVideo,"NodeRemotePlay","it.tecnomedia.stream")
 
    ' Hook VideoPlay to node
    StreamSetup.HookPlayer(NodeLocalPlay,VideoLocalPlay.Player)
    StreamSetup.HookPlayer(NodeRemotePlay,VideoRemotePlay.Player)

    MainForm.Show
End Sub

Send

private Sub ButtonSender_Click
    Log("Start sender")
    LabelInfo.Text="STARTING"

    'VideoStream.StreamCam(LocalIP,"prova.mp4",8554,180)  ' for Internal Cam
    VideoStream.StreamDesktop(LocalIP,"prova.mp4",8554) ' For Desktop
End Sub

Private Sub VideoStream_ReadyToSend
    Log("Start to send")
    If VideoLocalPlay.IsPlaying Then VideoLocalPlay.Stop
    VideoLocalPlay.Play($"rtsp://${LocalIP}:8554/prova.mp4"$)
    VideoLocalPlay.SetVolume(0)
End Sub

Receive

Private Sub ButtonRiceve_Click
    Log("Start connect")
    LabelInfo.Text="Connect"
  
    Sleep(15000)
    If VideoRemotePlay.IsPlaying Then VideoRemotePlay.Stop
    VideoRemotePlay.Play($"rtsp://${TextRemote.Text.Trim}:8554/prova.mp4"$)
    VideoRemotePlay.SetVolume(100)
End Sub

 

[Star-Dust]

I developed a second library starting from @moster67's java code and modifying it for my needs. One of the few times I've managed to compile with SLC.
This library allows you to access the webcam and see it on screen, stream and save the video. You can also capture live screen with video, or a part of the screen or a secondary screen, view it, stream it or save the video to a file.
To creates the necessary layouts to view video and colors in runtime

SD_Streaming

Author: Star-Dust
Version: 1.0

• SD_Streaming
  • Events:
    • ChangeState (state As Int)
    • Error
    • Finished
    • Prepared
  • Fields:
    • player As player.VideoPlayer
  • Functions:
    • CanPause As Boolean
      Can the current media be paused?
      returns true if the current media can be paused, otherwise false
    • GetAspectRatio As String
      Get the video aspect ratio.
      Returns aspect ratio ONLY if you have set the AspectRatio yourself.
    • GetLength As Long
      Get the length of the current media item.
      Returns: length, in milliseconds
    • GetPlaySpeedRate As Float
      Get the current video play speed rate.
      speedRate, where 1.0 is normal speed, 0.5 is half speed, 2.0 is double speed and so on
    • GetPosition As Float
      Get the current play-back position.
      Returns current position, expressed as a percentage (e.g. 0.15 is returned for 15% complete)
    • GetScale As Float
      Get the current video scale (zoom factor).
    • GetTime As Long
      Get the current play-back time.
      Returns current time, expressed as a number of milliseconds
    • GetVolume As Int
      Get the current volume.
      Returns volume, a percentage of full volume in the range 0 to 200
    • Initialize (EventName As String)
      Initializes B4JVlcj. You must set an event-name if you want to monitor events.
      Important: Always check if VLC is installed before initializing using
      the IsVLCInstalled-method.
      Important: When exiting your app, always make sure to use the release-method!
    • IsInitialized As Boolean
    • IsPlayable As Boolean
      Is the current media playable?
      returns true if the current media is playable, otherwise false
    • IsPlaying As Boolean
      Is the media player playing?
      returns true if the media player is playing, otherwise false
    • IsSeekable As Boolean
      Is the current media seekable?
      returns true if the current media is seekable, otherwise false
    • IsVLCInstalled As Boolean
      This method should be the first code you run before initializing B4JVlcj.
      Returns if VLC has been installed on the computer or not. If VLC has been
      installed in a non-standard directory, VLC might not be found.
    • Mute
      Mutes volume
    • Pause
      Pauses media
    • Play (source As String)
      Plays media (can be a local file or a streaming source)
    • PlayExtra (Options As String(), source As String)
    • release
      Use this on the b4j closing event. Final step to do is to release vlc.
    • SaveCam (filename As String, DegreeRotate As Int)
    • SaveDesktop (filename As String)
    • SetAspectRatio (AspectRatio As String)
      Set the video aspect ratio
      Param: aspectRatio - aspect ratio, e.g. "16:9", "4:3", "185:100" for 1:85.1 and so on
    • SetPlaySpeedRate (speedRate As Float) As Int
      Set the video play speed rate.
      Some media protocols are not able to change the speed rate.
      speed rate - rate, where 1.0 is normal speed, 0.5 is half speed, 2.0 is double speed and so on
      Returns: -1 on error, 0 on success
    • SetPosition (position As Float)
      Jump to a specific position. If the requested position
      is less than zero, it is normalised to zero.
      Param: position - position value, a percentage (e.g. 0.15 is 15%)
    • SetScale (factor As Float)
      Set the video scaling factor
      Param: factor - scaling factor, or zero to scale the video the size of the container
    • SetTime (time As Long)
      Jump to a specific moment.
      If the requested time is less than zero, it is normalised to zero
      Param: time - time since the beginning, in milliseconds
    • SetVolume (volumeLevel As Int)
      Set the volume.
      The volume is actually a percentage of full volume, setting a volume
      over 100 may cause audible distortion.
      Param: volumeLevel - a percentage of full volume in the range 0 to 200
    • Skip (delta As Long)
      Skip forward or backward by a period of time.
      To skip backwards specify a negative delta.
      Param: delta - time period, in milliseconds
    • SkipPosition (delta As Float)
      Skip forward or backward by a change in position.
      To skip backwards specify a negative delta.
      Param: delta - amount to skip
    • Stop
      Stops playing media
    • StreamCam (localip As String, filename As String, port As Int, DegreeRotate As Int)
    • StreamDesktop (localip As String, filename As String, port As Int)
    • Unmute (VolumeLevel As Int)
      Unmutes volume
  • Properties:
    • Version As Double [read only]
    • VideoHeight As Double [read only]
      Returns Video Height
    • VideoWidth As Double [read only]
      Returns Video Width

---

In this example the webcam is captured and you will see a preview on a panel and it will be sent in streaming (H.264)

Sub Process_Globals
    Private fx As JFX
    Private MainForm As Form
    Private xui As XUI
 
    Private ExternaFileName As String
 
    Private VideoStream As SD_Streaming
    Private VideoLocalPlay As SD_Streaming
    Private VideoRemotePlay As SD_Streaming
 
    Private NodeLocal As Node
    Private NodeRemote As Node
    Private BasePane As Pane
    Private BaseLocal As Pane
End Sub

Sub AppStart (Form1 As Form, Args() As String)
    MainForm = Form1
 
    '** IMPORTANT - controllo se VLC è installato
    If VideoStream.IsVLCInstalled = False Then
        Log("VLC must be installed on the computer to run this program.")
        ExitApplication
    End If
    VideoStream.Initialize("VideoStream")
    VideoLocalPlay.Initialize("VideoLocalPlay")
    VideoRemotePlay.Initialize("VideoRemotePlay")
 
    MainForm.RootPane.LoadLayout("Layout1")
    BasePane.LoadLayout("scenevlc")
    BaseLocal.LoadLayout("scenelocal")
 
    ' VLC node
    NodeLocal.As(JavaObject).RunMethodJO("setStyle", Array("-fx-background-color: #000000;"))
    NodeLocal.As(JavaObject).RunMethodJo("getChildren", Null).RunMethod("add", Array(VideoLocalPlay.player))
    NodeRemote.As(JavaObject).RunMethodJO("setStyle", Array("-fx-background-color: #000000;"))
    NodeRemote.As(JavaObject).RunMethodJo("getChildren", Null).RunMethod("add", Array(VideoRemotePlay.player))
 
    MainForm.Show
End Sub

Start capture

private Sub ButtonCam_Click
    Log("Start sender")
    LabelInfo.Text="STARTING"
 
    VideoStream.StreamCam(LocalIP,"prova.mp4",8554,180)
End Sub

Private Sub ButtonDesktop_Click
    VideoStream.StreamDesktop(LocalIP,"prova.mp4",8554)
End Sub


Private Sub ButtonVideo_Click
    ExternaFileName="file.mp4"
    If File.Exists(File.DirApp,ExternaFileName) Then
        Dim Options() As String = Array As String(":screen-fps=24", _
        $":sout=#transcode{vcodec=h264,vb=0,scale=0,acodec=mp4a,ab=128,channels=2,samplerate=44100}:rtp{sdp=rtsp://${LocalIP}:8554/${ExternaFileName}"$, _
        ":sout-keep")
        VideoLocalPlay.PlayExtra(Options,ExternaFileName)
    Else
        fx.Msgbox(MainForm, "source file don't exist","ERRORE")
    End If
End Sub

Receive stream

Private Sub ButtonRiceve_Click
    Log("Start connect")
    LabelInfo.Text="Connect"
  
    Sleep(1000)
    If VideoLocalPlay.IsPlaying Then VideoLocalPlay.Stop
    VideoLocalPlay.Play($"rtsp://${TextRemote.Text.Trim}:8554/prova.mp4"$)
  
    VideoLocalPlay.SetVolume(100)
End Sub

Private Sub ButtonSaveDesk_Click
    Dim desktop As String = File.Combine(GetSystemProperty("user.home", ""), "Desktop")
 
    If File.IsDirectory(desktop, "") Then VideoStream.SaveDesktop(File.Combine(desktop,"video.flv"))
End Sub