Android Question [Solved] buffer fragmented packet

Mr Blue Sky

Active Member
Licensed User
Longtime User
Hello,
I encounter a small problem with the reading of the buffer.

B4X:
Private Sub AStream_NewData (Buffer() As Byte)
     Dim msg As String
     msg = BytesToString(Buffer, 0, Buffer.Length, "UTF8")
     Log(msg)
End Sub

I send a string of type from a PC with realterm : 2BC 00 00 61 00 00 00 00 00 or 2BC 00 00 00 00 00 00 00 41

and I get this kind of thing on the logger?

Sans titre.jpg
 

Star-Dust

Expert
Licensed User
Longtime User
you ask why do you get the sequence broken into several takes?
 
Upvote 0

Star-Dust

Expert
Licensed User
Longtime User
it is normal that it sends a fragmented packet. even if it seems strange to me that you break it up into such small parts.

on the forum you can find some examples of Erel to send strings only in sending and thus receive them in a single package
 
Upvote 0

Mr Blue Sky

Active Member
Licensed User
Longtime User
it is normal that it sends a fragmented packet. even if it seems strange to me that you break it up into such small parts.

on the forum you can find some examples of Erel to send strings only in sending and thus receive them in a single package
I use already the exemple to Erel in regular mode. no-prefix. thanks anyway for your reply
 
Upvote 0

Star-Dust

Expert
Licensed User
Longtime User
I meant something else. Erel wrote a small source to avoid this problem. Unfortunately I can't find it now.

The concept was that each end of string put a special character (EOF or CRLF etc ...)

Upon receipt AsyncStream reconstructed the packet up to the end of the string, upon completion it raised the event and passed it to the application that received the complete and not broken text.

It was roughly like that
 
Upvote 0

emexes

Expert
Licensed User
I meant something else. Erel wrote a small source to avoid this problem. Unfortunately I can't find it now.
The concept was that each end of string put a special character (EOF or CRLF etc ...)

Presumably this: class asyncstreamstext useful when-working with streams of text

Which would work great with the example incoming data, if that data has end-of-line markers CR (ASCII 13 / 0x0D) or LF (ASCII 10 / 0x0A). It is not clear from the example log output whether that data has any end-of-line markers.

If it does not have those markers, then the simplest solution is probably to accumulate the incoming data into a string buffer, capped at length say 60 characters, and whenever the string buffer is 27 characters or longer, search the buffer for a packet. If the incoming packets are always 9 hex values long, and the leading 2BC never changes to another hex value, then you could search for:

"2BC hh hh hh hh hh hh hh hh" (where h is a hex digit 0-9 A-F)

or if the 2BC can also be other values but is always 3 hex digits, cf the other 8 bytes of the packet being always 2 hex digits, then you could search for:

"hhh hh hh hh hh hh hh hh hh"

If you find a packet, then extract it out of the string buffer and pass it to the packet handler, and then delete everything in the string buffer up to the last character of the packet just extracted, and repeat the search if the string buffer is still 27 characters or longer (ie, Do While InBuffer.Length >= 27)

If the packets can be different lengths, ie not always 8 x 2 hex digit values, and there are no end-of-line markers, then it becomes slightly trickier to determine when you've received a whole packet. Eg you might need to wait for the 2BC of the next packet, which means you'd always be one packet behind real time.
 
Last edited:
Upvote 0

emexes

Expert
Licensed User
I encounter a small problem with the reading of the buffer.

This is how I would unencounter the small problem, by collecting the 2BC header and the following 24 bytes into full 27 byte packets:

B4X:
Sub Process_Globals
 
    Dim PacketBytes(27) As Byte   ' 3-byte header "2BC" plus 8 x 3-byte space and 2-digit hex values
    Dim NumPacketBytes As Int = 0
 
End Sub

Private Sub AStream_NewData (Buffer() As Byte)
 
    For I = 0 To Buffer.Length - 1    'process each received byte, one by one
        Dim Ch As Byte = Buffer(I)

        PacketBytes(NumPacketBytes) = Ch    'NumPacketBytes should always be less than 27 at this point,
        NumPacketBytes = NumPacketBytes + 1    'since when it reaches 27 here, it is then reset back to 0 after the packet is handled below
   
        Select Case NumPacketBytes
            Case 1:
                If Ch <> 50 Then    '2'
                    NumPacketBytes = 0
                End If
               
            Case 2:
                If Ch <> 66 Then    'B'
                    NumPacketBytes = 0
                End If

            Case 3:
                If Ch <> 67 Then    'C'
                    NumPacketBytes = 0
                End If

            Case PacketBytes.Length:
                HandlePacket(PacketBytes())
                NumPacketBytes = 0

        End Select
    Next

End Sub

Sub HandlePacket(Pkt() As Byte)
 
    Dim msg As String
    msg = BytesToString(Pkt, 0, Pkt.Length, "UTF8")
    Log(msg)
 
End Sub
 
Last edited:
Upvote 0

Mr Blue Sky

Active Member
Licensed User
Longtime User
This is how I would unencounter the small problem, by collecting the 2BC header and the following 24 bytes into full 27 byte packets:

B4X:
Sub Process_Globals
 
    Dim PacketBytes(27) As Byte   ' 3-byte header "2BC" plus 8 x 3-byte space and 2-digit hex values
    Dim NumPacketBytes As Int = 0
 
End Sub

Private Sub AStream_NewData (Buffer() As Byte)
 
    For I = 0 To Buffer.Length - 1    'process each received byte, one by one
        Dim Ch As Byte = Buffer(I)

        PacketBytes(NumPacketBytes) = Ch    'NumPacketBytes should always be less than 27 at this point,
        NumPacketBytes = NumPacketBytes + 1    'since when it reaches 27 here, it is then reset back to 0 after the packet is handled below
  
        Select Case NumPacketBytes
            Case 1:
                If Ch <> 50 Then    '2'
                    NumPacketBytes = 0
                End If
              
            Case 2:
                If Ch <> 66 Then    'B'
                    NumPacketBytes = 0
                End If

            Case 3:
                If Ch <> 67 Then    'C'
                    NumPacketBytes = 0
                End If

            Case PacketBytes.Length:
                HandlePacket(PacketBytes())
                NumPacketBytes = 0

        End Select
    Next

End Sub

Sub HandlePacket(Pkt() As Byte)
 
    Dim msg As String
    msg = BytesToString(Pkt, 0, Pkt.Length, "UTF8")
    Log(msg)
 
End Sub
sorry it does not work, on my side I managed to have it. Now I have the full string but I don't understand how to dump if I take sb.Length or buffer.Length I end up with the same problem as before.
I saw that there is the class (AsyncStreamsText) but I can't find it how to embed it in the EREL demo https://www.b4x.com/android/forum/threads/b4xpages-bluetooth-chat- example.119014/

B4X:
Private Sub AStream_NewData (Buffer() As Byte)
    sb.Append(BytesToString(Buffer, 0, Buffer.Length, "utf8"))
    Dim s As String = sb.ToString
    Log(s)
    sb.Remove(0, 0)
    '//sb.Remove(0, sb.Length)
    '//sb.Remove(0, Buffer.Length)
End Sub

Sans titre.jpg
 
Upvote 0

emexes

Expert
Licensed User
sorry it does not work

My fault - I didn't have any actual serial stream to try it on, so I just relied on the IDE picking up syntax errors. How embarrassment... turns out one slipped through... change line 33 from:

HandlePacket(PacketBytes())

to just:

HandlePacket(PacketBytes)

Once that was done, and I wrote some test code to simulate an incoming serial stream:

B4X:
AStream_NewData(StringToByteArray("2BC 01 02 03"))
AStream_NewData(StringToByteArray(" 04 05 06 AA 08"))
AStream_NewData(StringToByteArray("2BC 01 02"))
AStream_NewData(StringToByteArray(" 03 04 05 06 BB 08noise2B"))
AStream_NewData(StringToByteArray("C 01 02 03"))
AStream_NewData(StringToByteArray(" 04 05 06 CC 08"))
AStream_NewData(StringToByteArray("noise2BC 01 02"))
AStream_NewData(StringToByteArray(" 03 04 05 06 DD 08"))

Sub StringToByteArray(S As String) As Byte()
    Dim BA(S.Length) As Byte
    For I = 0 To S.Length - 1
        BA(I) = Asc(S.CharAt(I))
    Next
    Return BA 
End Sub

then it worked a treat:

Log file output:
Waiting for debugger to connect...
Program started.
2BC 01 02 03 04 05 06 AA 08
2BC 01 02 03 04 05 06 BB 08
2BC 01 02 03 04 05 06 CC 08
2BC 01 02 03 04 05 06 DD 08
 
Last edited:
Upvote 0

emexes

Expert
Licensed User
then it worked a treat:

If it still doesn't work a treat at your end, then add the following debug log line:

B4X:
NumPacketBytes = NumPacketBytes + 1
        
Log(I & TAB & NumPacketBytes & TAB & Ch & TAB & Chr(Ch))
        
Select Case NumPacketBytes

and post the resultant log output after a few packets have been processed, plus a copy of your (presumably-modified) AStream_NewData sub.

Although it's just after midnight here, so there might not be an instant response from this end. ?
 
Upvote 0

Mr Blue Sky

Active Member
Licensed User
Longtime User
If it still doesn't work a treat at your end, then add the following debug log line:

B4X:
NumPacketBytes = NumPacketBytes + 1
       
Log(I & TAB & NumPacketBytes & TAB & Ch & TAB & Chr(Ch))
       
Select Case NumPacketBytes

and post the resultant log output after a few packets have been processed, plus a copy of your (presumably-modified) AStream_NewData sub.

Although it's just after midnight here, so there might not be an instant response from this end. ?
Thank you very much emexes, thank you all, really perfect. Still a lot to learn for me.

I send here the whole B4XMainpage class based on the demonstration of Bluetooth Chat of EREL modified to accept this kind of situation.

B4X:
Sub Class_Globals
    Private Root As B4XView 'ignore
    Private xui As XUI 'ignore
    Private btnSearchForDevices As B4XView
    Private btnAllowConnection As B4XView
    Private rp As RuntimePermissions
    Private AStream As AsyncStreams
    Private serial As Serial
    Private admin As BluetoothAdmin
    Type NameAndMac (Name As String, Mac As String)
    Public BluetoothState, ConnectionState As Boolean
    Private ChatPage1 As ChatPage
    Private AnotherProgressBar1 As B4XView
    Private CLV As CustomListView
    Private phone As Phone
    Private ImgLogo As B4XView
    Private ImgList As B4XView
    
    Dim PacketBytes(27) As Byte   ' 3-byte header "2BC" plus 8 x 3-byte space and 2-digit hex values
    Dim NumPacketBytes As Int = 0
End Sub

'You can add more parameters here.
Public Sub Initialize
    admin.Initialize("admin")
    serial.Initialize("serial")
    If admin.IsEnabled = False Then
        If admin.Enable = False Then
            ToastMessageShow("Error enabling Bluetooth adapter.", True)
        Else
            ToastMessageShow("Enabling Bluetooth adapter...", False)
        End If
    Else
        BluetoothState = True
    End If

    sb.Initialize
    
    ChatPage1.Initialize
End Sub

'This event will be called once, before the page becomes visible.
Private Sub B4XPage_Created (Root1 As B4XView)
    Root = Root1
    'load the layout to Root
    Root.LoadLayout("main")
    AnotherProgressBar1.Tag = 1
    B4XPages.AddPageAndCreate("Chat Page", ChatPage1)
End Sub

Private Sub Admin_StateChanged (NewState As Int, OldState As Int)
    Log("state changed: " & NewState)
    BluetoothState = NewState = admin.STATE_ON
    StateChanged
End Sub

Sub btnSearchForDevices_Click
    rp.CheckAndRequest(rp.PERMISSION_ACCESS_FINE_LOCATION)
    Wait For B4XPage_PermissionResult (Permission As String, Result As Boolean)
    If Result = False And rp.Check(rp.PERMISSION_ACCESS_COARSE_LOCATION) = False Then
        ToastMessageShow("No permission...", False)
        Return
    End If
    If phone.SdkVersion >= 31 Then
        For Each Permission As String In Array("android.permission.BLUETOOTH_SCAN", "android.permission.BLUETOOTH_CONNECT")
            rp.CheckAndRequest(Permission)
            Wait For B4XPage_PermissionResult (Permission As String, Result As Boolean)
            If Result = False Then
                ToastMessageShow("No permission...", False)
                Return
            End If
        Next
    End If
    Dim success As Boolean = admin.StartDiscovery
    If success = False Then
        ToastMessageShow("Error starting discovery process.", True)
    Else
        CLV.Clear
        Dim Index As Int = ShowProgress
        Wait For Admin_DiscoveryFinished
        HideProgress(Index)
        If CLV.Size = 0 Then
            ToastMessageShow("No device found.", True)
        End If
    End If
End Sub

Sub btnAllowConnection_Click
    If phone.SdkVersion >= 31 Then
        rp.CheckAndRequest("android.permission.BLUETOOTH_CONNECT")
        Wait For B4XPage_PermissionResult (Permission As String, Result As Boolean)
        If Result = False Then
            ToastMessageShow("No permission...", False)
            Return
        End If
    End If
    'this intent makes the device discoverable for 300 seconds.
    Dim i As Intent
    i.Initialize("android.bluetooth.adapter.action.REQUEST_DISCOVERABLE", "")
    i.PutExtra("android.bluetooth.adapter.extra.DISCOVERABLE_DURATION", 300)
    StartActivity(i)
    serial.Listen
End Sub

Public Sub SendMessage (msg As String)
    AStream.Write(msg.GetBytes("utf8"))
End Sub

Private Sub AStream_NewData (Buffer() As Byte)
    
    For I = 0 To Buffer.Length - 1    'process each received byte, one by one
        Dim Ch As Byte = Buffer(I)

        PacketBytes(NumPacketBytes) = Ch    'NumPacketBytes should always be less than 27 at this point,
        NumPacketBytes = NumPacketBytes + 1    'since when it reaches 27 here, it is then reset back to 0 after the packet is handled below
   
        Select Case NumPacketBytes
            Case 1:
                If Ch <> 50 Then    '2'
                    NumPacketBytes = 0
                End If
               
            Case 2:
                If Ch <> 66 Then    'B'
                    NumPacketBytes = 0
                End If

            Case 3:
                If Ch <> 67 Then    'C'
                    NumPacketBytes = 0
                End If

            Case PacketBytes.Length:
                HandlePacket(PacketBytes)
                NumPacketBytes = 0
        End Select
    Next
    
End Sub

Sub HandlePacket(Pkt() As Byte)
    Dim msg As String
    msg = BytesToString(Pkt, 0, Pkt.Length, "UTF8")
    ChatPage1.NewMessage(msg)
    'Log(msg)
End Sub

Private Sub AStream_Error
    ToastMessageShow("Connection is broken.", True)
    ConnectionState = False
    StateChanged
End Sub

Private Sub AStream_Terminated
    AStream_Error
End Sub

Public Sub Disconnect
    If AStream.IsInitialized Then AStream.Close
    serial.Disconnect
End Sub

Private Sub Admin_DeviceFound (Name As String, MacAddress As String)
    Log(Name & ":" & MacAddress)
    Dim nm As NameAndMac = CreateNameAndMac(Name, MacAddress)
    CLV.AddTextItem($"${nm.Name}: ${nm.Mac}"$, nm)
End Sub

Private Sub StateChanged
    btnSearchForDevices.Enabled = BluetoothState
    btnAllowConnection.Enabled = BluetoothState
    ChatPage1.btnSend.Enabled = ConnectionState
End Sub

Sub CLV_ItemClick (Index As Int, Value As Object)
    Dim nm As NameAndMac = Value
    ToastMessageShow($"Trying to connect to: ${nm.Name}"$, True)
    serial.Connect(nm.Mac)
    Dim Index As Int = ShowProgress
    Wait For Serial_Connected (Success As Boolean)
    HideProgress(Index)
    If Success = False Then
        Log(LastException.Message)
        ToastMessageShow("Error connecting: " & LastException.Message, True)
    Else
        AfterSuccessfulConnection
    End If
    StateChanged
End Sub

'will be called when a client connects to this device
Sub Serial_Connected (Success As Boolean)
    If Success Then
        AfterSuccessfulConnection
        StateChanged
    End If
End Sub

Sub AfterSuccessfulConnection 
    If AStream.IsInitialized Then AStream.Close
    'prefix mode! Change to non-prefix mode if communicating with non-B4X device.
    'AStream.InitializePrefix(serial.InputStream, False, serial.OutputStream, "astream")
    AStream.Initialize(serial.InputStream, serial.OutputStream, "astream")
    ConnectionState = True
    B4XPages.ShowPage("Chat Page")
End Sub

Sub ShowProgress As Int
    AnotherProgressBar1.Tag = AnotherProgressBar1.Tag + 1
    AnotherProgressBar1.Visible = True
    Return AnotherProgressBar1.Tag
End Sub

Sub HideProgress (Index As Int)
    If Index = AnotherProgressBar1.Tag Then
        AnotherProgressBar1.Visible = False
    End If
End Sub

Public Sub CreateNameAndMac (Name As String, Mac As String) As NameAndMac
    Dim t1 As NameAndMac
    t1.Initialize
    t1.Name = Name
    t1.Mac = Mac
    Return t1
End Sub

Private Sub B4XPage_Appear
    CLV.Clear
End Sub

Sans titre.jpg
 
Upvote 0

Mr Blue Sky

Active Member
Licensed User
Longtime User
Looks good. ?

But just in case there's a problem in that log output that I'm not seeing: is it receiving packets ok now?
Yes everything is perfect it takes into account the packets received, and that alone of course. problem solved thanks again ???
 
Upvote 0
Top