Android Question BLE Transmission of long packet has gaps, how to remove them?

[ema01]

A bit of context: Sending data from phone to a HM11 device (write characteristic -> characteristic data is sent through uart).
The HM11 is a BLE 4.0 device, so it has a limit of 20 byte per packet. This means that long packets must be divided in 20 bytes chunks, then sent in order.
In order to see if it would be feasible, i used one the "BLE Scanner" app to connect and send a long string of text. BLE Scanner has no problem in taking a long string and sending it so that the data is transmitter continously. In fact, by looking with an oscilloscope i see the 20 byte packets being sent with some 600us gaps between them. So at least in theory it's possible.

However, i can't find a way to get even close with B4A. I tried two approaches:
-calling Manager.WriteData in a loop
-calling Manager.WriteData, then on the Manager_WriteComplete event check if more data needs to be written then call Manager.WriteData again

unfortunately in both cases i get huge gaps, 20 to 100ms wide.
Is there a way i could get around the problem?
If i had to guess, it would be to implement synchronous operation: a block of native code in which i write and wait for completition, because i think the B4A event loop is getting in the middle
Suggestions?

 

[emexes]

A bit of context: Sending data from phone to a HM11 device (write characteristic -> characteristic data is sent through uart).
The HM11 is a BLE 4.0 device, so it has a limit of 20 byte per packet. This means that long packets must be divided in 20 bytes chunks, then sent in order.
Suggestions?

Simplest solution is to collect up the received subpackets yourself, into an array-of-bytes buffer, until you have a full packet (and possibly a fraction more), whereupon you cut out the full packet from the buffer and deliver it to program's packet handler "callback" routine.

The maximum effective transer rate is rather low, maybe like 1-2 kilobytes per second maximum, something to do with only transferring one chunk of 20 bytes per BLE connection interval. I vaguely remember it is discussed in one of the datasheets, which I now can't find.

Ugh I just noticed that I'm thinking of a HM10 whereas you mentioned a HM11, and I can't remember what the difference between the two is. Nonetheless:

this blog post about the HM-10 BLE UART module is worth a read:

HM-10 Bluetooth 4 BLE Modules - Martyn Currey

 

[ema01]

Hi,
it's the other way around.
I have no problem in receiving, i have problems in transmitting. I thought that was clear

(HM10 and HM11 are the same device, different pinout)

 

[ema01]

Maybe this will be more clear.

code i've written:

Public Sub SendTxData(data() As Byte, dSize As Int)
    mDataToSend = data
    mDataSent = 0
    
    If dSize <= 20 Then
        'Send In one tranche
        Dim dataToSend(dSize) As Byte
    Else
        Dim dataToSend(20) As Byte
    End If
    
    Dim idx As Int
    For idx=0 To dataToSend.Length-1
        dataToSend(idx) = mDataToSend(idx)
    Next
    mDataSent = idx
    
    Try
        mManager.WriteData(mServiceId, mCharId, dataToSend)
    Catch
        Log("Attempted Write while disconnected")
        Log(LastException)
    End Try
End Sub
#If B4A
Private Sub mManager_WriteComplete (Characteristic As String, Status As Int)
#Else If B4I
Private Sub mManager_WriteComplete (Characteristic As String, Success As Boolean)
#End If
    If mDataSent < mDataToSend.Length Then
        'More Data To Send
        Dim remaining As Int = mDataToSend.Length - mDataSent
        
        If remaining <= 20 Then
            'Send In one tranche
            Dim dataToSend(remaining) As Byte
        Else
            Dim dataToSend(20) As Byte
        End If
    
        Dim idx As Int
        For idx=0 To dataToSend.Length-1
            dataToSend(idx) = mDataToSend(mDataSent + idx)
        Next
        mDataSent = mDataSent + idx
    
        Try
            mManager.WriteData(mServiceId, mCharId, dataToSend)
        Catch
            Log("Attempted Write while disconnected")
            Log(LastException)
        End Try
    Else
        'Finished Writing
        mTmrRxTimeout.Enabled = True
    End If
End Sub

mDataToSend is a Byte Array

At the same timebase:

Reduce room so everything can be displayed:

Notice that i saw no difference between debug and release builds, maybe that in release the delays are less consistent

 

[emexes]

it's the other way around.
I have no problem in receiving, i have problems in transmitting. I thought that was clear

How embarrassment. Your thought is correct. I saw "BLE" and "sending data" and "20 bytes" and that somehow misdirected my reading.

Those are some ripper screenshots. I am about to look at them more closely but, tbh, it looks likes you've got a good grip on the situation and I doubt I'll be able to add much to that.

 

[emexes]

I tried two approaches:
-calling Manager.WriteData in a loop
-calling Manager.WriteData, then on the Manager_WriteComplete event check if more data needs to be written then call Manager.WriteData again

Suggestions?

Try a small delay (1 ms) between WriteData calls?

I have a vague recollection that small delays are all much the same anyway, eg Sleep(1) or Sleep(2) or Sleep(3) all end up being around 7 ms.

Maybe WriteComplete only happens when all buffered (?) packets have been sent, and it is then too late to add another packet to that send slot.

Yes, I am grasping at straws. On the bright side, that BLE Scanner app gives hope that close-together transmissions are possible.

 

[techknight]

I have had nothing but issues with B4A and its communication stack libraries. It is all over the place when it comes to timing, and I think its because it is running on the main thread and in the message queue along with everything else.

For example on Bluetooth classic, I can do Asyncstreams.write and then a sleep, and then another asyncstreams.write, and sometimes they get the proper delay, but sometimes they get jammed together. Packet timing is like going shooting nearsighted without your glasses. Ultimately I had to create a class and use a software FIFO to handle communication traffic to allow everything to return to prevent that "jamming" effect, But this doesn't fix the timing variances though.

So you are not alone in this. The only way to fix it in the above scenario is to return from literally everything and allow the message queue to process. But if background tasks get tossed on the same queue, it increases the delay. AS above.

I wonder if there is a way to move the Bluetooth library/stack over to its own thread, so the next experiment would be to use the threading library, launch a class in another thread with the bluetooth stuff, and then setup a software FIFO between the threads.

Anyways the bottom line from my experience is "Write" never completes until the message queue is free for it to do so.

 

[Erel]

The actual network calls are never done on the main thread.

The events are raised on the main thread. A gap of 20ms is not the fault of the message queue. You need to do a lot of processing to keep the main thread busy for 20 ms (in release mode of course).
The underlying BLE stack is huge and my guess that the delay happens somewhere inside. It is not built for streaming.

 

[ema01]

So, i revisited the issue.
Currently, the compiled java for BLEManager2.WriteData calls BluetoothGatt.writeCharacteristic (which by the way is currently maked as deprecated in the form that it's used) and then introduces a delay of a certain time (150ms? correlates with the delays i'm seeing).
I'm sure the delay is there for a reason (methods that some times return false and sometimes true because the BLE stack is a mess. retry but don't retry immediately), however i tried writing another method based on the original one, reducing the delay and then removing it altogether (but increasing the retry counter as otherwise it would fail if the whole data was bigger than some size.. i probably filled a FIFO without letting it empty) and in this specific application it seems to be working. If anything, a 10+ minutes task has been reduced to take a couple of minutes, gaps between packets are smaller, but random.

I will probably reimplement it to use the current version of the writeCharacteristic method and see if there is a difference