Wish Serial Port Non-Prefix End of data signature and timeout

rgarnett1955 · Mar 21, 2024

Hi,

When using the Serial Port and associated async streams without the Prefix feature it is difficult to coordinate reception of data with periodic transmit from B4j to the serial device.
The astream_NewData function is executed on some sort of timer such that data comes in fragmented with the fragments varying in size according to system utilisation.

A lot of hardware devices such as the Tiny Spectrum Analyser and Tiny VNA have serial ports that are designed for terminal applications. They echo character inputs and reply with data or error messages then a prompt. In many cases the prompt could be used to signify the end of the data packet. Other more sophisticated instruments use industry standards such as
SCPI - Standard Commands for Programmable Instruments which is also a text based arbitrary length protocol.

For a system where the B4x app sends periodic commands to a serial port and on receipt of the reply it sends a new command it is currently necessary to send the commands on a fixed time period where the period is longer than the maximum "reply time" because there is no way of "waiting" for data completion. When this time varies depending on the configuration of the serial port device the time period has to be adjusted manually to ensure new commands are not sent before the entire reply is received.

What is needed is a configurable end of packet signature from 1 to 6 bytes such that astreams_NewData is only triggered when that pattern is received. This scheme would also require a timeout to capture comm's failure and could also specify a maximum byte limit to the same ends. i.e. if a data stream was corrupted and the "end of packet" signature missed an error would be raised on some byte count.

Regards
Rob

Erel · Mar 21, 2024

This is something that should be implemented in the application layer, not in the library.
Search for AsyncStreamsText.

emexes · Mar 21, 2024

rgarnett1955 said:
such as the Tiny Spectrum Analyser

Hey I just ordered one (the 2.8" original) and apparently it'll be on my doorstep April 12th. I've occasionally had use for one in the past, and no doubt will again in the future, but it'll be fun to get it linked up to B4J via USB serial too. What could possibly go rwong? ?

Re: the tv antenna saga at my parents' place, I've come to the conclusion that there must be something causing a massive multipath problem between the transmitter on Mount Dandenong and my parents 7970 metres away. I gave up with the rooftop antenna, bought a $10 indoor Kmart special, and after a lot of trial and error I've found an antenna position in the room next to the room that the tv's in, that gives signal ????? on four of the five channels and ??? on the other.

It'll be great to confirm that my theory about multipath standing waves cancelling out is correct, or at least some clue as to wtf is going on.

rgarnett1955 · Mar 22, 2024

If that is the case why have a prefix mode? This is the same thing as having an end of packet method, but it requires both sides of the comm's link to have the ability to handle the prefix format B4x uses and is based on byte count not a byte signature?

Perhaps I should explain what I am trying to do. It is quite possible I am using B4x incorrectly. I have asked a question on the problem in the B4j forum, but they didn't answer the question that I asked:

ResumableSub - Wait For (resub) Complete(ret as Boolean) Not waiting - Runs through

Hi I have a project that sends scan commands sequentially over a serial port to a Spectrum Analyser (TinySA). For each scan I have to wait for the data to be completely received, 21 lines of data viz: 527000000 -1.192500e+02 0.000000000... The scan commands are initiated by a timer in...

www.b4x.com

I send an ascii formatted command terminated by a Chr(13) to a Tint Spectrum Analyser (TSA) the format is:

"scan 527000000 532000000 21 3 Chr(13)"

This represents start frequency, stop frequency no frequency points and output format

Send Command to TSA:

'--------------------------------------------------------------------------------------------Public Sub writeCmdSerial
    Dim str As String = "scan 527000000 532000000 21 3" 'This is the command to the instrument
    Private byteBuffer() As Byte
    Private bc As ByteConverter

    str = str.ToLowerCase & Chr(13)
    byteBuffer = bc.StringToBytes(str, "UTF8")
    Log(str)
    astream.Write(byteBuffer)
End Sub

After a time dependent of the setup of the TSA the TSA responds with lines of output data where each line represents the value at each frequency point (21 lines)
Output from TSA for "scan 527000000 532000000 21 3 chr(13)"

527000000 -8.843750e+01 0.000000000
527250000 -8.743750e+01 0.000000000
527500000 -8.546875e+01 0.000000000
527750000 -8.696875e+01 0.000000000
528000000 -8.796875e+01 0.000000000
528250000 -8.746875e+01 0.000000000
528500000 -8.746875e+01 0.000000000
528750000 -8.746875e+01 0.000000000
529000000 -8.646875e+01 0.000000000
529250000 -8.696875e+01 0.000000000
529500000 -8.796875e+01 0.000000000
529750000 -8.696875e+01 0.000000000
530000000 -8.546875e+01 0.000000000
530250000 -8.896875e+01 0.000000000
530500000 -8.696875e+01 0.000000000
530750000 -8.946875e+01 0.000000000
531000000 -8.846875e+01 0.000000000
531250000 -9.096875e+01 0.000000000
531500000 -8.846875e+01 0.000000000
531750000 -8.846875e+01 0.000000000
532000000 -8.896875e+01 0.000000000
ch>

On receipt of all the lines 21 in the example code (one for each frequency point) I then loop over all of the 6 bands.
I use a do-while loop to wait for a processComplete flag a bit like a semaphore.

Batch Processing Loop:

Private Sub processingLoop    For i = 0 To 6
        'Do a batch of 6 scans for this scanTimer Tick
        startTime = DateTime.Now
        Log("Start Time = " & DateTime.Time(startTime))
        writeCmdSerial
       
        'Wait here for all of the data to be processed.
        Do While processComplete = False
            Sleep(10)
        Loop

        processComplete = False
        endTime = DateTime.Now
        delta_ms = endTime - startTime
        Log("End period = " & delta_ms)
        Log(" ")
    Next
End Sub

The processComplete flag is set in a sub called from astreams_NewData viz:

AStream_NewData processing:

'--------------------------------------------------------------------------------------------
Private Sub AStream_NewData (Buffer() As Byte)
    cleanUpScanData(Buffer)
End Sub


'--------------------------------------------------------------------------------------------
Private Sub cleanUpScanData(buffer() As Byte)
    Private sb As StringBuilder
    sb.Initialize
    Private charset As String = "UTF8"
    Private newDataStart As Int = sb.Length
    sb.Append(BytesToString(buffer, 0, buffer.Length, charset))
    Private s As String = sb.ToString
    Private start As Int = 0
    
    For i = newDataStart To s.Length - 1
        noOfPackets = noOfPackets + 1
        Dim c As Char = s.CharAt(i)
        If i = 0 And c = Chr(10) Then '\n...
            start = 1 'might be a broken end of line character
            Continue
        End If
        If c = Chr(10) Then '\n
            CallSubDelayed2(Me, "checkAndProcessScanLine", s.SubString2(start, i))
'            checkAndProcessScanLine(s.SubString2(start, i))
            start = i + 1
        Else If c = Chr(13) Then '\r
            CallSubDelayed2(Me, "checkAndProcessScanLine", s.SubString2(start, i))
'            checkAndProcessScanLine(s.SubString2(start, i))
            If i < s.Length - 1 And s.CharAt(i + 1) = Chr(10) Then '\r\n
                i = i + 1
            End If
            start = i + 1
        End If
    Next
    If start > 0 Then sb.Remove(0, start)
End Sub


'--------------------------------------------------------------------------------------------
private Sub checkAndProcessScanLine(scanLineStr As String) As ResumableSub
    Private somevar As Double
    
    If scanLineStr.Contains("level") Or scanLineStr.Contains("scan") Then
        Return False
    End If

    scanDataStreamText = scanDataStreamText  & Chr(10) & scanLineStr
'    Log(scanDataStreamText)
    noValidScanLines = noValidScanLines + 1

    If noValidScanLines = 21 Then
        noValidScanLines = 0
    For i = 1 To 10000
            somevar = Sqrt(3458.9)
    Next
        '(...)
    End If
    processComplete = True
    Return True
End Sub

The process is then repeated on the next scanTimer tick in the example set at 4000 ms
Scan Timer Sub:
```
Public Sub scanTimer_Tick    startTime = DateTime.Now
    processingLoop
End Sub
```
With realtime badwidth setting of 10 kHz on the TSA the time to scan a single set of 21 points is 1.5 +/- 35 ms.

The problem is that the application does not behave as I expect. I log the serial port with Advanced Serial Port Monitor and I get:

Serial Port Sniffer Log - Abridged:

[*][CODE lang="b4x" title="Serial Port (Sniffer) Log - Abridged" highlight="47-54"]<20240322130541.234 TX>
scan 527000000 532000000 21 3            'comment Scan 1 complete response shown
<20240322130542.741 RX>
527000000 -8.793750e+01 0.000000000
527250000 -8.993750e+01 0.000000000
527500000 -8.946875e+01 0.000000000
527750000 -8.996875e+01 0.000000000
528000000 -8.896875e+01 0.000000000
528250000 -8.896875e+01 0.000000000
528500000 -8.846875e+01 0.000000000
528750000 -8.796875e+01 0.000000000
529000000 -8.896875e+01 0.000000000
529250000 -8.946875e+01 0.000000000
529500000 -8.996875e+01 0.000000000
529750000 -9.096875e+01 0.000000000
530000000 -8.796875e+01 0.000000000
530250000 -9.046875e+01 0.000000000
530500000 -8.996875e+01 0.000000000
530750000 -8.696875e+01 0.000000000
531000000 -8.946875e+01 0.000000000
531250000 -9.046875e+01 0.000000000
531500000 -8.896875e+01 0.000000000
531750000 -8.996875e+01 0.000000000
532000000 -8.996875e+01 0.000000000
ch> scan 527000000 532000000 21 3        'comment Scan 2
<20240322130544.252 RX>
527000000 -8.943750e+01 0.000000000
527250000 -9.043750e+01 0.000000000
'(...) comment removed these to shorten log
531750000 -8.946875e+01 0.000000000
532000000 -8.996875e+01 0.000000000
ch> scan 527000000 532000000 21 3        'comment Scan 3
<20240322130545.762 RX>
527000000 -8.943750e+01 0.000000000
527250000 -9.093750e+01 0.000000000
'(...) comment removed these to shorten log
531750000 -8.996875e+01 0.000000000
532000000 -8.896875e+01 0.000000000
ch> scan 527000000 532000000 21 3        'comment Scan 4
<20240322130547.170 RX>
527000000 -8.843750e+01 0.000000000
527250000 -8.793750e+01 0.000000000
'(...) comment removed these to shorten log
531750000 -9.046875e+01 0.000000000
532000000 -9.046875e+01 0.000000000
ch> scan 527000000 532000000 21 3        'comment Scan 5
<20240322130547.689 TX>                    'comment anomaly (
scan 527000000 532000000 21 3            '    ...
<20240322130548.691 TX>                    '    ...
scan 527000000 532000000 21 3            '    ...
<20240322130550.203 TX>                    '   ...
scan 527000000 532000000 21 3            '    ...
<20240322130551.616 TX>                    '    ...
scan 527000000 532000000 21 3            ')
<20240322130553.121 RX>
527000000 -8.843750e+01 0.000000000
527250000 -9.093750e+01 0.000000000
'(...) comment removed these to shorten log
531750000 -8.896875e+01 0.000000000
532000000 -8.846875e+01 0.000000000
ch> scan 527000000 532000000 21 3
<20240322130554.533 RX>
527000000 -8.943750e+01 0.000000000
527250000 -8.843750e+01 0.000000000
'(...) comment removed these to shorten log
531750000 -9.046875e+01 0.000000000
532000000 -8.896875e+01 0.000000000
ch> scan 527000000 532000000 21 3
<20240322130556.043 RX>
'(...)

The output of the B4j log is:

B4j Log Output:

Start Time = 13:05:39
scan 527000000 532000000 21 3        'comment: Scan 1
End period = 1547
 
Start Time = 13:05:41
scan 527000000 532000000 21 3        'comment: Scan 2
End period = 1511
 
Start Time = 13:05:42                'comment: Scan 3
scan 527000000 532000000 21 3
Start Time = 13:05:43
scan 527000000 532000000 21 3        'comment: anomaly scan 4?
End period = 567
 
Start Time = 13:05:445?4            'comment: anomaly
End period = 1507
 
Start Time = 13:05:45
scan 527000000 532000000 21 3
End period = 1414
 
Start Time = 13:05:47
scan 527000000 532000000 21 3
Start Time = 13:05:47
scan 527000000 532000000 21 3
End period = 1002
 
Start Time = 13:05:48
scan 527000000 532000000 21 3
End period = 1511
 
Start Time = 13:05:50
scan 527000000 532000000 21 3
End period = 1414
 
Start Time = 13:05:51
scan 527000000 532000000 21 3
Start Time = 13:05:51
scan 527000000 532000000 21 3
End period = 1438

I have attached the two log files in the zip of the project;

The problem is that I need a semaphore to control the batcj loop, but the way I have implemented it it doesn't work. I suspect it is because there is only a single thread, but I don't really know.

If I could force the astreams_NewData to wait until it gets the end of frame signature ">cr " I would not reauire a semaphore.

I have struggeld with this for a 100 hours and have gotten nowhere. In the end I ran the thing so that each scan was initiated by the timer tick with a period long enough to guarantee the TSA complete its response. The problem with this approach is the processing time of the TSA varies with, the number of points, the real time bandwidth setting and a couple of other variables. The time can be anywhere from 0.1 seconds to tens of seconds and can only be determined from a test run.

Regards
Rob

emexes · Apr 4, 2024

emexes said:
Hey I just ordered one (the 2.8" original) and apparently it'll be on my doorstep April 12th.

Arrived yesterday afternoon. ?

emexes said:
get it linked up to B4J via USB serial too. What could possibly go rwong?

Just started that this afternoon. So far, starting from a B4J New UI project, I am at:

B4X:

Sub Process_Globals
    'Private fx As JFX
    'Private MainForm As Form
    'Private xui As XUI
    'Private Button1 As B4XView

    Dim bc As ByteConverter

    Private sp As Serial
    Private astr1 As AsyncStreams
End Sub

Sub AppStart (Form1 As Form, Args() As String)
    'MainForm = Form1
    'MainForm.RootPane.LoadLayout("Layout1")
    'MainForm.Show

    sp.Initialize("")
  
    Dim ports As List = sp.ListPorts
    If ports.Size = 0 Then
        Log("No serial ports found")
        Return
    End If
  
    For I = 0 To ports.Size - 1
        Log(I & TAB & ports.Get(I))
    Next

    Log("Opening serial port " & ports.Get(0))
    sp.Open(ports.Get(0))
    astr1.Initialize(sp.GetInputStream, sp.GetOutputStream, "Astr1")

    Dim TestCommands() As String = Array As String("", "", "version", "help", "data 2", "")
  
    For Each S As String In TestCommands
        Astr1_SendCommand(S)
        Sleep(500)
    Next
End Sub

Sub Astr1_SendCommand(S As String)
    astr1.Write(bc.StringToBytes(S, "UTF8"))
    astr1.Write(Array As Byte(13))
  
    Log(DateTime.Now & " > " & TAB & """" & S & "<CR>""")
End Sub

Sub Astr1_NewData (Buffer() As Byte)
    Dim s As String = BytesToString(Buffer, 0, Buffer.Length, "UTF8")
  
    s = s.Replace(Chr(10), "<LF>").Replace(Chr(13), "<CR>")
  
    Log(DateTime.Now & "   " & TAB & """" & s & """")
End Sub

Log Output:

Waiting for debugger to connect...
Program started.
0    COM4
Opening serial port COM4
1712210949954 >     "<CR>"
1712210949974       "<CR><LF>?<CR><LF>ch> "
1712210950467 >     "<CR>"
1712210950509       "<CR><LF>?<CR><LF>ch> "
1712210950974 >     "version<CR>"
1712210950990       "version<CR><LF>tinySA_v1.4-40-g2f63e1c<CR><LF>ch> "
1712210951485 >     "help<CR>"
1712210951493       "help"
1712210951504       "<CR><LF>commands: version reset freq dac sweep_voltage saveconfig clearconfig data frequencies scan scanraw zero sweep test touchcal touchtest pause resume repeat status caloutput save recall trace trigger marker line usart usart_cfg capture refresh touch release vbat vbat_offset help info color if actual_freq attenuate level sweeptime leveloffset levelchange modulation rbw mode spur load ext_gain output deviceid selftest correction calc threads<CR><LF>ch> "
1712210951994 >     "data 2<CR>"
1712210952011       "data 2<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.154062e+02<CR><LF>-1.154062e+02<CR><LF>-1.154062e+02<CR><LF>-1.124062e+02<CR><LF>-1.154062e+02<CR><LF>-1.109062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.134062e+02<CR><LF>-1.124062e+02<CR><LF>-1.109062e+02<CR><LF>-1.154062e+02<CR><LF>-1.124062e+02<CR><LF>-1.134062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.109062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.154062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.154062e+02<CR><LF>-1.154062e+02<CR><LF>-1.109062e+02<CR><LF>-1.139062e+02<CR><LF>-1.154062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.154062e+02<CR><LF>"
1712210952013       "-1.109062e+02<CR><LF>-1.109062e+02<CR><LF>-1.139062e+02<CR><LF>-1.154062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e"
1712210952015       "+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.154062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.154062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.154062e+02<CR><LF>-1.139062e+02<CR><LF>-1.154062e+02<CR><LF>-1.124062e+02<CR><LF>-1.109062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.154062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.109062e+02<CR><LF>-1.154062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.104062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.154062e+02<CR><LF>-1.139062e+02<CR><LF>-1.154062e+02<CR><LF>-1.109062e+02<CR><LF>-1.109062e+02<CR><LF>-1.109062e+02<CR><LF>-1.109062e+02<CR><LF>-1.154062e+02<CR><LF>-1.109062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.094062e+02<CR><LF>-1.124062e+02<CR><LF>-1.154062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.109062e+02<CR><LF>-1.094062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.154062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.119062e+02<CR><LF>-1.119062e+02<CR><LF>-1.134062e+02<CR><LF>-1.094062e+02<CR><LF>-1.094062e+02<CR><LF>-1.109062e+02<CR><LF>-1.089062e+02<CR><LF>-1.089062e+02<CR><LF>-1.124062e+02<CR><LF>-1.039062e+02<CR><LF>-1.074062e+02<CR><LF>-1.104062e+02<CR><LF>-9.990625e+01<CR><LF>-1.049062e+02<CR><LF>-9.940625e+01<CR><LF>"
1712210952016       "-1.024062e+02<CR><LF>-1.024062e+02<CR><LF>-9.640625e+01<CR><LF>-9.740625e+01<CR><LF>-9.540625e+01<CR><LF>-9.490625e+01<CR><LF>-9.840625e+01<CR><LF>-9.740625e+01<CR><LF>-9.640"
1712210952031       "625e+01<CR><LF>-9.640625e+01<CR><LF>-9.590625e+01<CR><LF>-9.440625e+01<CR><LF>-9.190625e+01<CR><LF>-9.690625e+01<CR><LF>-9.790625e+01<CR><LF>-9.990625e+01<CR><LF>-1.009062e+02<CR><LF>-1.004062e+02<CR><LF>-9.990625e+01<CR><LF>-1.009062e+02<CR><LF>-1.059062e+02<CR><LF>-1.054062e+02<CR><LF>-1.094062e+02<CR><LF>-1.089062e+02<CR><LF>-1.074062e+02<CR><LF>-1.079062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.089062e+02<CR><LF>-1.109062e+02<CR><LF>-1.109062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.154062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.089062e+02<CR><LF>-1.109062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.094062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.154062e+02<CR><LF>-1.109062e+02<CR><LF>-1.109062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.124062e+02<CR><LF>-1.119062e+02<CR><LF>-1.109062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.089062e+02<CR><LF>-1.124062e+02<CR><LF>-1.134062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.134062e+02<CR><LF>-1.109062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.139062e+0"
1712210952038       "2<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.094062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.154062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.109062e+02<CR><LF>-1.154062e+02<CR><LF>-1.124062e+02<CR><LF>-1.109062e+02<CR><LF>-1.109062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.154062e+02<CR><LF>-1.154062e+02<CR><LF>-1.119062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.094062e+02<CR><LF>-1.094062e+02<CR><LF>-1.154062e+02<CR><LF>-1.154062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.154062e+02<CR><LF>-1.109062e+02<CR><LF>-1.154062e+02<CR><LF>-1.154062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.154062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.139062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.124062e+02<CR><LF>-1.109062e+02<CR><LF>-1.154062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.154062e+02<CR><LF>-1.154062e+02<CR><LF>-1.139062e+02<CR><LF>-1.109062e+02<CR><LF>-1.154062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>-1.124062e+02<CR><LF>-1.139062e+02<CR><LF>ch> "
1712210952503 >     "<CR>"
1712210952523       "<CR><LF>?<CR><LF>ch> "

emexes · Apr 4, 2024

I'll have a go at receiving response for this command:

rgarnett1955 said:

Log output:

scan 527000000 532000000 21 3            'comment Scan 1 complete response shown
<20240322130542.741 RX>
527000000 -8.793750e+01 0.000000000
527250000 -8.993750e+01 0.000000000
527500000 -8.946875e+01 0.000000000

when I get back home tonight.

peacemaker · Apr 4, 2024

I use something like this: https://www.b4x.com/android/forum/threads/asyncstream-com-port-buffer.143231/post-907643

emexes · Apr 4, 2024

emexes said:
I'll have a go at receiving response for this command

Give this a burl:

B4X:

Sub Process_Globals
	'Private fx As JFX
	'Private MainForm As Form
	'Private xui As XUI 
	'Private Button1 As B4XView

	Dim bc As ByteConverter

	Private sp As Serial
	Private AStream As AsyncStreams

	Private TsaCommand As String    'the command that was sent (so that when we get a response, we know for what)
	Private TsaBuffer As StringBuilder    'collect up incoming response bytes

	Private TsaStartOfPacket As String    'marker string immediately before response bytes
	Private TsaEndOfPacket As String    'marker string immediately after response bytes
	
	Private TsaCommandQueue As List    'list of commands still to be sent
End Sub

Sub AppStart (Form1 As Form, Args() As String)
	'MainForm = Form1
	'MainForm.RootPane.LoadLayout("Layout1")
	'MainForm.Show

	'Dim SupEnc() As String = bc.SupportedEncodings
	'For I = 0 To SupEnc.Length - 1
	'	Log(I & TAB & SupEnc(I))
	'Next
	
	sp.Initialize("")
	
	Dim ports As List = sp.ListPorts
	If ports.Size = 0 Then
		Log("No serial ports found")
		Return
	End If
	
	For I = 0 To ports.Size - 1
		Log(I & TAB & ports.Get(I))
	Next

	Log("Opening serial port " & ports.Get(0))
	sp.Open(ports.Get(0))
	AStream.Initialize(sp.GetInputStream, sp.GetOutputStream, "AStream")

	Dim TestCommands() As String = Array As String( _
	    ""                                        , _
		"version"                                 , _
		"help"                                    , _
		"data 2"                                  , _
		"capture"                                 , _    'returns binary data 320 x 240 pixels x 2 bytes, with no terminating CR LF
		"scan  527000000 532000000 21 3"          , _
		"howdy"                                   , _    'invalid
		"DATA 2"                                  , _    'invalid
		"daTa 2"                                  , _    'invalid
		"data"                                    , _    'invalid
		"data -1"                                 , _    'invalid
		"data 7"                                  , _    'invalid
		""                                          _
	)
	
	TsaCommandQueue.Initialize
	TsaCommandQueue.AddAll(TestCommands)

	TsaSendNextCommand       'start automatic processing of queued commands
	
	Sleep(2000)              'enough time for 6 commands, given 300 ms gap between commands
	TsaCommandQueue.Clear    'delete commands still in queue
End Sub

'reveal control codes
Sub Readable(S As String) As String
	Return S.Replace(Chr(10), "<LF>").Replace(Chr(13), "<CR>")
End Sub

'cap string length at specified maximum
Sub CapLength(S As String, MaxLength As Int) As String
	If S.Length > MaxLength Then
		Return S.SubString2(0, MaxLength - 3) & "..."
	Else
		Return S
	End If
End Sub

'log label + quoted string
Sub LogString(L As String, S As String)
	Log(L & TAB & S.Length & TAB & "= """ & CapLength(Readable(S), 200) & """")
End Sub

'send command to TSA
Sub TsaSendCommand(S As String)
	TsaCommand = S
	TsaBuffer.Initialize

	AStream.Write(bc.StringToBytes(TsaCommand & Chr(13), "ISO-8859-1"))
	
	TsaStartOfPacket = S & Chr(13) & Chr(10)
	TsaEndOfPacket = "ch> "
	
	'prepend CR LF to TsaEndOfPacket except for binary data responses
	If TsaCommand.StartsWith("capture") Then
	Else
		TsaEndOfPacket = Chr(13) & Chr(10) & TsaEndOfPacket	
	End If

End Sub

'is called when receive TSA response
Sub TsaHandleResponse(LaCommand As String, LaResponse As String)
    LogString("  Command", LaCommand)
	LogString("  Response", LaResponse)

	Sleep(300)    '300 ms gap between commands	
	TsaSendNextCommand
End Sub


Sub TsaSendNextCommand
	If TsaCommandQueue.Size > 0 Then
		Dim NextCommand As String = TsaCommandQueue.Get(0)
		TsaCommandQueue.RemoveAt(0)
		
		TsaSendCommand(NextCommand)
		Log("<<< sent command """ & NextCommand & """ with " & TsaCommandQueue.Size & " more to go >>>")
	End If
End Sub

Sub AStream_NewData (Buffer() As Byte)
	TsaBuffer.Append(bc.StringFromBytes(Buffer, "ISO-8859-1"))
	
	Dim BufferContainsSpaceFlag As Boolean = False
	For I = 0 To Buffer.Length - 1
		If Buffer(I) = 32 Then    'ASCII 32 = space
			BufferContainsSpaceFlag = True
			Exit
		End If
	Next
	
	If BufferContainsSpaceFlag Then    'might have just received terminating sequence CR LF ch> space
		Dim TempBuffer As String = TsaBuffer.ToString
		Dim I2 As Int = TempBuffer.IndexOf(TsaEndOfPacket)
		If I2 > -1 Then
			Dim I1 As Int = TempBuffer.IndexOf(TsaStartOfPacket)
			If I1 > -1 Then
				If I1 < I2 Then
					Dim TempResponse As String = TempBuffer.SubString2(I1 + TsaStartOfPacket.Length, I2)
					TsaHandleResponse(TsaCommand, TempResponse)
				End If
			End If
			TsaBuffer.Remove(0, I2 + TsaEndOfPacket.Length)
		End If
	End If
End Sub

including test cases:

B4X:

	Dim TestCommands() As String = Array As String( _
	    ""                                        , _
		"version"                                 , _
		"help"                                    , _
		"data 2"                                  , _
		"capture"                                 , _    'returns binary data 320 x 240 pixels x 2 bytes, with no terminating CR LF
		"scan  527000000 532000000 21 3"          , _
		"howdy"                                   , _    'invalid
		"DATA 2"                                  , _    'invalid
		"daTa 2"                                  , _    'invalid
		"data"                                    , _    'invalid
		"data -1"                                 , _    'invalid
		"data 7"                                  , _    'invalid
		""                                          _
	)

Log output:

WARNING: package com.sun.javafx.embed.swing.oldimpl not in javafx.swing
Waiting for debugger to connect...
Program started.
0	COM4
Opening serial port COM4
<<< sent command "" with 12 more to go >>>
  Command	0	= ""
  Response	1	= "?"
<<< sent command "version" with 11 more to go >>>
  Command	7	= "version"
  Response	23	= "tinySA_v1.4-40-g2f63e1c"
<<< sent command "help" with 10 more to go >>>
  Command	4	= "help"
  Response	441	= "commands: version reset freq dac sweep_voltage saveconfig clearconfig data frequencies scan scanraw zero sweep test touchcal touchtest pause resume repeat status caloutput save recall trace trigger..."
<<< sent command "data 2" with 9 more to go >>>
  Command	6	= "data 2"
  Response	4348	= "-8.646875e+01<CR><LF>-8.696875e+01<CR><LF>-8.846875e+01<CR><LF>-8.896875e+01<CR><LF>-8.796875e+01<CR><LF>-8.746875e+01<CR><LF>-8.846875e+01<CR><LF>-8.546875e+01<CR><LF>-8.646875e+01<CR><LF>-8.79687..."
<<< sent command "capture" with 8 more to go >>>
  Command	7	= "capture"
  Response	153600	= "ÿÿÿÿ..."
<<< sent command "scan  527000000 532000000 21 3" with 7 more to go >>>
  Command	30	= "scan  527000000 532000000 21 3"
  Response	796	= "527000000 -8.696875e+01 0.000000000 <CR><LF>527250000 -8.796875e+01 0.000000000 <CR><LF>527500000 -8.546875e+01 0.000000000 <CR><LF>527750000 -8.796875e+01 0.000000000 <CR><LF>528000000 -8.696875e+..."

rgarnett1955 · Apr 9, 2024

Hi,

Sorry to be slow getting back to you on this and thank's for all the work you put in. I ended up doing basically what you advised, using a time delay between commands that is long enough for the slowest response from _NewData. Because the time response from the TinySA varies greatly depending on its setup, I measure the time and then set the time delay based on this test. It is not ideal, but I asked Erel for the ability to set end of frame characters in serial streams, but he said:

"This is something that should be implemented in the application layer, not in the library."

Hmmm?

Probably Erel is just too busy on other things.

I am sure that the underlying java code could do what I want, but I don't have a clue about how to do that. I wish I was smart enough to write wrappers, but I don'e know any Java and I don't have the time to learn it.

Anyhow, I have the application working and it has been scanning for two weeks now and has done over 2 million scans without a hitch so I'm a happy camper now.

The RF Monitor App can be found at: https://www.b4x.com/android/forum/t...logger-for-tine-spectrum-analyzer-tsa.160585/

Kind Regards
Rob

Wish Serial Port Non-Prefix End of data signature and timeout

rgarnett1955

Active Member

Erel

B4X founder

emexes

Expert

rgarnett1955

Active Member

ResumableSub - Wait For (resub) Complete(ret as Boolean) Not waiting - Runs through

Attachments

emexes

Expert

emexes

Expert

peacemaker

Expert

emexes

Expert

rgarnett1955

Active Member

Similar Threads