B4J Question [PyBridge] Waiting For (Py.Flush) Complete (Success As Boolean) doesn't seem to Wait

[William Lancee]

This is just the example as shown in the PyBridge tutorial.
Does anyone have any idea why the Wait For here doesn't work? Is Complete set somehow?
I tested Py.Flush.Completed but it is false.

Private Sub B4XPage_Created (Root1 As B4XView)
    Root = Root1
    Root.LoadLayout("MainPage")
    Py.Initialize(Me, "Py")
    Dim opt As PyOptions = Py.CreateOptions("Python/python/python.exe")
    Py.Start(opt)
    Wait For Py_Connected (Success As Boolean)
    If Success = False Then
        LogError("Failed to start Python process.")
        Return
    End If
    PrintPythonVersion
    
    Dim Tuple As PyWrapper = Py.WrapObject(Array(1, 2, 3, 4, 5))
    Tuple.Print
    Tuple.TypeOf.Print2("type is:", "", False)
    Dim list As PyWrapper = Tuple.ToList
    list.Run("append").Arg(6)
    list.Print
    Wait For (Py.Flush) Complete (Success As Boolean)        'NOT waiting
    Log("after " & Success)
    Log(Py.Flush.Completed)
End Sub

'Logs:
'Server Is listening on port: 62118
'Python path: C:\Program Files\Anywhere Software\B4J\libraries\Python\python\python.exe
'Call B4XPages.GetManager.LogEvents = True To enable logging B4XPages events.
'connected
'starting PyBridge v1.00
'watchdog set To 30 seconds
'Connecting To port: 62118
'after true
'false
'Python version: 3.12.8 (tags/v3.12.8:2dc476b, Dec  3 2024, 19:30:04) [MSC v.1942 64 Bit (AMD64)]
'(1, 2, 3, 4, 5)
'type is: <class 'tuple'>
'[1, 2, 3, 4, 5, 6]

 

[William Lancee]

Same problem with non-UI

Server is listening on port: 53022
Python path: C:\Program Files\Anywhere Software\B4J\libraries\Python\python\python.exe
connected
starting PyBridge v1.00
watchdog set to 30 seconds
Connecting to port: 53022
after true
true
Python version: 3.12.8 (tags/v3.12.8:2dc476b, Dec  3 2024, 19:30:04) [MSC v.1942 64 bit (AMD64)]
(1, 2, 3, 4, 5)
type is: <class 'tuple'>
[1, 2, 3, 4, 5, 6]
PyBridge disconnected
Process completed. ExitCode: 1

Private Sub Start
    Py.Initialize(Me, "Py")
    Dim opt As PyOptions = Py.CreateOptions("Python/python/python.exe")
    Py.Start(opt)
    Wait For Py_Connected (Success As Boolean)
    If Success = False Then
        LogError("Failed to start Python process.")
        Return
    End If
    PrintPythonVersion
   
    makeitseparate

    'When possible, kill the python process before exiting:
    Sleep(500)
    CloseProgram
End Sub

Sub makeitseparate
    Dim Tuple As PyWrapper = Py.WrapObject(Array(1, 2, 3, 4, 5))
    Tuple.Print
    Tuple.TypeOf.Print2("type is:", "", False)
    Dim list As PyWrapper = Tuple.ToList
    list.Run("append").Arg(6)
    list.Print
    Dim rs As ResumableSub = Py.Flush
    Wait For (Py.Flush) Complete (Success As Boolean)        'It is waiting - NOT for me
    Log("after " & Success)
    Log(rs.Completed)
End Sub

 

[William Lancee]

I am now going to try it on my Laptop.

 

[William Lancee]

Like on @Daestrum and @aeric computers, my Laptop does this the way it is expected, with the same project folders as my desktop.
So we got narrowed down to a problem with my desktop. What could it be?

 

[aeric]

You are not alone.
I also experience the same but it happens randomly.

As stated by Erel, python and B4J (Java) are running on their own threads.

I think by using wait for still cannot guarantee the logs will appear in order.

Maybe something happening after the Flush method and jShell.

I understand that it should be the last thing pywrapper finish all the python code and close the object.

If this doesn't affect the execution of python code then I think we don't need to worry too much.

I am new to pybridge so I can be wrong.

 

[William Lancee]

Worried I'm not. I started to examine the source code of PyBridge, PyUtils, PyComm, and PyWrapper.
The Py.Flush is send to the following in PyUtils. This waits for task-specific "AsyncTask_Received".
Something else must have raised that event.

Public Sub Flush As ResumableSub
    Dim task As PyTask = CreatePyTask(0, TASK_TYPE_FLUSH, Array())
    Comm.SendTaskAndWait(task)
    Wait For (task) AsyncTask_Received (task As PyTask)
    If task.TaskType = TASK_TYPE_ERROR Then
        mBridge.PyLastException = task.Extra.Get(0)
        Return False
    Else
        Return True
    End If
End Sub

The PyComm module raises the event:

Private Sub AStream_NewData (Buffer() As Byte)
    Dim o() As Object = ser.ConvertBytesToObject(Buffer)
    Dim Task As PyTask = mBridge.Utils.CreatePyTask(o(0), o(1), o(2))
    If WaitingTasks.ContainsKey(Task.TaskId) Then
        jME.RunMethod("raiseEventWithSenderFilter", Array(mBridge.Utils, "asynctask_received", WaitingTasks.Remove(Task.TaskId), Array(Task)))
    Else
        CallSub2(mBridge, "Task_Received", Task)   
    End If
End Sub

Note that "WaitingTasks" is checked whether it contains the Flush task.
Could that be it? If a CPU is really fast, could Flush task be put on the queue while other tasks are not yet put there?
At this point I remain guessing.

 

[Daestrum]

In a non-ui , using your last code (added sleep(15) ) will produce consistent result. If I reduce to 10 then the after can be before the list.print output
(on my previous run I posted, I did one run and printed result, when I multiple runs it was inconsistent where after was printed)

Sub makeitseparate
    Dim Tuple As PyWrapper = Py.WrapObject(Array(1, 2, 3, 4, 5))
    Tuple.Print
    Tuple.TypeOf.Print2("type is:", "", False)
    Dim list As PyWrapper = Tuple.ToList
    list.Run("append").Arg(6)
    list.Print
    Sleep(15)  ' <------ ADDED THIS LINE
    Dim rs As ResumableSub = Py.Flush
    Wait For (Py.Flush) Complete (Success As Boolean)        'It is waiting - NOT for me
    Log("after " & Success)
    Log(rs.Completed)
End Sub

 

[William Lancee]

@Daestrum We're pretty much on the same page.
Here is my testing, there a small range 15 to 22 on my system, but then it goes wrong again:

Sub makeitseparate
    For delay = 15 To 25
        Log($"Delay set for: ${delay} msecs"$)
        Dim Tuple As PyWrapper = Py.WrapObject(Array(1, 2, 3, 4, 5))
        Tuple.Print
        Tuple.TypeOf.Print2("type is:", "", False)
        Dim list As PyWrapper = Tuple.ToList
        list.Run("append").Arg(6)
        list.Print
        Sleep(delay)
        Dim marktime As Long = DateTime.now
        Dim rs As ResumableSub = Py.Flush
        Log(rs.Completed)            'This is shown before the Flush
        Wait For (Py.Flush) Complete (Success As Boolean)
        Log(DateTime.now - marktime)
        Log("after " & Success)        'This is shown after the Flush
        Log(rs.Completed)
        Sleep(1000)
        Log(TAB)
    Next
End Sub

 

[Daestrum]

What does the elapsed time say - out of interest

Also (no idea if correct) does Py.Flush work on the stream to python and not python return stream. (Maybe to make sure python has all the code)

 

[William Lancee]

The durations are in 1 to 6 msecs.
The event is raised in AStream_NewData. But I don't know is this is a handshake or not.

 

[Erel]

I think that it was mentioned above. The python code is running in a different process. The B4J process reads the python process output asynchronously. This causes the logs of the two processes to become interleaved.

The code is still executed in the correct order.

 

[William Lancee]

Thank you all. @aeric I have marked your post #24 as the Solution to what appeared to be a mystery.
I understand it now. It's just the Logs that are out of sync - the B4X and Python execution order is correct.

This discussion has been another good example of why I love B4X!
Can you imagine getting an answer to such a complex issue from anyone else?