B4J Question How to update UI during long computation?

[ranul]

Hi,

I have read a lot in the forum but still couldn't find a solution.
Let's say I need to parse a huge file (file size: googolplex bytes). Let's say I need to count how many "Q" characters are in the file.
I have a form with a label and during the parsing of the file I would like to update the label with the number of "Q" found.
The parsing algorithm is simply checking every character in the file and comparing it to "Q".

How do I update the label during parsing?

Thanks

 

[drgottjr]

not sure what you've been reading (perhaps the chitchat thread).

have you tried label1.text = qcount ?

there are also options like:
if qcount mod 10 = 0 then
label1.text = qcount
end if

 

[angel_]

Maybe it will be useful to you:

[B4X] Resumable Subs - Sleep / Wait For

New video tutorial: Resumable subs is a new feature added in B4J v5.50 / B4i v4.00 / B4A v7.00. It dramatically simplifies the handling of asynchronous tasks. (This feature is a variant of stackless coroutines.) The special feature of resumable subs is that they can be paused, without...

www.b4x.com

 

[emexes]

Sleep(0) after updating label, ot after every (say) x thousand bytes read from file or x thousand times around a loop.

For I = 1 to 1000000000
    If (I Mod 12345) = 0 Then Sleep(0)    'every 12345th loop
    If Bit.And(I, 0xFFF) = 0 Then Sleep(0)    'every 4096th loop
    '... calculations etc
Next

Although the Sleep turns it into a ResumableSub, so you'll probably need a WaitFor after the call to it, to catch the "early exit".

 

[Erel]

1. Before you do anything - make sure to test it in release mode. The program will run much faster.

2. The actual details are very important here. If you are reading a very large file and it is too slow then you should read it in chunks and call Sleep(0) as @emexes wrote. You can use TextReader to read it line by line (and don`t call Sleep(0) after every line).

 

[emexes]

I would like to update the label with the number of "Q" found

If you know how large the file is and how much you've checked so far, then you could update the label with the estimated number of 'Q' in the file, eg:

EstimatedFinalCount = Count * (InputFile.Size / Max(InputFile.CurrentPosition, 1))    'avoid divide-by-zero

and provide a way to abort out if that estimate is good enough and the user doesn't need or want to wait for an exact final count.

 

[ranul]

Maybe I confused you with my big file example, but my intention is to understand how to update the UI during a blocking computation.

See the following simple example:

dim x as int = 0
Do While x < 100000
    x = x + 1
    lbl.Text = x
Loop

I would like to update lbl during the counting.

How can I do it?

 

[emexes]

how to update the UI during a blocking computation.

Did you try this? How did it go?

Sleep(0) after updating label, ot after every (say) x thousand bytes read from file or x thousand times around a loop.

 

[ranul]

Did you try this? How did it go?

Thanks, It works ok when all the code is in the upper layer.
Like this:

Sub Button1_Click
    Dim x As Int = 0
    
    Do While x < 10000
        x = x + 1
        Label1.Text = x
        Sleep(0)
    Loop
End Sub

But my real code is very nested (like 10 levels down) and it is harder to use Sleep(0) in that scenario.

 

[emexes]

it is harder to use Sleep(0) in that scenario.

Is that because the sub with the Sleep in it "exits early" and the sub above it continues before the sub that it just called has finished.

The Sleep(0) doesn't have to be in the bottom layer of your "10 levels down" nesting.

It just has to be somewhere that it gets looped through regularly eg 1-100 times per second.

 

[ranul]

Is that because the sub with the Sleep in it "exits early" and the sub above it continues before the sub that it just called has finished.

The Sleep(0) doesn't have to be in the bottom layer of your "10 levels down" nesting.

It just has to be somewhere that it gets looped through regularly eg 1-100 times per second.

If the loop (with the sleep) is 10 levels down, I will be forced to change all the nesting functions to ResumableSubs and use "Wait For" all the way down.

 

[emexes]

use "Wait For" all the way down.

Made me laugh, thinking of my Indian housemate and his turtles all the way down story.

If the loop (with the sleep) is 10 levels down

The Sleep(0) doesn't need to be the full 10 levels down - it just needs to be far enough down that it is called often enough to keep your users happy.

Also, the Sleep(0) doesn't need to be in the same Sub as the Label.Text = status update. That can be ten levels deep, and then only updated to the screen when the Sleep(0), several levels back up the call stack, yields control back to the UI updater.

 

[emexes]

The bit about the multiple nesting has me wondering if you actually need Wait For at each level. I think I've only ever done it with long-running code one level down.

But I remember that sometimes what I'd do instead of a Wait For is set a Timer to check every 100 milliseconds for a "manual" BackgroundProcessingFinished flag that is set once all the processing has finished.

Actually, not a flag: it's usually a Long that returns the total execution time of the background processing. 0 means not finished, anything else means it's finished and took x milliseconds.

 

[ranul]

The bit about the multiple nesting has me wondering if you actually need Wait For at each level. I think I've only ever done it with long-running code one level down.

But I remember that sometimes what I'd do instead of a Wait For is set a Timer to check every 100 milliseconds for a "manual" BackgroundProcessingFinished flag that is set once all the processing has finished.

Actually, not a flag: it's usually a Long that returns the total execution time of the background processing. 0 means not finished, anything else means it's finished and took x milliseconds.

The blocking loop is in Level 10, so the Sleep(0) must be inside the blocking loop in level 10.
Once I put Sleep(0) in level 10, it changes the function to ResumableSub and forced me to use Wait For in the father function and so on until I reach level 1.

 

[emexes]

forced me to use Wait For in the father function and so on until I reach level 1

Fair enough, maybe it's more trouble than it's worth.

The blocking loop

What is doing the blocking? I thought it was a compute-bound limit. How long does it block for? Like, is it downloading a web page? Or reading a file over a slowish netowrk?

 

[ranul]

Never mind, thanks for you help @emexes

 

[MicroDrie]

Let's say I need to parse a huge file (file size: googolplex bytes). Let's say I need to count how many "Q" characters are in the file.
I have a form with a label and during the parsing of the file I would like to update the label with the number of "Q" found.

The bottom line is that you don't put a whole loaf of bread in your mouth, but eat it in pieces of the bread cut into sandwiches (okay, maybe you can stuff a whole sandwich in your mouth ).
The same goes for your software challenge: divide the (downloadable) file into bite-sized chunks, investigate how many search criteria you can find in that bite-sized chunk, update the label with the total number of criteria found (add a sleep(0) command if necessary) and repeat this with the next block until you have read all the blocks.
Depending on the speed of the device you are going to use the program on, you can adjust the buffer size in a multiple of 512 bytes.

Sub Class_Globals
    Private Root As B4XView 'ignore
    Private xui As XUI 'ignore
    Private lblfoundchar As Label
End Sub

'You can add more parameters here.
Public Sub Initialize
    
End Sub

'This event will be called once, before the page becomes visible.
Private Sub B4XPage_Created (Root1 As B4XView)
    Root = Root1
    'load the layout to Root
    Root.LoadLayout("frmB4XRegexQ")
    B4XPages.SetTitle(Me, "RegexQ Download Page")
End Sub

Private Sub DownloadAndCountQs(Link As String, BlockSize As Int)
    Dim j As HttpJob
    j.Initialize("", Me)
    j.Download(Link)
    Wait For (j) JobDone(j As HttpJob)
    
    If j.Success Then
        Dim in As InputStream = j.GetInputStream
        Dim buffer(BlockSize) As Byte
        Dim bytesRead As Int
        Dim totalQs As Int = 0
        Dim blockCount As Int = 0
        
        Do While in.BytesAvailable > 0
            bytesRead = in.ReadBytes(buffer, 0, BlockSize)
            Dim blockText As String = BytesToString(buffer, 0, bytesRead, "UTF-8")
            'Count Q characters in this block
            Dim qCount As Int = CountChar(blockText, "R")
            totalQs = totalQs + qCount
            blockCount = blockCount + 1
            lblfoundchar.Text = $"Block ${blockCount}: ${qCount} Qs (Total: ${totalQs})"$
            Sleep(0)
            Log($"Block ${blockCount}: ${qCount} Qs (Total: ${totalQs})"$)
        Loop
        in.Close
        Log($"Final count: ${totalQs} Q characters found"$)
    Else
        Log("Download failed")
    End If
    j.Release
End Sub

Private Sub CountChar(text As String, CountCharStr As String) As Int
    Dim pattern As String = CountCharStr 'Simple pattern to match the character
    Dim tmp_count As Int = 0
    Dim Matcher1 As Matcher = Regex.Matcher(pattern, text)
    Do While Matcher1.Find
        tmp_count = tmp_count + 1
    Loop
    Return tmp_count
End Sub

Private Sub btnTest_Click
    lblfoundchar.Text = "Start download"
    Sleep(0)
    DownloadAndCountQs("https://files.testfile.org/PDF/200MB-TESTFILE.ORG.pdf",1024000)
End Sub

 

[cklester]

@MicroDrie , just a heads up on that code! :-D

            'Count Q characters in this block
            Dim qCount As Int = CountChar(blockText, "R")

That should probably be a "Q".

 

[MicroDrie]

Good observation, but you can't count a letter Q if it's not in the download file. Choosing a different letter doesn't make much difference for a Proof Of Concept.

 

[cklester]

Good observation, but you can't count a letter Q if it's not in the download file. Choosing a different letter doesn't make much difference for a Proof Of Concept.

True. I just didn't want any newbs to trip up on thinking the way to find the "Q" for qCount was to look for the "R". :-D