B4J Question [SOLVED] Wait For Download more than one file at the same time

[Elric]

Hi guys!

I already read this: https://www.b4x.com/android/forum/threads/b4x-okhttputils2-with-wait-for.79345/ where there is a snippet to "How to download a list of resources, one by one".

My question is: how to wait for completing all downloads and then do something?

Something like the project attached but I get this error:

Waiting for debugger to connect...
Program started.
Call B4XPages.GetManager.LogEvents = True to enable logging B4XPages events.
Start downloading
Error occurred on line: 56 (cDownloadAndExtract)
java.lang.ClassCastException: class anywheresoftware.b4a.keywords.Common$ResumableSubWrapper cannot be cast to class anywheresoftware.b4a.BA$ResumableSub (anywheresoftware.b4a.keywords.Common$ResumableSubWrapper and anywheresoftware.b4a.BA$ResumableSub are in unnamed module of loader 'app')
    at b4j.example.cdownloadandextract$ResumableSub_DownloadingAllFiles.resume(cdownloadandextract.java:384)
    at b4j.example.cdownloadandextract._downloadingallfiles(cdownloadandextract.java:278)
    at b4j.example.cdownloadandextract$ResumableSub_DownloadAndExtract.resume(cdownloadandextract.java:148)
    at b4j.example.cdownloadandextract._downloadandextract(cdownloadandextract.java:109)
    at b4j.example.b4xmainpage._button1_click(b4xmainpage.java:70)
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
    at java.base/java.lang.reflect.Method.invoke(Method.java:578)
    at anywheresoftware.b4a.shell.Shell.runMethod(Shell.java:629)
    at anywheresoftware.b4a.shell.Shell.raiseEventImpl(Shell.java:234)
    at anywheresoftware.b4a.shell.Shell.raiseEvent(Shell.java:167)
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
    at java.base/java.lang.reflect.Method.invoke(Method.java:578)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:111)
    at anywheresoftware.b4a.shell.ShellBA.raiseEvent2(ShellBA.java:100)
    at anywheresoftware.b4a.BA$1.run(BA.java:236)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:457)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(PlatformImpl.java:456)
    at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:96)
    at javafx.graphics/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
    at javafx.graphics/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(WinApplication.java:184)
    at java.base/java.lang.Thread.run(Thread.java:1589)

So, the problem seems being in:

Private Sub DownloadingAllFiles(mylstURL As List) As ResumableSub
    Dim mylstTasks As List
    mylstTasks.Initialize
    For i = 0 To mylstURL.Size - 1
        Dim rs As ResumableSub = DownloadingEachFile(mylstURL.Get(i), i)
        mylstTasks.Add(rs)
    Next
  
    Dim myynAllSuccess As Boolean = True
  
    For Each rs As ResumableSub In mylstTasks
        Wait For (rs) Complete (success As Boolean)
        If Not(success) Then myynAllSuccess = False
    Next

    Return myynAllSuccess
End Sub

May someone help me to understand and get the wished result?

Thanks in advance!

Edit: the strange behavior is that if I put a breakpoint and go on step by step (i.e. F8 key), the error is not casted, but, however, I cannot see the logs of RangeDownload Sub and seems looping never reaching ExtractingAllZipFiles Sub.

 

[Daestrum]

Think you just need

Private Sub DownloadingAllFiles(mylstURL As List) As ResumableSub
    Dim mylstTasks As List
    mylstTasks.Initialize
    Dim myynAllSuccess As Boolean = True
    For i = 0 To mylstURL.Size - 1
        Dim rs As ResumableSub = DownloadingEachFile(mylstURL.Get(i), i)
        Wait For (rs) Complete (success As Boolean)
        If Not(success) Then myynAllSuccess = False
    Next

    Return myynAllSuccess
End Sub

 

[emexes]

Or if you still want to do the downloads simultaneously in parallel, then consider:

[B4X] Wait for multiple tasks to complete

A pattern such as this one can be used when you want to run multiple resumable subs concurrently and wait for all of them to complete before continuing: Dim status(1) As Int Test1(status) Test2(status) Test3(status) Do While status(0) < 3 Sleep(50) Loop...

www.b4x.com

I have a vague recollection that Wait For doesn't work if the task has already been completed, or maybe it was that the return value was lost: regardless, hence the use of a Sleep polling loop (as a poor man's handcrafted Wait For).

In your case, the download sub should store its Success result in the tracking array, and decrement a NumDownloadsActive status variable (which is an array so that it is passed by reference to the download sub so that the calling sub can see its changing value).

 

[Erel]

Or if you still want to do the downloads simultaneously in parallel, then consider:

I think that this is the best method to run multiple tasks concurrently and wait for all of them to complete.

And don't worry about the "sleep polling loop". This is not a busy-loop and it isn't resource intensive. It is similar to using a timer to check the state.

 

[Elric]

Thank you guys!

Not immediate... but I got it!

Here the important part of my code (the main changes of my previous attached project) to download simultaneously and in parallel more files and wait the end of downloads before start other tasks. I hope this may help someone!

...
Public Sub DownloadAndExtract
    PopulateURLList
    LogColor($"Start downloading"$, intMagenta)
    ' Calls the asynchronous DownloadingAllFiles subroutine and waits for it to complete.
'    Wait For (DownloadingAllFiles(lstURL)) Complete (myynDownloadSuccess As Boolean)
   
    Dim myintStatus(1) As Int
    Dim myynDownloadSuccess As Boolean = True
'    DownloadingAllFilesStatus(lstURL, myintStatus, myynDownloadSuccess)
    For i = 0 To lstURL.Size - 1    ' Loops through each URL in lstURL.
        DownloadingEachFile(lstURL.Get(i), i, myintStatus)
    Next
   
    Do While myintStatus(0) < lstURL.Size
        Sleep(50)
        Dim myIndex As Int
        If lstFileDownloadError.IndexOf($"Error"$) >= 0 Then
            myIndex = lstFileDownloadError.IndexOf($"Error"$)
            LogColor($"Error during download ${lstURL.Get(myIndex)}"$, intRed)
            myynDownloadSuccess = False
            Exit
        End If
    Loop
    If myynDownloadSuccess = True Then
        Log($"all tasks completed. Continue with your next task..."$)
    End If
    LogColor($"Downloads completed: ${myynDownloadSuccess}"$, intMagenta)

    ' Checks if the downloads were successful.
    If myynDownloadSuccess Then
        ' Here the other tasks or subs
    Else
        LogColor($"Download failed!"$, intRed)
    End If
End Sub

...

Private Sub DownloadingEachFile(mystrUrl As String, myIndex As Int, myintStatus() As Int)
    'download new app file from webserver and save it locally as temp file
    LogColor($"Start downloading ${File.GetName(mystrUrl)}"$, intMagenta)
    Dim myTracker As RangeDownloadTracker = CreateTracker 'Creates a new instance of the RangeDownloadTracker class to manage the download.
    myTracker.strUserName = strUserName 'Assigns the username for authentication to the tracker.
    myTracker.strPassword = strUPassword 'Assigns the password for authentication to the tracker.
   
    lstFileDownloadError.Add($""$)
   
    Dim myynSuccess As Boolean
    Try
        Wait For (RangeDownload(strSharedFolder, $"${File.GetName(mystrUrl)}.temp"$, mystrUrl, myTracker)) Complete (myynSuccess As Boolean)
    Catch
        Log($"Error during download: ${LastException}"$)
        mapDownloadSuccess.Put(myIndex, False)
        lstFileDownloadError.Set(myIndex, $"Error"$)
    End Try

    LogColor($"Downloading ${File.GetName(mystrUrl)}.temp complete, Success = ${myynSuccess}"$, intMagenta)

    If myynSuccess = True Then
        File.Copy(strSharedFolder, $"${File.GetName(mystrUrl)}.temp"$, strSharedFolder, File.GetName(mystrUrl))
        File.Delete(strSharedFolder, $"${File.GetName(mystrUrl)}.temp"$)
        lstFileDownloadError.Set(myIndex, $""$)
    Else
        Log($"Downloading ${File.GetName(mystrUrl)} Error: ${myTracker.strHErrorMessage}"$)
        Log($"Downloading ${File.GetName(mystrUrl)} StatusCode: ${myTracker.intHStatusCode}"$)
        Log($"Downloading ${File.GetName(mystrUrl)} Error: ${myTracker.strJErrorMessage}"$)
        Log($"Downloading ${File.GetName(mystrUrl)} StatusCode: ${myTracker.intJStatusCode}"$)
        lstFileDownloadError.Set(myIndex, $"Error"$)
    End If
    mapDownloadSuccess.Put(myIndex, myynSuccess)
   
    Log("DownloadingEachFileStatus completed")
    myintStatus(0) = myintStatus(0) + 1
End Sub