iOS Question B4i MLKit BarcodeScanner – Random app termination during QR code recognition

[b4x-de]

Hello,

Based on examples found in the B4X forum that demonstrate MLKit’s Selfie Segmentation and Text Recognition features, I have been experimenting with integrating MLKit’s BarcodeScanner into a B4i project.

In my implementation, I use the low-level camera (LLCamera), which raises a Preview event that provides each frame as a Bitmap. Each bitmap is passed to MLKit’s BarcodeScanner for QR code detection.

The current implementation generally works and correctly detects QR codes, but under certain runtime conditions the app terminates randomly and without any visible exception or error. There is no fixed pattern or reproducible frame count. Sometimes it runs for a while, sometimes it crashes after only a few frames. Logs show no diagnostic entry at all.

This suggests that either a memory leak or a race condition occurs somewhere inside MLKit’s asynchronous processing chain. To mitigate possible concurrency issues, I have already tried several synchronization strategies:

• serializing image processing calls with semaphores,
• adding short delays (Sleep) between frames,
• ensuring that only one recognition task runs at a time, and
• explicitly releasing temporary Bitmaps and invoking garbage collection after each scan.

None of these measures has fully prevented the random termination.

I would appreciate any insight on how to correctly manage memory and threading when repeatedly calling processImage:completion: from the preview stream, or whether MLKit requires a different approach to safely reuse the same BarcodeScanner instance for continuous frame analysis.

Thank you in advance for any analysis or advice!

Update 2025/11/03:
Below is the current version of my test class (QRCodeScanner.bas). The complete example project is attached.
Please find the last version of QRCodeScanner.bas and my example projekt in the post below.

I am aware that there already exists a working QR code scanning implementation for B4i. However, my goal here is different: I am not looking for another ready-made solution, but rather to understand how to make this specific MLKit-based integration run reliably and correctly. I am interested in understanding the internal cause of this random termination and in achieving a stable implementation using MLKit.

 

[Erel]

Check the device logs: https://www.b4x.com/android/forum/threads/tip-on-device-logs.156056/#content

 

[b4x-de]

I checked the system logs on the iPhone as Erel suggested. It is now clear that this is not a threading or race condition problem. It is a memory leak.

The memory footprint of the app keeps growing all the time. When it reaches about 2 GB, iOS stops the app automatically. There is still no crash or error in the B4i logs. But the system log shows a JetsamEvent. This means the app was killed by iOS because of memory pressure.

I found these lines in log:

"largestProcess" : "LLCameraQrCode",

{
    "uuid" : "ec53937c-47f9-3d83-984f-d1dbeb94b029",
    "states" : [
      "frontmost"
    ],
    "killDelta" : 3845,
    "genCount" : 0,

    "reason" : "per-process-limit",
    "physicalPages" : {
      "internal" : [
        38064,
        89470
      ]
    },
    "freeze_skip_reason:" : "none",
    "pid" : 13697,
    "cpuTime" : 39.264316999999998,
    "name" : "LLCameraQrCode",
    "lifetimeMax" : 134272
  }

I have now updated my project and tried to make it as memory-efficient as possible. I carefully reviewed the code and made sure that all memory objects I allocate are released again. I also wrote a small module to monitor the memory usage of the app in real time.

Application_Start
Start at: 13:02:43
Application_Active
Process preview frame no. 1 at: 13:02:44
...
Process preview frame no. 22 at: 13:02:46
[MEM] App footprint: 127.94 MB | System free: 1013.44 MB
...
Process preview frame no. 275 at: 13:02:55
[MEM] App footprint: 154.67 MB | System free: 969.59 MB
...
Process preview frame no. 441 at: 13:03:01
[MEM] App footprint: 170.56 MB | System free: 958.22 MB
...
Process preview frame no. 926 at: 13:03:20
[MEM] App footprint: 215.11 MB | System free: 970.81 MB
...
Process preview frame no. 5461 at: 13:05:54
[MEM] App footprint: 936.56 MB | System free: 867.36 MB
...

Even with these changes, the log output still shows that the memory footprint keeps increasing after every call. At some point, the system memory limit is reached and the app is stopped again because of memory pressure.

At this point, I am honestly out of ideas. I cannot find anything else in my code that could cause this behavior. So I am posting the latest version of the project here in case it might be useful for someone else who wants to investigate further.

My current conclusion is that this memory leak comes from inside ML Kit version 3 itself.

It seems that the framework keeps some internal references or cached objects that are never released when using continuous image processing. Because of that, ML Kit version 3 seems not suitable for continuous QR code scanning in its current state.

Maybe this will change with a future version of ML Kit. If that happens, I will return to this topic and try again with a newer release to see if the problem has been fixed.

A small note: I am aware that newer versions of ML Kit exist, but unfortunately I am not able to integrate those frameworks into my local B4i builder setup.
So for now, this is where my work on this experiment stops.

'QRCodeScanner.bas
Sub Class_Globals
    ' CornerPoints clockwise starting from top left to bottom left
    Type QRCodeScannerResult (Success As Boolean, Codes As List, ErrorMessage As String)
    Type QRCode(Text As String, CornerPoints(4) As QRCodePoint)
    Type QRCodePoint (x As Float, y As Float)
   
    Private mnoNativMe As NativeObject
    Private scanner As NativeObject
   
    Private Const MLKBarcodeFormatQRCode As Int = 256   ' 0x0100
   
End Sub

Public Sub Initialize
    mnoNativMe = Me

    Dim options As NativeObject
    'options = options.Initialize("MLKBarcodeScannerOptions").RunMethod("new", Null)
    options = options.Initialize("MLKBarcodeScannerOptions").RunMethod("alloc", Null).RunMethod("initWithFormats:", Array(MLKBarcodeFormatQRCode))

    Dim cls As NativeObject
    cls.Initialize("MLKBarcodeScanner")

    Try
        scanner = cls.RunMethod("barcodeScannerWithOptions:", Array(options))
    Catch
        ' Fallback
        scanner = cls.RunMethod("barcodeScanner", Null)
    End Try
End Sub


Public Sub Finalize
    Log("QRCodeScanner Finalize called")
    Try
        If scanner.IsInitialized Then
            Dim no As NativeObject = Me
            no.RunMethod("closeScanner", Null)
            'scanner = Null
        End If
        mnoNativMe = Null
    Catch
        Log($"Finalize error: ${LastException.Message}"$)
    End Try
End Sub


#If OBJC
- (void)closeScanner {
    @autoreleasepool {
        // 1. Laufende Dispatch-Tasks abbrechen
        if (mlkitQueue != nil) {
            dispatch_sync(mlkitQueue, ^{ });
        }

        // 2. Alle noch gehaltenen Bilder entfernen
        if (inflightImages != nil) {
            [inflightImages removeAllObjects];
            inflightImages = nil;
        }

        // 3. Barcode-Scanner-Objekt freigeben (wenn du eines hältst)
        self->__scanner = nil;

        // 4. Optional: URL-Cache & CoreAnimation flushen
        [[NSURLCache sharedURLCache] removeAllCachedResponses];
        [CATransaction flush];

        NSLog(@"[QRCodeScanner] closeScanner: resources released");
    }
}
#End If

Public Sub Recognize(bmp As Bitmap) As ResumableSub
    Dim res As QRCodeScannerResult
   
    Dim code As NativeObject
    Dim v As NativeObject
    Dim cg As NativeObject
    Dim points As NativeObject
   
    Dim fmt As Int
    Dim n As Int
   
    Dim xy() As Float
    Dim px As Float
    Dim py As Float
   
    res.Initialize
    res.Codes.Initialize
   
    'Log("Start Recognize")
   
    Dim no As NativeObject = Me
    no.RunMethod("processImage:withScanner:", Array(ImageToMLImage(bmp), scanner))

    Wait For Process_Result(Success As Boolean, Barcodes As Object)
   
    bmp = Null
   
    res.Success = Success

    If Success Then
       
        For Each code As NativeObject In Barcodes
            fmt = code.GetField("format").AsNumber
            If fmt = MLKBarcodeFormatQRCode Then
                Dim resCode As QRCode
                resCode.Initialize
               
                resCode.Text = code.GetField("displayValue").AsString   ' alternativ: rawValue
                points = code.GetField("cornerPoints")
                If points <> Null And points.IsInitialized Then
                    n = points.RunMethod("count", Null).AsNumber

                    For j = 0 To n - 1
                        v = points.RunMethod("objectAtIndex:", Array(j)) ' NSValue<CGPoint>
                        cg = v.RunMethod("CGPointValue", Null)            ' -> NSData (struct)
                        xy = mnoNativMe.ArrayFromPoint(cg)                             ' -> [x,y]
                        px = xy(0)
                        py = xy(1)
                        resCode.CornerPoints(j).x = px
                        resCode.CornerPoints(j).y = py
                    Next
                End If
                res.Codes.Add(resCode)
            End If
        Next
       
        code = Null
        v = Null
        cg = Null
        points = Null
    End If

    'Log("End Recognize")
    Return res
End Sub

' Converts B4i bitmap to MLKit VisionImage
Private Sub ImageToMLImage(bmp As Bitmap) As NativeObject
    Dim image As NativeObject
    Return image.Initialize("MLKVisionImage").RunMethod("alloc", Null).RunMethod("initWithImage:", Array(bmp))
End Sub

#If OBJC
#import <MLKitVision/MLKitVision.h>
#import <MLKitBarcodeScanning/MLKitBarcodeScanning.h>

static dispatch_queue_t mlkitQueue;
static NSMutableArray<MLKVisionImage*> *inflightImages;

- (dispatch_queue_t)mlkitQueue {
    if (mlkitQueue == nil) {
        mlkitQueue = dispatch_queue_create("com.example.qr.mlkit.serial", DISPATCH_QUEUE_SERIAL);
    }
    return mlkitQueue;
}

- (NSMutableArray *)inflightImages {
    if (inflightImages == nil) {
        inflightImages = [NSMutableArray new];
    }
    return inflightImages;
}

- (void)processImage:(MLKVisionImage*)image withScanner:(MLKBarcodeScanner*)scanner {
    __weak typeof(self) weakSelf = self;
    NSMutableArray *strongInflight = [weakSelf inflightImages];
    dispatch_queue_t queue = [weakSelf mlkitQueue];

    [strongInflight addObject:image];

    dispatch_async(queue, ^{
        @autoreleasepool {
            [scanner processImage:image
                       completion:^(NSArray<MLKBarcode *> * _Nullable barcodes,
                                    NSError * _Nullable error) {
                @autoreleasepool {
                    __strong typeof(self) strongSelf = weakSelf;
                    if (!strongSelf) return;

                    [[strongSelf inflightImages] removeObject:image];

                    BOOL ok = (error == nil && barcodes != nil);
                    id payload = ok ? ({ B4IList *lst = [B4IList new]; lst.object = barcodes; lst; }) : [NSNull null];

                    if (!ok && error) {
                        NSLog(@"MLKit barcode error: %@", error.localizedDescription);
                    }

                    dispatch_async(dispatch_get_main_queue(), ^{
                        __strong typeof(self) strongSelf2 = weakSelf;
                        if (!strongSelf2) return;

                        [strongSelf2.bi raiseUIEvent:nil
                                              event:@"process_result::"
                                             params:@[@(ok), payload]];
                    });
                }
            }];
        }
    });
}
#End If

'CameraTest.bas
Sub Class_Globals
    Private llc As LLCamera


    Private PreviewPanel As Panel

    Private frontCamera As Boolean = False

    Private scanner As QrCodeScanner
    Private busy As Boolean
    Private overdue As Boolean
    Private previewNum As Int
    Private previewLast As Long
    
    Private CallBack As Object
    Private Eventname As String
    
    Private memMon As MemoryMonitor

End Sub

Public Sub Initialize(pCallBack As Object, pstrEventname As String, pnl As Panel)
    CallBack = pCallBack
    Eventname = pstrEventname
    PreviewPanel = pnl
    
    InitializeCamera(frontCamera)

    scanner.Initialize
    
    busy = False
    overdue = False
    previewNum = 0
    previewLast = DateTime.Now
    
    memMon.Initialize(True)   ' True = AutoStart alle 2 Sekunden
    Log("Start at: " & DateTime.Time(DateTime.Now))
End Sub

Sub InitializeCamera(front As Boolean)
    If llc.IsInitialized Then
        llc.StopPreview
    End If
    
    llc.Initialize(PreviewPanel, "llc", front)
    llc.StartPreview

    ConfigureCamera
End Sub

Private Sub ConfigureCamera
    Try
        llc.BeginConfiguration
        llc.FlashMode = llc.FLASH_AUTO
        llc.PreserveRatio = True
        llc.CommitConfiguration
    Catch
        Log("Error configuring camera: " & LastException.Description)
    End Try
End Sub

Private Sub Page1_Resize(Width As Int, Height As Int)
    llc.Resize
End Sub

Sub llc_Preview(Image As Bitmap)
    
    'Log($"Preview no. ${previewNum} at: ${DateTime.Time(DateTime.Now)}"$)
    previewNum = previewNum + 1

    If previewNum Mod 500 = 0 Then
        overdue = True
    End If
    
    Dim lngNow As Long = DateTime.Now
    If lngNow - previewLast < 250 Then
        'Log("Too early")
        llc.ReleaseFrame(Image)
        Return
    End If
    
    previewLast = lngNow
    Log($"Process preview frame no. ${previewNum} at: ${DateTime.Time(DateTime.Now)}"$)
    
    If Not(busy) Then
        busy = True
        If overdue Then
            resetScanner(False)
        End If
        Dim bmp As Bitmap = Image.Resize(Image.Width / 2, Image.Height / 2, True)
        llc.ReleaseFrame(Image)
        Image = Null
        processBitmap(bmp)
    Else
        Log("Still busy")
        llc.ReleaseFrame(Image)
    End If
    
End Sub

Sub resetScanner(hard As Boolean)
    Log("Before Reset scanner")
    memMon.LogCurrentUsage
    scanner.Finalize
    If hard Then
        scanner = Null
        Dim s As QrCodeScanner
        s.Initialize
        scanner = s
    Else
        scanner.Initialize
    End If
    Log("After Reset scanner")
    memMon.LogCurrentUsage
    overdue = False
End Sub

Sub processBitmap(pBmp As Bitmap)
    Wait For (scanner.Recognize(pBmp)) Complete (Result As QRCodeScannerResult)
    pBmp = Null
    If Result.Success And Result.Codes.Size > 0 Then
        CallSub2(CallBack, Eventname & "_Found", Result)
    End If
    busy = False
End Sub

 

[Erel]

A small note: I am aware that newer versions of ML Kit exist, but unfortunately I am not able to integrate those frameworks into my local B4i builder setup.

This will be simpler with the next version, soon to be released.