Android Question vision.TextRecognizer.Builder

drgottjr said:

i use the text recognizer (based on the above code) all the time.
i don't understand why erel says he didn't get valid results. i don't use pages (yet. a slow learning curve for me.), so my use is non-page and with his camex class).
in comparing the code above with the code in my app, the only difference i noticed has to do with frame orientation. as you probably know, ocr can't read anything tending towards upside down.
my case is, perhaps, special, because it involved a nexus 5 and its particular camera orientation. i also use the code on non-nexus 5 devices. in any case, below find my code relating to frame orientation:

B4X:

            Dim rotation As Int = camex.PreviewOrientation
            Log("rotation: " & rotation)
            Dim newrotation As Int
            '      .setRotation(metadata.getRotation())
            Select rotation
                Case 0
                    newrotation = 0
                Case 180
                    newrotation = 2
                Case 90
                    newrotation = 1
                Case 270
                    newrotation = 3
            End Select
          
            If newrotation <> 0 Then
                frameBuilder.RunMethod("setRotation",Array(newrotation))          ' 3 = 270
            End If

it goes just before passing the frame off to the detector

B4X:

Dim SparseArray As JavaObject = ocrdetector.RunMethod("detect", Array(frame))

to be clear, it doesn't matter what you think you see on the screen; the detector may see the same thing at 90 degrees (or more) offset and cannot detect the text.

if erel happens to visit this thread again, he might confirm whether or not he thinks frame orientation is relevant to his code. you, of course, are welcome to try the code yourself.

Click to expand...

Erel said:

I made some tests and didn't get valid results.

You can try it. Changes based on that example:

Add to CameraEx class:

B4X:

Public Sub SetPreviewFormat(Format As Int)
    r.target = parameters
    r.RunMethod2("setPreviewFormat", Format, "java.lang.int")
End Sub

Public Sub GetSupportedPreviewFormats As List
    r.target = parameters
    Dim list1 As List = r.RunMethod("getSupportedPreviewFormats")
    Return list1
End Sub

B4XMainPage:

B4X:

Private Sub StartCamera
    rp.CheckAndRequest(rp.PERMISSION_CAMERA)
    Wait For B4XPage_PermissionResult (Permission As String, Result As Boolean)
    If Result = False Then
        toast.Show("No permission!")
        Return
    End If
    StartCameraShared
    camEx.Initialize(pnlPreview, False, Me, "Camera1")
    Wait For Camera1_Ready (Success As Boolean)
    If Success Then
        camEx.SetContinuousAutoFocus
        Log(camEx.GetSupportedPreviewFormats) 'ignore
        camEx.SetPreviewFormat(17) 'NV21
        camEx.CommitParameters
        camEx.StartPreview
    Else
        toast.Show("Error opening camera")
        StopCamera
    End If
End Sub

Private Sub CreateTextDetector
    Dim ctxt As JavaObject
    ctxt.InitializeContext
    Dim builder As JavaObject
    builder.InitializeNewInstance("com/google/android/gms/vision/text/TextRecognizer.Builder".Replace("/", "."), Array(ctxt))
    detector = builder.RunMethod("build", Null)
    Dim operational As Boolean = detector.RunMethod("isOperational", Null)
    If operational = False Then
        toast.Show("Failed to create detector")
    End If
End Sub

Private Sub Camera1_Preview (data() As Byte)
    If DateTime.Now > LastPreview + IntervalBetweenPreviewsMs Then
        'Dim n As Long = DateTime.Now
        Dim frameBuilder As JavaObject
        Dim bb As JavaObject
        bb = bb.InitializeStatic("java.nio.ByteBuffer").RunMethod("wrap", Array(data))
        frameBuilder.InitializeNewInstance("com/google/android/gms/vision/Frame.Builder".Replace("/", "."), Null)
        Dim cs As CameraSize = camEx.GetPreviewSize
        frameBuilder.RunMethod("setImageData", Array(bb, cs.Width, cs.Height,  17))
        Dim frame As JavaObject = frameBuilder.RunMethod("build", Null)
        Dim SparseArray As JavaObject = detector.RunMethod("detect", Array(frame))
        LastPreview = DateTime.Now
        Dim Matches As Int = SparseArray.RunMethod("size", Null)
        For i = 0 To Matches - 1
            Dim text As JavaObject = SparseArray.RunMethod("valueAt", Array(i))
            Log(text.RunMethod("getValue", Null))
        Next
    End If
End Sub

And call CreateTextDetector in B4XPage_Created.

Click to expand...

drgottjr said:

i use the text recognizer (based on the above code) all the time.
i don't understand why erel says he didn't get valid results. i don't use pages (yet. a slow learning curve for me.), so my use is non-page and with his camex class).
in comparing the code above with the code in my app, the only difference i noticed has to do with frame orientation. as you probably know, ocr can't read anything tending towards upside down.
my case is, perhaps, special, because it involved a nexus 5 and its particular camera orientation. i also use the code on non-nexus 5 devices. in any case, below find my code relating to frame orientation:

B4X:

            Dim rotation As Int = camex.PreviewOrientation
            Log("rotation: " & rotation)
            Dim newrotation As Int
            '      .setRotation(metadata.getRotation())
            Select rotation
                Case 0
                    newrotation = 0
                Case 180
                    newrotation = 2
                Case 90
                    newrotation = 1
                Case 270
                    newrotation = 3
            End Select
          
            If newrotation <> 0 Then
                frameBuilder.RunMethod("setRotation",Array(newrotation))          ' 3 = 270
            End If

it goes just before passing the frame off to the detector

B4X:

Dim SparseArray As JavaObject = ocrdetector.RunMethod("detect", Array(frame))

to be clear, it doesn't matter what you think you see on the screen; the detector may see the same thing at 90 degrees (or more) offset and cannot detect the text.

if erel happens to visit this thread again, he might confirm whether or not he thinks frame orientation is relevant to his code. you, of course, are welcome to try the code yourself.

Click to expand...

arastoo said:

Hey guys

After the release of the program, I noticed user feedback

Numerous users have said they do not recognize texts


For example devices :

SAMSUNG Galaxy g6 android 10
SAMSUNG note 10 plus android 10
google oneplus 6 android 10
asua 4zen android 8.1


Huawei honor 8X android 10
SAMSUNG Galaxy A30 android 10
samsung galaxy note 8 android 9
Huawei Nova 3e android 9



After multiple controls on the real device

I noticed there is an error in this section [[ Failed to create detector ]]
How do I know why this is a problem?

Thank you for your guidance??

B4X:

Private Sub CreateDetector
    LogColor("Sub CreateDetector ",Colors.Magenta)

    Dim ctxt As JavaObject
    ctxt.InitializeContext
    Dim builder As JavaObject
   
    builder.InitializeNewInstance("com/google/android/gms/vision/text/TextRecognizer.Builder".Replace("/", "."), Array(ctxt))
    detector_java = builder.RunMethod("build", Null)
   
    Dim operational As Boolean = detector_java.RunMethod("isOperational", Null)
   
   
    If operational = False Then
        LogColor("Failed to create detector",Colors.Red)
        ToastMessageShow("Failed to create detector",True)
    Else
        LogColor("create detector",Colors.Green)
        ToastMessageShow("create detector",True)
    End If
   
   
End Sub

Click to expand...