B4A Library OCR - Extracting text from a bitmap using the Play Services Vision API

This request comes from here:
https://www.b4x.com/android/forum/threads/ocr-offline-on-screen.84867/#post-538774

The attached project extracts text from a bitmap that you can pass to the library making use of the Android Vision API.

Sample code:
B4X:
#Region  Project Attributes
    #ApplicationLabel: b4aMobileVisionBitmap
    #VersionCode: 1
    #VersionName:
    'SupportedOrientations possible values: unspecified, landscape or portrait.
    #SupportedOrientations: portrait
    #CanInstallToExternalStorage: False
#End Region


#Region  Activity Attributes
    #FullScreen: False
    #IncludeTitle: True
#End Region

Sub Process_Globals
    'These global variables will be declared once when the application starts.
    'These variables can be accessed from all modules.

End Sub

Sub Globals
    'These global variables will be redeclared each time the activity is created.
    'These variables can only be accessed from this module.
 
    Dim mvbm As MobileVisionBitmap

    Private Button1 As Button
    Private iv1 As ImageView
    Private bm As Bitmap
End Sub

Sub Activity_Create(FirstTime As Boolean)
    'Do not forget to load the layout file created with the visual designer. For example:
    Activity.LoadLayout("main")
    bm.Initialize(File.DirAssets,"saying1.png")   'saying.png and saying1.png are in the /Files folder of the B4A project
    iv1.Bitmap = bm
 
    mvbm.Initialize("mvbm")

End Sub

Sub Activity_Resume

End Sub

Sub Activity_Pause (UserClosed As Boolean)

End Sub


Sub Button1_Click
 
    mvbm.decodeBitmap(bm) 
    bm = Null
    bm.Initialize(File.DirAssets,"saying1.png")   'saying.png and saying1.png are in the /Files folder of the B4A project
    iv1.Bitmap = bm
 
End Sub

Sub mvbm_blocks_result(blocks As String)
 
    Log("B4A Blocks = " & blocks)
 
End Sub

Sub mvbm_lines_result(lines As String)
 
    Log("B4A Lines = " & lines)
 
End Sub

Sub mvbm_words_result(words As String)
 
    Log("B4A Words = " & words)
 
End Sub

Sub mvbm_error_result(error As String)
 
    Log("B4A ERROR = " & error)
 
End Sub

Take note of the B4A manifest file:
B4X:
'This code will be applied to the manifest file during compilation.
'You do not need to modify it in most cases.
'See this link for for more information: https://www.b4x.com/forum/showthread.php?p=78136
AddManifestText(
<uses-sdk android:minSdkVersion="5" android:targetSdkVersion="22"/>
<supports-screens android:largeScreens="true"
    android:normalScreens="true"
    android:smallScreens="true"
    android:anyDensity="true"/>)
SetApplicationAttribute(android:icon, "@drawable/icon")
SetApplicationAttribute(android:label, "$LABEL$")
'End of default text.

AddApplicationText(<meta-data
            android:name="com.google.android.gms.version"
            android:value="@integer/google_play_services_version" />
        <meta-data
            android:name="com.google.android.gms.vision.DEPENDENCIES"
            android:value="ocr" />)

If you don't have the Android Vision dependencies installed then you will need an initial internet connection for the dependencies to be installed. After that an internet connection will not be required.

This is version 1.00. It will expire on 31 October 2017.


Bitmap loaded into the imageview.
1.png



B4A Log when clicking on button Read Text
2.png
 

Attachments

  • MobileVisionBitmap.xml
    1.2 KB · Views: 1,654
  • MobileVisionBitmap.jar
    10 KB · Views: 790
  • b4aMobileVisionBitmap.zip
    130 KB · Views: 846
Last edited:

Johan Schoeman

Expert
Licensed User
Longtime User

Attachments

  • MobileVisionBitmap.jar
    2.8 KB · Views: 530
  • MobileVisionBitmap.xml
    1.7 KB · Views: 509
  • b4aMobileVisionBitmap.zip
    235.7 KB · Views: 527

ThRuST

Well-Known Member
Licensed User
Longtime User
:);):cool::p:D:rolleyes: Johan Schoeman wooohoo I must try this. You're the king of libraries. Monster can feel embarrassed hahahaha :))))))))))))))))))
 

Johan Schoeman

Expert
Licensed User
Longtime User
:);):cool::p:D:rolleyes: Johan Schoeman wooohoo I must try this. You're the king of libraries. Monster can feel embarrassed hahahaha :))))))))))))))))))
@ThRuST, this is not about embarrassing anyone on the forum. I have corresponded with @moster67 for a very long time and he has my utmost respect. I respect his assistance when I PM'ed him for some assistance, solutions that he provided me with, and his valuable contributions to the forum - more than what I have ever or will ever contribute.
 

ThRuST

Well-Known Member
Licensed User
Longtime User
You're right. We must all stay together and be friends, there's no other solution. No more wars in the world ever, we have learned our lessons from history everyone.
Your library looks awesome and I need it for my new project. I found a similar solution for B4i that uses images. Scanning data is the future, we can make a mark in the universe all of us and should better remain as brothers and heros of our generation. At least I am working on that, in case you didn't know me :) Enjoy scanning.
 

ThRuST

Well-Known Member
Licensed User
Longtime User
Dear @Johan Schoeman , would you please rename files with "v11" or "v1.1" at the end to prevent multiple versions spreading that will be confused with your first post, since they use the same filenames. Then you can move them to the first post as it will be convenient to find library updates in first post. Just my best two cents.
 
Last edited:

Douglas Farias

Expert
Licensed User
Longtime User
Hi @Johan Schoeman .
how are you?
do you plan update this lib? i m looking for a lib like this, only ocr from bitmap.

i m testing your lib, this is working fine, but only a question.
its possible add isOperational event? i think need some time to download on the first time right?

thank you.
 

Johan Schoeman

Expert
Licensed User
Longtime User
Hi @Johan Schoeman .
how are you?
do you plan update this lib? i m looking for a lib like this, only ocr from bitmap.

i m testing your lib, this is working fine, but only a question.
its possible add isOperational event? i think need some time to download on the first time right?

thank you.
Hello Douglas

Long time since I have last looked at this lib. Will see if I can revisit it sometime this weekend to add the isOperational event.

JS
 

Johan Schoeman

Expert
Licensed User
Longtime User
Hi @Johan Schoeman .
how are you?
do you plan update this lib? i m looking for a lib like this, only ocr from bitmap.

i m testing your lib, this is working fine, but only a question.
its possible add isOperational event? i think need some time to download on the first time right?

thank you.
Douglas, the event should already be there:

B4X:
                if (detector.isOperational() && bitmap != null) {
                    //BA.Log("detector is operational");
                    Frame frame = new Frame.Builder().setBitmap(bitmap).build();
                    SparseArray<TextBlock> textBlocks = detector.detect(frame);
                    String blocks = "";
                    String lines = "";
                    String words = "";
                    for (int index = 0; index < textBlocks.size(); index++) {
                        //extract scanned text blocks here
                        TextBlock tBlock = textBlocks.valueAt(index);
                        blocks = blocks + tBlock.getValue() + "\n" + "\n";
                        for (Text line : tBlock.getComponents()) {
                            //extract scanned text lines here
                            lines = lines + line.getValue() + "\n";
                            for (Text element : line.getComponents()) {
                                //extract scanned text words here
                                words = words + element.getValue() + ", ";

                            }
                        }
                    }
                    if (textBlocks.size() == 0) {
                        if (ba.subExists(eventName + "_error_result")) {
                            ba.raiseEventFromDifferentThread(ba.applicationContext, null, 0, eventName + "_error_result", true, new Object[]
                                            {"Nothing Found!"});
                        }           

                    } else {
                        if (ba.subExists(eventName + "_blocks_result")) {
                            ba.raiseEventFromDifferentThread(ba.applicationContext, null, 0, eventName + "_blocks_result", true, new Object[]
                                            {blocks});
                        }                           
                        
                        if (ba.subExists(eventName + "_lines_result")) {
                            ba.raiseEventFromDifferentThread(ba.applicationContext, null, 0, eventName + "_lines_result", true, new Object[]
                                            {lines});
                        }                                                   
                        
                        if (ba.subExists(eventName + "_words_result")) {
                            ba.raiseEventFromDifferentThread(ba.applicationContext, null, 0, eventName + "_words_result", true, new Object[]
                                            {words});
                        }                           
                    }
                } else {
                    if (ba.subExists(eventName + "_error_result")) {
                        ba.raiseEventFromDifferentThread(ba.applicationContext, null, 0, eventName + "_error_result", true, new Object[]
                                        {"Could not set up the detector!"});
                    }                                   
                    
                }

In B4A example that I have posted:
B4X:
Sub mvbm_error_result(error As String)
    
    Log("B4A ERROR = " & error)
    
End Sub

If detector is not there the event should be raised with "error As String" => Could not set up the detector!
 

roberto64

Active Member
Licensed User
Longtime User
Hi, when I launch the example this error
B4A ERROR = Could not set up the detector!
 

Johan Schoeman

Expert
Licensed User
Longtime User
Hi, when I launch the example this error
B4A ERROR = Could not set up the detector!
Your device had not downloaded and installed the dependencies. It should happen automatically but it sometimes takes a while.
 

roberto64

Active Member
Licensed User
Longtime User
Hi Johan, it always gives me the same error, I'm using an android ver.6.0.1, or given the permissions "rp.CheckAndRequest (rp.PERMISSION_CAMERA)" e rp.CheckAndRequest (rp.PERMISSION_WRITE_EXTERNAL_STORAGE) "but it always gives me a mistake" B4A ERROR = Could not set up "the detector! Instead with the version of android 8.0 and everything is ok, because in the version androi 6.01 I get the error?
 

Johan Schoeman

Expert
Licensed User
Longtime User
Hi Johan, it always gives me the same error, I'm using an android ver.6.0.1, or given the permissions "rp.CheckAndRequest (rp.PERMISSION_CAMERA)" e rp.CheckAndRequest (rp.PERMISSION_WRITE_EXTERNAL_STORAGE) "but it always gives me a mistake" B4A ERROR = Could not set up "the detector! Instead with the version of android 8.0 and everything is ok, because in the version androi 6.01 I get the error?
It checks for this before it raises the error that you get:

B4X:
if (detector.isOperational() && bitmap != null) {

I don't have an Android 6.0.1 device to test it on.
 

roberto64

Active Member
Licensed User
Longtime User
I have wounded and education is there.

B4X:
bitmap = getResizedBitmap(bitmap);
                if (detector.isOperational() && bitmap != null) {
                    //BA.Log("detector is operational");
 

Johan Schoeman

Expert
Licensed User
Longtime User
I have wounded and education is there.

B4X:
bitmap = getResizedBitmap(bitmap);
                if (detector.isOperational() && bitmap != null) {
                    //BA.Log("detector is operational");
Not sure I am following what you are saying....? Can you maybe explain?
 

roberto64

Active Member
Licensed User
Longtime User
Sorry for the English I'm doing with the translator, you asked me if in the source code of your library to insert an error event?
 

Johan Schoeman

Expert
Licensed User
Longtime User
Sorry for the English I'm doing with the translator, you asked me if in the source code of your library to insert an error event?
The error event is already in the library. That is why you get:
B4X:
Could not set up the detector!

The error is raised by the library because either the detector = null or bitmap = null (as far as the library is concerned).
 

roberto64

Active Member
Licensed User
Longtime User
I think the library restores the error already discussed, my thoughts and that in the 8.1 huawei version it works correctly by executing the same example, instead with another smatphone with android 6.0.1 version the error already described occurs, I updated the SDK but had no result.
thank you
 

Johan Schoeman

Expert
Licensed User
Longtime User
Working code for B4A V13, targetSDK 34 and on an Android 13 device:

Android 13 (SDK 34) working code:
#Region  Project Attributes
    #ApplicationLabel: b4aMobileVisionBitmap
    #VersionCode: 1
    #VersionName:
    'SupportedOrientations possible values: unspecified, landscape or portrait.
    #SupportedOrientations: portrait
    #CanInstallToExternalStorage: False
#End Region

#AdditionalJar: com.google.android.gms:play-services-code-scanner
#AdditionalJar: kotlin-stdlib-1.6.10
#AdditionalJar: androidx.arch.core:core-runtime
#AdditionalJar: androidx.resourceinspection:resourceinspection-annotation
#AdditionalJar: androidx.transition:transition-ktx
#AdditionalJar: androidx.emoji2:emoji2


#Region  Activity Attributes
    #FullScreen: False
    #IncludeTitle: True
#End Region

Sub Process_Globals
    'These global variables will be declared once when the application starts.
    'These variables can be accessed from all modules.
    Dim nativeMe As JavaObject

End Sub

Sub Globals
    'These global variables will be redeclared each time the activity is created.
    'These variables can only be accessed from this module.
    
    Dim mvbm As MobileVisionBitmap

    Private Button1 As Button
    Private iv1 As ImageView
    Private bm As Bitmap
    Dim ttsu As TTSutilities

End Sub

Sub Activity_Create(FirstTime As Boolean)
    'Do not forget to load the layout file created with the visual designer. For example:
    Activity.LoadLayout("main")
    nativeMe.InitializeContext

    mvbm.Initialize("mvbm")
    mvbm.TargetWidth = 1000                  'play around with this value
    mvbm.TargetHeight = 1000                 'play around with this value
        
    ttsu.Initialize
    
    bm.Initialize(File.DirAssets,"saying1.png")   'saying.png and saying1.png are in the /Files folder of the B4A project
    bm = nativeMe.RunMethod("createBinaryImage", Array(bm))
'   
'    Dim Out As OutputStream
'    Out = File.OpenOutput(File.DirInternal, "Test.png", False)
'    bm.WriteToStream(Out, 100, "PNG")
'    Out.Close
'   
'    bm = Null
'   
'    bm.Initialize(File.DirInternal,"Test.png")
    iv1.Bitmap = bm

    

End Sub

Sub Activity_Resume

End Sub

Sub Activity_Pause (UserClosed As Boolean)

End Sub


Sub Button1_Click
    
    mvbm.decodeBitmap(bm)   
    bm = Null
    bm.Initialize(File.DirAssets,"saying1.png")   'saying.png and saying1.png are in the /Files folder of the B4A project
    iv1.Bitmap = bm
    
End Sub

Sub mvbm_blocks_result(blocks As String)
    
    Log("B4A Blocks = " & blocks)
    
End Sub

Sub mvbm_lines_result(lines As String)
    
    Log("B4A Lines = " & lines)
    ttsu.speakOut(lines)
    
End Sub

Sub mvbm_words_result(words As String)
    
    Log("B4A Words = " & words)

    
End Sub

Sub mvbm_error_result(error As String)
    
    Log("B4A ERROR = " & error)
    
End Sub

#IF Java

import android.graphics.Bitmap;
import android.graphics.Color;



public Bitmap createBinaryImage( Bitmap bm )
{

    Bitmap bmp = Bitmap.createBitmap(bm.getWidth(), bm.getWidth(), Bitmap.Config.RGB_565);

    int[] pixels = new int[bm.getWidth()*bm.getHeight()];
    bm.getPixels( pixels, 0, bm.getWidth(), 0, 0, bm.getWidth(), bm.getHeight() );
    int w = bm.getWidth();

    // Calculate overall lightness of image
    long gLightness = 0;
    int lLightness;
    int c;
    for ( int x = 0; x < bm.getWidth(); x++ )
    {
        for ( int y = 0; y < bm.getHeight(); y++ )
        {
            c = pixels[x+y*w];
            lLightness = ((c&0x00FF0000 )>>16) + ((c & 0x0000FF00 )>>8) + (c&0x000000FF);
            pixels[x+y*w] = lLightness;
            gLightness += lLightness;
        }
    }
    gLightness /= bm.getWidth() * bm.getHeight();
    gLightness = gLightness * 5 / 6;

    // Extract features
    boolean[][] binaryImage = new boolean[bm.getWidth()][bm.getHeight()];

    for ( int x = 0; x < bm.getWidth(); x++ ) {
        for ( int y = 0; y < bm.getHeight(); y++ ) {
            binaryImage[x][y] = pixels[x+y*w] <= gLightness;
            if (binaryImage[x][y] == true) {
                bmp.setPixel(x, y, Color.BLACK);
            } else {
                bmp.setPixel(x, y, Color.WHITE);
            }   
        }
    }   

    return bmp;
    
}


#End If
 
Top