Android Tutorial Large & Searchable List with SearchView + B4XSerializator

Better to use: https://www.b4x.com/android/forum/threads/b4x-sqlsearchview-sqlite-based-search-view.133379/

Two very important methods were added to B4XSerializator in RandomAccessFile v2.20:
ConvertObjectToBytesAsync and ConvertBytesToObjectAsync.

With these methods you can asynchronously convert objects to bytes and vice versa. This allows doing things that previously were not possible.
RDC2 for example is based on these methods: https://www.b4x.com/android/forum/threads/61801/#content

SearchView was introduced in 2012. SearchView shows a list with an EditText that acts as a filter.
It uses an in-memory index to find the matching items. SearchView is very useful, however it was limited to about 500 items as it takes time to build the index and the UI is frozen until the index is ready (the main thread is busy building the index).

Now with B4XSerializator and a small B4J program we can easily load 10,000 items or more to SearchView.
The B4J program will create the index and save it into two files. The first file holds the items list and the prefix based index
The second file holds the secondary index (full text index) which is much larger.

In the B4A program we will first load the smaller index and then the larger index (they were added to the Files tab). Both are loaded asynchronously so the UI is not affected.
Until the second index is loaded the user will only be able to search for items based on the prefix. It is highly unlikely that the user will notice this at all.

upload_2015-12-31_15-53-41.png



Why are the new async methods so important here?

On a Nexus 5 it takes 1.5 seconds until the first index is ready and another 5-6 seconds for the second index.
As everything is done in the background, the UI is fully responsive while the data is loaded.
If we were using the non-async methods then the UI would have been frozen for 7.5+ seconds. This is a very bad UI and is also likely to be killed by the OS.

Notes

- This solution is also compatible with B4i as B4XSerializator is cross platform (with some changes to work with iTableView).
- The SearchView class is a modified version.
- The data in this example is Alexa's top 10,000 sites.


I'm sure that going forward we will see more and more usages for B4XSerializator.
 

Attachments

  • B4J_IndexBuilder.zip
    57.9 KB · Views: 1,701
  • LargeSearchView.zip
    733.6 KB · Views: 2,812
Last edited:

shashkiranr

Active Member
Licensed User
Longtime User
Hi Erel,

If we want to use this searchview with data coming from the server db, then should we install the b4j program in the server,get the first and last index files an use that in the android program?

Or is there a better way to do it? I am using the searchview to search for names downloaded from a mysql db. How can i implement this?

Best,
SK
 

Erel

B4X founder
Staff member
Licensed User
Longtime User
If we want to use this searchview with data coming from the server db, then should we install the b4j program in the server,get the first and last index files an use that in the android program?

Or is there a better way to do it?
The best way way is to implement your server with B4J and then convert the data inside the server code. Another option is to create a small B4J program and call it from your server.
 

Mahares

Expert
Licensed User
Longtime User
Please clarify this brief tutorial by @Erel for me. Based on my understanding of this tutorial, here is the procedure I implemented to use this SearchView from a SQLite database:
a. Created a database on the PC via B4J and inserted records.
b. Copied all records to a list via B4J that I subsequently used to create the two .dat index files.
c. Next, copied the 2 files manually to the asset folder of the B4A application and started using the B4A app.

My questions are:
1. Are the above steps correct? It appears that there are too many steps just to get to the index. Is there a better way?
2. What happens if the database from which the two .dat files are created change frequently? Do I have to go through the same lengthy process outlined above.

Thank you
 

Mahares

Expert
Licensed User
Longtime User
All three steps could be executed by the same B4J program.
How would you copy the 2 .dat files programmatically from B4J app folder to the files (assets) folder of the B4A program?
Thank you.
 

Erel

B4X founder
Staff member
Licensed User
Longtime User
Next, copied the 2 files manually to the asset folder of the B4A application and started using the B4A app.
This is equivalent to using File.Copy from the B4J program.

My guess is that you want to update the database at runtime. In that case you need to implement the same solution inside the B4J server.
 

Mahares

Expert
Licensed User
Longtime User
This is equivalent to using File.Copy from the B4J program.

Are we talking about something as simple as this, which seems to work?
B4X:
File.Copy(File.DirApp,"second.dat","C:\Temp\Junk\B4JindexBuilder\Searchview\Files","second.dat")
File.Copy(File.DirApp,"first.dat","C:\Temp\Junk\B4JindexBuilder\Searchview\Files","first.dat")
 

Roberto P.

Well-Known Member
Licensed User
Longtime User
The best way way is to implement your server with B4J and then convert the data inside the server code. Another option is to create a small B4J program and call it from your server.


Erel, would be very important to your example, certainly well done and optimized, server-side with b4j to provide the real-time data, such as a web server, to the list.
Thank you in advance.
 

Erel

B4X founder
Staff member
Licensed User
Longtime User
This server code will convert the index to an array of bytes:
B4X:
Dim search As SearchView
search.Initialize
Dim obj() As Object = search.SetItems(dataToIndex)
Dim ser As B4XSerializator
Dim DataToSend() As Byte = ser.ConvertObjectToBytes(obj)

Now you should send DataToSend to the client.

On the client you should read it with ser.ConvertBytesToObjectAsync and set it with this code:
B4X:
Private Sub ser_BytesToObject (Success As Boolean, NewObject As Object)
   If Success Then
     Dim obj() As Object = NewObject
     firstIndex = Array (obj(0), obj(1))
     secondIndex = obj(2)
     sv.LoadFirst(firstIndex)
     sv.LoadSecont(secondIndex)
   End If
End Sub

In this case, where the data comes from a remote server, it is less important to split the task as done in first post example.
 

Anser

Well-Known Member
Licensed User
Longtime User
I avoided the use of SearchView because I face the problem specified in the first post of this thread ie "The searchView takes time to build the index and the UI is frozen until the index is ready (the main thread is busy building the index)."

I would like to use this new faster SearchView in my B4A app so that my UI doesn't freeze.

My Challenge
I am using jRDC2. The data in the SearchView has to come from a Remote MySQL DB via jRDC2 and not from a file that resides in the File.DirAssets Folder

I may be really dump, I can't understand where to start to get this done.

Do I need to run the B4J app (B4J_IndexBuilder.zip) attached in the First post of this thread on my VPS Server ? I mean the same server where my jRDC2 is running. ? If it is to run on the same VPS server, then on which port should it run ?. How do I call this from my B4A app.

As I already said, my data comes from a remote MySQL DB.

So far I understand that in the B4J app. I should make the following changes
B4X:
'Non-UI application (console / server application)
#Region  Project Attributes
    #CommandLineArgs:
    #MergeLibraries: True
#End Region

Sub Process_Globals

End Sub

Sub AppStart (Args() As String)

    Dim dataToIndex As List = File.ReadList(File.DirAssets, "sites.txt")
    Dim search As SearchView
    search.Initialize
    Dim obj() As Object = search.SetItems(dataToIndex)
    Dim ser As B4XSerializator
    ' The following lines commented by Anser
    'WriteFile("first.dat", ser.ConvertObjectToBytes(Array(obj(0), obj(1))))
    'WriteFile("second.dat", ser.ConvertObjectToBytes(obj(2)))

    'Added the following line as per Erel's post
    Dim DataToSend() As Byte = ser.ConvertObjectToBytes(obj)
End Sub

'May be the following Sub is not required at all
Sub WriteFile(FileName As String, b() As Byte)
    Dim out As OutputStream = File.OpenOutput(File.DirApp, FileName, False)
    out.WriteBytes(b, 0, b.Length)
    out.Close
End Sub

And in the B4A example (LargeSearchView.zip) available in the fisrt post of this thread, the following Sub
B4X:
Private Sub ser_BytesToObject (Success As Boolean, NewObject As Object)
    ProgressBar1.Visible = False
    If Success Then
        Dim ser As B4XSerializator = Sender
        Log($"BytesToObject: ${ser.Tag}"$)
        If ser.Tag = 1 Then
            firstIndex = NewObject
            sv.LoadFirst(firstIndex)
            ser.Tag = 2
            'read the secondary index
            ser.ConvertBytesToObjectAsync( _
                Bit.InputStreamToBytes(File.OpenInput(File.DirAssets, "second.dat")), "ser")
        Else if ser.Tag = 2 Then
            secondIndex = NewObject
            sv.LoadSecond(secondIndex)
        End If
    End If
End Sub

The above Sub should be changed as follows
B4X:
Private Sub ser_BytesToObject (Success As Boolean, NewObject As Object)
   If Success Then
     Dim obj() As Object = NewObject
     firstIndex = Array (obj(0), obj(1))
     secondIndex = obj(2)
     sv.LoadFirst(firstIndex)
     sv.LoadSecont(secondIndex)
   End If
End Sub

So what would be the equivalent code for the following Sub in the B4A sample
B4X:
Private Sub LoadIndices
    Dim ser As B4XSerializator
    ser.Tag = 1
    ser.ConvertBytesToObjectAsync( _
        Bit.InputStreamToBytes(File.OpenInput(File.DirAssets, "first.dat")), "ser")
End Sub
In my case, the above said data does not reside on File.DirAssets, this has to come from the Remote MySQL DB via jRDC. I am confused, how is this going to work. May be I didn't understand the concept well.
Is that right what I have understood so far ?
Doesn't that B4J app run on some specific port on the same VPS server where I run my jRDC ?

Any push in the right direction will be appreciated.
 
Last edited:

Anser

Well-Known Member
Licensed User
Longtime User
Any plans to incorporate this feature in jRDC in its future versions ?

My knowledge level in developing Tools is limited. I still don't know from where in the RDCHandler.bas in jRDC should I call the following function.


B4X:
Sub CreateSvIndex (dataToIndex As List)
   
    Dim search As SearchView
    search.Initialize
    Dim obj() As Object = search.SetItems(dataToIndex)
    Dim ser As B4XSerializator
    Dim DataToSend() As Byte = ser.ConvertObjectToBytes(obj)
End Sub

A List should also be generated.

Can anyone help me on this ?

Regards

Anser
 
Last edited:

Erel

B4X founder
Staff member
Licensed User
Longtime User
Any plans to incorporate this feature in jRDC in its future versions ?
No. It is a very specific feature. The requirements will be different in each project.

It will be simpler to add another handler in the server that is responsible for building DataToSend and call it with HttpUtils2. For further discussion please start a new thread.
 

Roberto P.

Well-Known Member
Licensed User
Longtime User
Hello Erel,
I would understand if and how you can use this new object to read data from the local DB tables for put data in SearchView.
In other words, I would like to find a way to read data from a simple table, for example consisting of code and description, and transfer the data in SearchView.

You can indicate the best way to do this using B4XSerializator

Thank you
 

Roberto P.

Well-Known Member
Licensed User
Longtime User
the data are in the local database
 

Roberto P.

Well-Known Member
Licensed User
Longtime User
Hello Erel
I created a small example db to populate SearchView. I restored the setItems function to populate the SV.

I ask you a kind help to optimize the use of the new object with the Db.

Thanks so much
 

Attachments

  • SearchView.zip
    241.1 KB · Views: 1,004
Top