Android Question Searchtemplate large lists

harinder

Active Member
Licensed User
Longtime User
I am using below code to add 7000 airport choices into a Searchtemplate from a database. It takes 2-3 secs for searchtemplate to be loaded with the choices. Any faster method? Thanks
B4X: 
Sub airportquery
    Dim rs As ResultSet= Main.SQL1.ExecQuery( "SELECT airport FROM " & Main.DBTableplaces & " ORDER BY airport ")
    Dim mylist as list
    mylist.initialize
    Do While rs.NextRow
        mylist.Add(rs.GetString("airport"))       '7000 airport rows
    Loop
    SearchTemplateairport.SetItems(mylist)
    rs.close
End Sub
 
Sort by date Sort by votes

Peter Simpson

Expert
Licensed User
Longtime User
@harinder you say three seconds, but you do know that can also depend on the speed of the device it's being executed on.

Does your airport data look anything like this?

1,"Goroka Airport","Goroka","Papua New Guinea","GKA","AYGA",-6.081689834590001,145.391998291,5282,10,"U","Pacific/Port_Moresby","airport","OurAirports"
2,"Madang Airport","Madang","Papua New Guinea","MAG","AYMD",-5.20707988739,145.789001465,20,10,"U","Pacific/Port_Moresby","airport","OurAirports"
3,"Mount Hagen Kagamuga Airport","Mount Hagen","Papua New Guinea","HGU","AYMH",-5.826789855957031,144.29600524902344,5388,10,"U","Pacific/Port_Moresby","airport","OurAirports"
4,"Nadzab Airport","Nadzab","Papua New Guinea","LAE","AYNZ",-6.569803,146.725977,239,10,"U","Pacific/Port_Moresby","airport","OurAirports"
5,"Port Moresby Jacksons International Airport","Port Moresby","Papua New Guinea","POM","AYPY",-9.443380355834961,147.22000122070312,146,10,"U","Pacific/Port_Moresby","airport","OurAirports"
6,"Wewak International Airport","Wewak","Papua New Guinea","WWK","AYWK",-3.58383011818,143.669006348,19,10,"U","Pacific/Port_Moresby","airport","OurAirports"
7,"Narsarsuaq Airport","Narssarssuaq","Greenland","UAK","BGBW",61.1604995728,-45.4259986877,112,-3,"E","America/Godthab","airport","OurAirports"
8,"Godthaab / Nuuk Airport","Godthaab","Greenland","GOH","BGGH",64.19090271,-51.6781005859,283,-3,"E","America/Godthab","airport","OurAirports"
9,"Kangerlussuaq Airport","Sondrestrom","Greenland","SFJ","BGSF",67.0122218992,-50.7116031647,165,-3,"E","America/Godthab","airport","OurAirports"
10,"Thule Air Base","Thule","Greenland","THU","BGTL",76.5311965942,-68.7032012939,251,-4,"E","America/Thule","airport","OurAirports"
11,"Akureyri Airport","Akureyri","Iceland","AEY","BIAR",65.66000366210938,-18.07270050048828,6,0,"N","Atlantic/Reykjavik","airport","OurAirports"
12,"Egilsstaðir Airport","Egilsstadir","Iceland","EGS","BIEG",65.2833023071289,-14.401399612426758,76,0,"N","Atlantic/Reykjavik","airport","OurAirports"
13,"Hornafjörður Airport","Hofn","Iceland","HFN","BIHN",64.295601,-15.2272,24,0,"N","Atlantic/Reykjavik","airport","OurAirports"
14,"Húsavík Airport","Husavik","Iceland","HZK","BIHU",65.952301,-17.426001,48,0,"N","Atlantic/Reykjavik","airport","OurAirports"
15,"Ísafjörður Airport","Isafjordur","Iceland","IFJ","BIIS",66.05809783935547,-23.135299682617188,8,0,"N","Atlantic/Reykjavik","airport","OurAirports"
16,"Keflavik International Airport","Keflavik","Iceland","KEF","BIKF",63.985000610352,-22.605600357056,171,0,"N","Atlantic/Reykjavik","airport","OurAirports"
17,"Patreksfjörður Airport","Patreksfjordur","Iceland","PFJ","BIPA",65.555801,-23.965,11,0,"N","Atlantic/Reykjavik","airport","OurAirports"
18,"Reykjavik Airport","Reykjavik","Iceland","RKV","BIRK",64.1299972534,-21.9405994415,48,0,"N","Atlantic/Reykjavik","airport","OurAirports"
19,"Siglufjörður Airport","Siglufjordur","Iceland","SIJ","BISI",66.133301,-18.9167,10,0,"N","Atlantic/Reykjavik","airport","OurAirports"
20,"Vestmannaeyjar Airport","Vestmannaeyjar","Iceland","VEY","BIVM",63.42430114746094,-20.278900146484375,326,0,"N","Atlantic/Reykjavik","airport","OurAirports"
21,"Sault Ste Marie Airport","Sault Sainte Marie","Canada","YAM","CYAM",46.48500061035156,-84.5093994140625,630,-5,"A","America/Toronto","airport","OurAirports"
22,"Winnipeg / St. Andrews Airport","Winnipeg","Canada",\N,"CYAV",50.0564002991,-97.03250122070001,760,-6,"A","America/Winnipeg","airport","OurAirports"
23,"Halifax / CFB Shearwater Heliport","Halifax","Canada",\N,"CYAW",44.639702,-63.499401,144,-4,"A","America/Halifax","airport","OurAirports"
24,"St. Anthony Airport","St. Anthony","Canada","YAY","CYAY",51.3918991089,-56.083099365200006,108,-3.5,"A","America/St_Johns","airport","OurAirports"
25,"Tofino / Long Beach Airport","Tofino","Canada","YAZ","CYAZ",49.079833,-125.775583,80,-8,"A","America/Vancouver","airport","OurAirports"
26,"Kugaaruk Airport","Pelly Bay","Canada","YBB","CYBB",68.534401,-89.808098,56,-7,"A","America/Edmonton","airport","OurAirports"
27,"Baie Comeau Airport","Baie Comeau","Canada","YBC","CYBC",49.13249969482422,-68.20439910888672,71,-5,"A","America/Toronto","airport","OurAirports"
28,"CFB Bagotville","Bagotville","Canada","YBG","CYBG",48.33060073852539,-70.99639892578125,522,-5,"A","America/Toronto","airport","OurAirports"
29,"Baker Lake Airport","Baker Lake","Canada","YBK","CYBK",64.29889678960001,-96.077796936,59,-6,"A","America/Winnipeg","airport","OurAirports"
30,"Campbell River Airport","Campbell River","Canada","YBL","CYBL",49.950801849365234,-125.27100372314453,346,-8,"A","America/Vancouver","airport","OurAirports"
31,"Brandon Municipal Airport","Brandon","Canada","YBR","CYBR",49.91,-99.951897,1343,-6,"A","America/Winnipeg","airport","OurAirports"
32,"Cambridge Bay Airport","Cambridge Bay","Canada","YCB","CYCB",69.1081008911,-105.138000488,90,-7,"A","America/Edmonton","airport","OurAirports"
33,"Nanaimo Airport","Nanaimo","Canada","YCD","CYCD",49.054970224899996,-123.869862556,92,-8,"A","America/Vancouver","airport","OurAirports"
34,"Castlegar/West Kootenay Regional Airport","Castlegar","Canada","YCG","CYCG",49.2963981628,-117.632003784,1624,-8,"A","America/Vancouver","airport","OurAirports"
35,"Miramichi Airport","Chatham","Canada","YCH","CYCH",47.007801,-65.449203,108,-4,"A","America/Halifax","airport","OurAirports"
36,"Charlo Airport","Charlo","Canada","YCL","CYCL",47.990799,-66.330299,132,-4,"A","America/Halifax","airport","OurAirports"
37,"Kugluktuk Airport","Coppermine","Canada","YCO","CYCO",67.816704,-115.143997,74,-7,"A","America/Edmonton","airport","OurAirports"
38,"Coronation Airport","Coronation","Canada","YCT","CYCT",52.0750007629,-111.444999695,2595,-7,"A","America/Edmonton","airport","OurAirports"
39,"Chilliwack Airport","Chilliwack","Canada","YCW","CYCW",49.1528015137,-121.939002991,32,-8,"A","America/Vancouver","airport","OurAirports"
40,"Clyde River Airport","Clyde River","Canada","YCY","CYCY",70.4860992432,-68.5167007446,87,-5,"A","America/Toronto","airport","OurAirports"
41,"Coral Harbour Airport","Coral Harbour","Canada","YZS","CYZS",64.1932983398,-83.3593978882,210,-5,"A","America/Coral_Harbour","airport","OurAirports"
42,"Dawson City Airport","Dawson","Canada","YDA","CYDA",64.04309844970703,-139.1280059814453,1215,-8,"A","America/Vancouver","airport","OurAirports"
43,"Burwash Airport","Burwash","Canada","YDB","CYDB",61.37110137939453,-139.04100036621094,2647,-8,"A","America/Vancouver","airport","OurAirports"
44,"Princeton Airport","Princeton","Canada",\N,"CYDC",49.4681015015,-120.511001587,2298,-8,"A","America/Vancouver","airport","OurAirports"
45,"Deer Lake Airport","Deer Lake","Canada","YDF","CYDF",49.21080017089844,-57.39139938354492,72,-3.5,"A","America/St_Johns","airport","OurAirports"
46,"Dease Lake Airport","Dease Lake","Canada","YDL","CYDL",58.4221992493,-130.031997681,2600,-8,"A","America/Vancouver","airport","OurAirports"

If yes then you should be loading your results into an xCustomListView using lazy loading. That way your MyList can be used to populate the xCustomListView and your list will appear almost instantly, it will not take 3 seconds to load your airport name data.

Search the forum for 'Lazy Loading'.

Plus you should never ever need to populate a list with 7000 items, add some way to add filtering for the end user.

Enjoy...
 
Upvote 0

harinder

Active Member
Licensed User
Longtime User
Hello Peter..it is a table in a database, not a .txt file, screenshot attached..

I will try Lazy Loading and revert..

How can I load say 500 items initially from db so that it doesen't delay showing the template, and after it shows up, continues loading the rest (delayed loading)...

I also found this post by Erel, but don't know how to implement ir, as I am using a db, not a .dat file..

Large & Searchable List with SearchView + B4XSerializator

Better to use: https://www.b4x.com/android/forum/threads/b4x-sqlsearchview-sqlite-based-search-view.133379/ Two very important methods were added to B4XSerializator in RandomAccessFile v2.20: ConvertObjectToBytesAsync and ConvertBytesToObjectAsync. With these methods you can asynchronously...
www.b4x.com
 

Attachments

  • Screenshot_2020-01-18-08-31-39-309_by.androld.app.dbreader.jpg
    347.4 KB · Views: 194
Upvote 0
M

Mahares

Expert
Licensed User
Longtime User
harinder said:
How can I load say 500 items initially from db so that it doesen't delay showing the template
Click to expand...
The purpose of the B4XSearchTemplate is to allow you to filter your data via a built in search box. If you only load 500 records and start the search using an example: If you type 'bi', the search list will display every item that contains 'bi'. If you have items that contain 'bi' in the remaining 6500 records, you will miss them from the search. So it is essential that you have the entire list of 7000 records available to search.
I have tested it with a table containing 6146 records on a modest tablet that is not blazing fast and I got 2.9 seconds to prepare the list in debug. The time was less in Release at 2.3 sec.
Run it in release mode, you will pick up some time and remember, you only need to prepare the list once
harinder said:
It takes 2-3 secs for searchtemplate to be loaded
Click to expand...
That is not terribly bad for 7000 records.
 
Upvote 0

kisoft

Well-Known Member
Licensed User
Longtime User
Hello
2-3 seconds is a good result for 7,000 records . Add a progress indicator and leave it as it is.
 
Upvote 0

harinder

Active Member
Licensed User
Longtime User
Thanks..I wonder if xCLV with Lazy Loading can be used with a search function, as my requirement is not only displaying the 7000 choices, but also being able to narrow down choices as one types.
I used this list on a searchview and loading is instantaneous. When I type ab, it shows multiple matching choices and as I scroll, there is no lag. Same if i type london and amsterdam.
But I want to use searchtemplate, as it is being used throughout my application for other smaller search functions.
 
Upvote 0

Erel

B4X founder
Staff member
Licensed User
Longtime User
Step 1: build the index with B4J and add it to B4A:
B4X: 
'read the data from the CSV file
Dim su As StringUtils
Dim list As List = su.LoadCSV("C:\Users\H\Downloads\airports.dat", "", ",")
Dim airports As List
airports.Initialize
For Each row() As String In list
    airports.Add(row(1))
Next
Dim search As B4XSearchTemplate
search.Initialize
Dim index As Object = search.SetItems(airports) '<--- this is the place where the index is built
Dim ser As B4XSerializator
File.WriteBytes("C:\Users\H\Downloads\index", "", ser.ConvertObjectToBytes(index))

Step 2: Load it asynchronously so the app never freezes:

B4X: 
Sub Activity_Create(FirstTime As Boolean)
    Activity.LoadLayout("1")
    dialog.Initialize(Activity)
    search.Initialize
    Dim ser As B4XSerializator
    Dim data() As Byte = File.ReadBytes(File.DirAssets, "index")
    ser.ConvertBytesToObjectAsync(data, "ser")
    Wait For ser_BytesToObject (Success As Boolean, NewObject As Object)
    search.SetIndex(NewObject)
    Button1.Enabled = True
End Sub

Sub Button1_Click
    Wait For (dialog.ShowTemplate(search, "Yes", "", "Cancel")) Complete (Result As Int)
End Sub
 
Upvote 0

harinder

Active Member
Licensed User
Longtime User
Erel..its taking same time, so I guess it is a delay with visuals of the searchtemplate..
 
Upvote 0

harinder

Active Member
Licensed User
Longtime User
I initiate the airportsearchtemplate via following sub:
B4X: 
Sub Btnfrom_Click
    airportquery   
    wait for (Dialog.ShowTemplate(SearchTemplateairport, "", "", "CANCEL")) Complete (Result As Int)
    If Result = xui.DialogResponse_Positive Then
        Dim airport As String =SearchTemplateairport.Selecteditem
        Dim sf As StringFunctions
        sf.Initialize
        btnfrom.xLBL.Text = sf.Left(airport,3)
        lblfrom.SetVisibleAnimated(1000,True)
    End If
End Sub

Loading of airportsearchtemplate via airportquery:
B4X: 
Sub airportquery
    Dim rs As ResultSet= Main.SQL1.ExecQuery( "SELECT airport FROM " & Main.DBTableplaces & " ORDER BY airport ")
    mylist.initialize
    Do While rs.NextRow
        mylist.Add(rs.GetString("airport"))
    Loop
    SearchTemplateairport.SetItems(mylist)
    rs.close
End Sub
It takes 3 secs for searchtemplate to show.
Best if you upload a small example. BTW, the index time is printed in the logs when you use SetItems.
Click to expand...
Index time: 806 ms (5654 Items)
 
Upvote 0

harinder

Active Member
Licensed User
Longtime User
The delay is being caused(3secs) because searchtemplate invariably loads all 5654 items and can be seen as a list and when user starts typing, then the search function kicks in narrowing doing the choices as per letters typed. Can the searchtemplate behaviour be modified to be like a searchview, where all items are not loaded first(I presume because no choice can be seen until user starts typing, and then the relevant choices are visual)..Thanks
 
Upvote 0
M

Mahares

Expert
Licensed User
Longtime User
Check the time it takes to execute the query and create the list as I show it below:
B4X: 
Sub airportquery
    Dim t1 As Long=DateTime.Now
    Dim rs As ResultSet= Main.SQL1.ExecQuery( "SELECT airport FROM " & Main.DBTableplaces & " ORDER BY airport ")
    mylist.initialize
    Do While rs.NextRow
        mylist.Add(rs.GetString("airport"))
    Loop
    Dim t2 As Long=DateTime.Now
    Dim dt As Long=t2-t1
    Log($"time to create the list ${dt}"$)
    SearchTemplateairport.SetItems(mylist)
    rs.close
End Sub
If the time is slow, it might help to create and index on the airport column for the your table.
Also, you should not use the 'StringFunctions' library as it it has not been supported for years and there are alternatives without the overhead of another library.
B4X: 
btnfrom.xLBL.Text = sf.Left(airport,3)
should be:
B4X: 
btnfrom.xLBL.Text = airport.substring2(0,3)

'
 
Upvote 0

harinder

Active Member
Licensed User
Longtime User
Erel said:
That's not true. However I cannot further help you here as you didn't upload a small project...
Click to expand...
Apologies Erel...I was doing a few routes from the airports list and did not find time earlier. Demo attached
 

Attachments

  • airportsearchtemplate.zip
    68.2 KB · Views: 159
Upvote 0
M

Mahares

Expert
Licensed User
Longtime User
harinder said:
Demo attached
Click to expand...
I am making some comments after seeing your project, but I leave it up to Erel to better help you and improve upon my comments if he deems necessary, since he asked you for your project:
  • Move airportquery line from Sub btnfrom_click to the end of Activity_Create. Otherwise, you are recreating the list and search template every time you click the button. It worked fast for me after I made the change.
  • You may want to add something like: SearchTemplateairport.Resize(200dip,600dip) after you initialize SearchTemplateairport, so you can show a lot more records on the screen.
  • In all previous examples, you use a database table, but in the example that you uploaded you use a text file. I do not know why you changed the code.
 
Upvote 0

harinder

Active Member
Licensed User
Longtime User
Hi Mahares..shifting airportqueryline did change the searchtemplate appearing time to about 2 secs
Adding resize only changes the panel size..template size with choices remain same
I would had to extract the table from database to show in project. Text file was handy. Index time with txt is 664ms..db is 806ms
 
Upvote 0
M

Mahares

Expert
Licensed User
Longtime User
harinder said:
appearing time to about 2 secs
Click to expand...
Mine shows up after 1.3 sec for 5654 records on my tablet. You must be using a slightly slower device.
harinder said:
Adding resize only changes the panel size..template size with choices remain same
Click to expand...
It is only meant to display more visible records on the screen by increasing the height. It only increases the height of the panel. To see more visible records, you can do:
SearchTemplateairport.Resize(200dip,Activity.Height)
 
Upvote 0
You must log in or register to reply here.

Similar Threads

  • Question
Android Question The best way to do massive searches
2
Replies
33
Views
3K
Mahares
M
  • Article
B4A Library [B4A] - AH_CheckInternet - Check network status (Library)
Replies
6
Views
6K
mhk1368
M
D
  • Question
Android Question Dialog parameters that change on the fly?
Replies
1
Views
757
dbprogramer
D
A
  • Question
Android Question Help with parsing xml data
Replies
17
Views
2K
apty
A
M
  • Question
Android Question I Populate a Random Access File With 2 Lists, But Only One Is Displayed
Replies
11
Views
2K
KMatle
Menu
Log in
Register
Cookies are required to use this site. You must accept them to continue using the site. Learn more…