B4J Question keyValueStore is sluggish

[xulihang]

Hi,

I am writing a small registration web server using abmaterial. I store users' info on a single json file.

I think it is too simple to use json as my database, so I sought to keyValueStore. SQLite+keyvalue sounds great. But my app just increases from 6mb to 12mb and costs running memory. Also, the code is not as easy to maintain as a json file.

The keyValueStore is like an in-memory NoSQL datebase based on SQLite. As I am not a professional, I'd rather to store on disk directly.

So I decide to roll my code back. Using Json to store data is fine right now. Maybe I should store users' info on separate files based on user's id, which can improve concurrent access.

 

[Erel]

SQLite+keyvalue sounds great. But my app just increases from 6mb to 12mb and costs running memory.

The SQLite library will add about 3mb to the jar size. It shouldn't matter for a server solution.

Also, the code is not as easy to maintain as a json file.

I don't see how it is possible. It is trivial to use KVS 2. If you haven't understood something then post a question and we can help you.

With that said, KVS 2 out of the box is not multithreaded safe as required for a server solution. You need to set it to wal mode: https://www.b4x.com/android/forum/t...ent-access-to-sqlite-databases.39904/#content

 

[xulihang]

The SQLite library will add about 3mb to the jar size. It shouldn't matter for a server solution.


I don't see how it is possible. It is trivial to use KVS 2. If you haven't understood something then post a question and we can help you.

With that said, KVS 2 out of the box is not multithreaded safe as required for a server solution. You need to set it to wal mode: https://www.b4x.com/android/forum/t...ent-access-to-sqlite-databases.39904/#content

I find out the reason. I used too many putasync methods where I should simply use put. These async methods make my system not responsive and I cannot see the result instantly.

thanks,

 

[Erel]

PutAsync should be used when you want to insert many items at once. I guess that you tested it in debug mode. It should be fast in release mode.

In a server solution it is better to use Put as each request is handled by its own thread anyway.

 

[xulihang]

I guess that you tested it in debug mode

It ran well on my laptop but is slow on my vps.

The new compiled program runs well on both.

 

[Erel]

BTW, if you are not setting the pragma to wal then the database will become corrupted.

 

[xulihang]

BTW, if you are not setting the pragma to wal then the database will become corrupted.

thanks, I have added these code to set the wal pragma when creating the db file.

'Initializes the store and sets the store file.
Public Sub Initialize (Dir As String, FileName As String)
    If sql1.IsInitialized Then sql1.Close
#if B4J
    If File.Exists(Dir,FileName) Then
        sql1.InitializeSQLite(Dir, FileName, True)
    Else
        sql1.InitializeSQLite(Dir, FileName, True)
        sql1.ExecNonQuery("PRAGMA journal_mode = wal")
    End If
#else
    sql1.Initialize(Dir, FileName, True)
#end if
    CreateTable
End Sub