B4J Question Live changes to MDB database

ziovec

Member
Licensed User
Hello there!

Is it possible that once a database connection is initialized B4J doesn't recognize if there are changes to said database?

I noticed that if I do initialize the database, and later I'll simply remove the db from te right path or rename it, the code is not aware about that, and all my queries still works.
Am I missing something?

I do connect in this way:
B4X:
sql.Initialize("net.ucanaccess.jdbc.UcanaccessDriver", $"jdbc:ucanaccess://${path}"$)

And all the stuff I do with the database are simple select queries:
B4X:
Dim username As String = sql.ExecQuerySingleResult2("SELECT Denom FROM ANG_SOGG WHERE WEBeMail = ?", Array As String(a))

I do really need to see live changes to the database, even if those changes are not made by B4J ☺
 

OliverA

Expert
Licensed User
Longtime User
Upvote 0

ziovec

Member
Licensed User
I'm afraid it is not :(

I'm not executing any transaction at all, as I said, I'm just doing only select queries.
That problem is that at a random moment this database is going to be replaced with another, and I need to perform those queries on the newest databse, while it seems that B4J holds a cached version of the database at the time it goes with the very first initialization >.<
 
Upvote 0

OliverA

Expert
Licensed User
Longtime User
Is sql a global object?
 
Upvote 0

OliverA

Expert
Licensed User
Longtime User
Can the database change happen in the same websocket session?
 
Upvote 0

OliverA

Expert
Licensed User
Longtime User
Actually, none of the information you have provided is irrelevant. It all provides clues. Please note:
WebSocket connections can break and re-connections would either cause
a) the client to open another database, without realizing that a new connection was established
b) the client opens the same database twice, which may cause issues with the ucannaccess driver (see https://stackoverflow.com/a/36932419)

You need to provide a means for a client to reconnect behind the scenes without being forced to change/re-open an existing open database. If you are dealing with webclients, then see this: https://www.b4x.com/android/forum/threads/server-automatic-reconnecting-websocket.62054/. If you are dealing with Android/iOS devices, then you have to create a means of tracking connections (for example: tokens). As to the databases, you need to track which ones are open and if a client reconnects, you need to re-assign the same SQL object to the client as it used previously. You need a proper logout mechanism/time-out mechanism that you can use to properly close any open databases.
 
Upvote 0

ziovec

Member
Licensed User
As to the databases, you need to track which ones are open and if a client reconnects, you need to re-assign the same SQL object to the client as it used previously.

At the moment, when the client reconnects the sql object remains the same.

I'll try to epxlain me better.

I have a B4J web app with jServer. There are about 4/5 static html pages, each with their own websocket class properly linked.
In 2 of those pages there are html tables that shows data fetched via queries to the database.
So, the client doesn't open databases, it's a server duty.
The database is located in the same machine where the server runs, and in any random moment there's a chance someone uploads via FTP ad updated version of the database, in wich some row are deleted or new rows are added.
Code works like a charm (I'm still learning and experimenting) but I feel I'm missing something required to clear the cache or refresh the connection. I looked throuh the forum but I found nothing :(

You need a proper logout mechanism/time-out mechanism that you can use to properly close any open databases.
I agree, but as I said I have no idea how to do it ^^

I'll attach the project...there will be no html pages nor the database due to size restrictions ^^
(code is badly commented...and where it is, it's done in italian :p)
 

Attachments

  • ziovec problem.zip
    13 KB · Views: 134
Upvote 0

OliverA

Expert
Licensed User
Longtime User
The issue is that if you do update the database in the background, the Server Socket code has no way to know that happened and therefore there is no mechanism in play to update the information. For example, in the Product socket handler you fill in the two select boxes in the WebSocket_Connected method. If the database changes, nothing in th handler updates these select boxes. So as long as the socket connection exists, no matter how often you update the database in the background, the content of these two select boxes will not change.

What's interesting is that ucanaccess seems to be handling your database replacement without tripping over it's own feet. Looks like it can detect changes to the underlying database. If found, it just does a complete reload.

Currently, for each incoming socket connection and even for the login handler, you open the database. Right now, since you are just reading information, this seems to work. If you ever update the information via the handlers in the B4J applications, you will most likely encounter problems. Testing will either confirm or not confirm this theory.
 
Upvote 0

ziovec

Member
Licensed User
For example, in the Product socket handler you fill in the two select boxes in the WebSocket_Connected method. If the database changes, nothing in th handler updates these select boxes. So as long as the socket connection exists, no matter how often you update the database in the background, the content of these two select boxes will not change.
Yup, and that's kind of intuitive: those select boxes are filled when the page is connected.
What I would expect is that if I update the database (or remove it entirely, whatever) and THEN refresh the Product page, the queries should now be performed to the new database (or giving out an error because the db is removed), while instead I do always get my select boxes filled with the same exact data.


Currently, for each incoming socket connection and even for the login handler, you open the database. Right now, since you are just reading information, this seems to work. If you ever update the information via the handlers in the B4J applications, you will most likely encounter problems. Testing will either confirm or not confirm this theory.
How to avoid this behaviour?
I remember I tried initializing the database in the Main class and then perform queries calling Main.sql.ExecuteQuery, but I always got an error saying that object should be initialized first ?‍♂️


The issue is that if you do update the database in the background, the Server Socket code has no way to know that happened and therefore there is no mechanism in play to update the information.
And here's the BIG question: how to properly close and eventually re-start a connection? sql.Close seems not working for that pourpouse :(


P.S: thanks a lot for your replies and engagement! I'm still learning how to handle databases and this is great help :)
 
Upvote 0

OliverA

Expert
Licensed User
Longtime User
THEN refresh the Product page, the queries should now be performed to the new database
Refreshing a web-page connected via WebSocket does not call the WebSocket_Connected routine again (unless a disconnect happened). It is there that you are populating the select boxes at.

I remember I tried initializing the database in the Main class and then perform queries calling Main.sql.ExecuteQuery, but I always got an error saying that object should be initialized first
Makes no sense. Did you report this?

And here's the BIG question: how to properly close and eventually re-start a connection? sql.Close seems not working for that pourpouse
As of now, you have so many SQL.Initializes happening, that one sql.close does not take care of the issue. Plus ucanaccess can do it's own caching that could produce some strange results in combination on how you are doing it now.
 
Upvote 0

ziovec

Member
Licensed User
Refreshing a web-page connected via WebSocket does not call the WebSocket_Connected routine again (unless a disconnect happened). It is there that you are populating the select boxes at.
Tested again but yes, refreshing the page actually causes a WebSocket_Disconnect immediatly followed by a WebSocket_Connected.
That's why I find everything strange.
 
Upvote 0

OliverA

Expert
Licensed User
Longtime User
Is it possible that once a database connection is initialized B4J doesn't recognize if there are changes to said database?
It has 0% to do with B4J and 100% to do with UCanAccess's JDBC driver.

1) Unlike SQLite's JDBC driver, with UCanAccess, you can open the same database multiple times (done via B4J's SQL's Initialize method). Internally, UCanAccess will return you the previously created connection object. In that object, UCanAccess keeps track of the number of active connections that a DB has. Each Initialize will increment this counter, each Close will decrement that counter. If this counter reaches 0 due to an invoked Close method, UCanAccess will shut down the underlying DB and the next call to Initialize will then create a new connection object and re-open the underlying database. If this counter does not reach 0, the database will not ever be reloading (via a Close statement)

2) Even if the counter in point #1 above reaches 0, out of the box, UCanAccess does not immediately close the the underlying DB, but waits for 2 minutes before doing so. If within two minutes the DB is opened again, UCanAccess will just re-use the still open connection (incrementing the active connections counter). The amount of time that UCanAccess waits for can be controlled with two configuration options:

inactivityTimeout (since UcanAccess 0.1.1): it is used only with the default memory mode (memory=true). If there are no active connections for the inactivityTimeout period (in minutes) HSQLDB will be temporarily shut down and any filesystem resources will be released. It will be restarted at the next connection. Setting inactivityTimeout=0 can be used for avoiding this behaviour. Default = 2 minutes.

immediatelyReleaseResources (replaces singleConnection which has been deprecated since UCanAccess 3.0.6): it is used for for ETL jobs, scheduled tasks, or "one-shot" use of UCanAccess with only one connection open. All resources (memory and filesystem) will be released at the closing of the connection. Default=false.

Note: inactivityTimeout is not applicable when setting memory=false. At that point, Close never really hardly ever closes the database (see next statement).

Of these two configuration options, immediatelyReleaseResources is the more interesting one. With this option, a final Close (where active connections = 0) will close the DB immediately (update: even with memory=false). If active connections <> 0, then the DB will remain open, even with this option set.

Solution #1:
a) Use one SQL object to open the database via Initialize. This can be done by declaring a Global variable in Main and initializing it before starter the server service. Do not initialize another SQL object for the database used anywhere else.
b) Modify the JDBC URL to include the immediatelyReleaseResources=True option. Using the example provided in post #1, the modified version would be
B4X:
sql.Initialize("net.ucanaccess.jdbc.UcanaccessDriver", $"jdbc:ucanaccess://${path};immediatelyReleaseResources=True"$)
c) Introduce code into the project that
1) Stops all clients from reading/writing the database
2) Calls Close on the current database
3) Replaces the current database
4) Opens the new database -> step a)
5) Inform clients that the database is available again

Solution #2:
a) Keep opening the DB multiple times (Note: no need for setting mmediatelyReleaseResources=True)
b) Introduce code into the project that
1) Stops all clients from reading/writing the database
2) Closes the database via the provided unloadDB method of the UcanaccessConnection class
B4X:
    Dim sql As SQL
    sql.Initialize("net.ucanaccess.jdbc.UcanaccessDriver", $"jdbc:ucanaccess://${Main.DBPath}"$)
    Dim jo As JavaObject = sql
    jo = jo.GetFieldJO("connection")
    jo.RunMethod("unloadDB", Null)
3) Replaces the current database
4) Inform clients that the database is available again

Notes:
1) I'm not an UCanAccess expert, so another more elegant solution may be out there. Point in case: UCanAccess supposedly is able to tell when an underlying change occurred from outside sources and then reload the database as necessary. I've not been able to reproduce this (in this limited, read data only case). Renaming the database (effectively deleting it) did not stop the application from reading information from the database (albeit it was the same information, same tables over and over again). I expected the application to crash once the database file was renamed. No such "luck".
2) Solution #1 would be preferable, since UCanAccess's ability to open the same local database may be unique. This way, the code could be adapted to other similar databases, such as SQLite. The unloadDB method call is not a JDBC standard. It's an extension of UCanAccess's implementation of the JDBC Connection object (SQLite has not such method in its connector).

Resources:
UCanAccess Getting Starte page: http://ucanaccess.sourceforge.net/site.html#examples
UCanAccess source code: https://sourceforge.net/projects/ucanaccess/files/
SQLite source code: https://github.com/xerial/sqlite-jdbc
 
Upvote 0

ziovec

Member
Licensed User
It has 0% to do with B4J and 100% to do with UCanAccess's JDBC driver.
[...]
Woah! That's a huge amount of information! :D

Tested both solutions and both works perfectly, now I only have to change a massive amount of logic in my code to get the things works all together in a proper way...but I guess I'll have to open a separate thread for that ;)

Thanks again @OliverA , you helped me a lot!
 
Upvote 0
Top