Android Question Multi core processing in Android

[incendio]

Today's, phones / tablets, is common to have more than 1 core.
When app doings some processing, is it using all cores or only 1 core used? And who's controlling how many cores used by app, Android OS, or developer?

 

[incendio]

But how do you get only the updated records? I don't understand. Your solution seems only valid for a complete database to download.

No, it won't download complete database. This is only additional table served as a flag, increased its value every time there are an update on table data.

Table data still have a timestamp field.
Instead off query directly to table data for every newer records, app will try to query this additional table first, caused it will faster since it is only contain 1 record. If this additional table indicated that there are an update on table data, then app will query these update on table data.

 

[Informatix]

No, it won't download complete database. This is only additional table served as a flag, increased its value every time there are an update on table data.

Table data still have a timestamp field.
Instead off query directly to table data for every newer records, app will try to query this additional table first, caused it will faster since it is only contain 1 record. If this additional table indicated that there are an update on table data, then app will query these update on table data.

Most of the time spent is in the exchange between the client and the server, not in selecting the records. I suppose that your DB engine (I don't know at all Firebird) is able to run a Select query on 20000 records in a few ms. So your solution will probably take more time than querying directly the updated records, and you can have an inconsistency between the ID and the retrieved records (imagine that you have your UpdateFlag=4, and while the device is getting back this information and reissuing a query, new records are added to the table so the new UpdateFlag=5; on next use, the device will do 2 useless queries because it has already the new records).

 

[thedesolatesoul]

I am still just curios about why we dont see an improvement in multiple requests to the server. There is either syncronisation happening at the client sending out requests (highly unlikely), the server is not creating concurrent connections, or when you receive your data, your JobDone thread is busy, so it happens there (maybe its the try catch).
The only way to know is to log the times of the events i.e. Log time when each request is sent out, log it when entering in the job done sub, also log when your JobDone sub exits.

 

[incendio]

Most of the time spent is in the exchange between the client and the server, not in selecting the records. I suppose that your DB engine (I don't know at all Firebird) is able to run a Select query on 20000 records in a few ms.

Yes it could run it without a problem, so no additional table then

 

[incendio]

I am still just curios about why we dont see an improvement in multiple requests to the server. There is either syncronisation happening at the client sending out requests (highly unlikely), the server is not creating concurrent connections, or when you receive your data, your JobDone thread is busy, so it happens there (maybe its the try catch).
The only way to know is to log the times of the events i.e. Log time when each request is sent out, log it when entering in the job done sub, also log when your JobDone sub exits.

I already did that. From the log, seem that, event fires 4 request instantly, each job submitted one by one not in parallel. The pattern is on my post #29.

On server side, I have set it to handle 250 connections, but RDC connect with server via JDBC, I still don't check if JDBC set/able to handle concurrent connections.

About JobDone, I don't know how to check whether it is busy or not.

 

[thedesolatesoul]

Yes, but it is not clear from that patter what event is logged when.
For JobDone, just put a Log("Starting Job Done:" & DateTime.Time(DateTime.now)) after Sub JobDone and
put a Log("Finishing Job Done:" & DateTime.Time(DateTime.now)) before End Sub.

 

[keirS]

post: 289586, member: 61321"]Thanks for your suggestion. Considering that there is a potential corrupted sqlite database in devices, which is a more big problems, it seem that this the only solution.

I have data logging solutions running which add over 1 million records to a SQLite table per day. I have never had a corrupted database. There are several solutions to ensure you DB doesn't suffer form corruption.

Use a memory based database then write it to the main DB and ensure you have Write Ahead Logging enabled.

oSQL.ExecNonQuery("ATTACH DATABASE ':memory:' AS DBTEMP")
oSQL.ExecNonQuery("CREATE TABLE DBTEMP.temptable (Field1 interger, Field2 integer, Field3 String, Field4 String, Field5 String, Field6 String, Field7 String)")

Your insert for the memory table would look like this.

SQL.AddNonQueryToBatch("insert into DBTEMP.temptable(Field1,Field2,Field3,Field4,Field5,Field6,Field7) values(?,?,?,?,?,?,?)",Array As Object(ValI1,ValI2,ValC1,ValC2,ValC3,ValC4,ValI3))

When you have collected all your data from the server:

oSQL.ExecNonQuery("INSERT INTO   table (Field1,Field2,Field3,Field4,Field5,Field6,Field7) SELECT
Field1,Field2,Field3,Field4,Field5,Field6,Field7)   FROM DBTEMP.temptable")

 

[incendio]

Use a memory based database then write it to the main DB and ensure you have Write Ahead Logging enabled.

How to check Write Ahead Logging on my sqlite already enabled?

oSQL.ExecNonQuery("INSERT INTO   table (Field1,Field2,Field3,Field4,Field5,Field6,Field7) SELECT
Field1,Field2,Field3,Field4,Field5,Field6,Field7)   FROM DBTEMP.temptable")

Should this code execute within transaction?

 

[incendio]

Yes, but it is not clear from that patter what event is logged when.
For JobDone, just put a Log("Starting Job Done:" & DateTime.Time(DateTime.now)) after Sub JobDone and
put a Log("Finishing Job Done:" & DateTime.Time(DateTime.now)) before End Sub.

Here is the log
4 Request submitted : 09:16:45
** Service (httputils2service) Create **
** Service (httputils2service) Start **
** Service (httputils2service) Start **
** Service (httputils2service) Start **
** Service (httputils2service) Start **

HandleJob: 2189
Starting Job Done:09:16:48
Adding query to batch :09:16:48
Adding query Finished :09:16:48
Start execute Query :09:16:48
Finishing Job Done:09:16:48
Executing batch finished:09:16:48

HandleJob: 1873
Starting Job Done:09:16:50
Adding query to batch :09:16:50
Adding query Finished :09:16:51
Start execute Query :09:16:51
Finishing Job Done:09:16:51

HandleJob: 2200
Starting Job Done:09:16:53
Adding query to batch :09:16:53
Adding query Finished :09:16:53
Start execute Query :09:16:53
Finishing Job Done:09:16:53

HandleJob: 2727
Starting Job Done:09:16:56
Adding query to batch :09:16:56
Adding query Finished :09:16:56
Start execute Query :09:16:56
Finishing Job Done:09:16:56

Executing batch finished:09:16:56
Executing batch finished:09:16:56
Executing batch finished:09:16:56

Seem that http2service not run in parallel, but execute one by one.

 

[Erel]

JobDone is executed on the main thread. So the events will always run one after another.

However the http requests are handled in parallel.

 

[incendio]

JobDone is executed on the main thread. So the events will always run one after another.

However the http requests are handled in parallel.

Is Job done execution time depend on number of data send with http request?

I tried that test with about 20K records, will it be more time needed if test it with 500K records?

 

[keirS]

How to check Write Ahead Logging on my sqlite already enabled?



Should this code execute within transaction?

I use my SQLExtended library to do this but you can do:

oCursor = oSQL.Exec
Query("PRAGMA journal_mode")

to get the current journal mode. This will return a single row; single column cursor with the current journal mode. You can use

oSQL.ExecNonQuery("PRAGMA journal_mode = WAL")

to set WAL.

As for transactions if you want to be able to roll back then by all means wrap it in a transaction. However transactions are implicit in SQLite so any INSERT, UPDATE, DELETE etc. command is automatically wrapped in a transaction. Explicit transactions are used for rollback and running a set of SQL statements because it removes the overhead of a transaction being created for each statement .

 

[thedesolatesoul]

JobDone is executed on the main thread. So the events will always run one after another.

However the http requests are handled in parallel.

Yes, so this is what I wanted @incendio to check. And now I have to agree with him that the Http Requests do not seem to be returning in parallel (it may be possible they are sent in parallel, but somewhere they are definitely being queued up).

Considering the first JobDone finishes execution at 09:16:48, it has 2 full seconds before the next response returns, and 8 seconds before the last one returns.

@incendio I used an app called Network Log recently to check network calls being made from each app.