Android Question Fastest way to process results from RDC?

[ezderek]

First let me say I'm a noobie so I accept that my question might be naive and/or stupid, and I apologize.
I built a php middleware server to connect with a remote MySQL database. Typically we need to download between 10 and 200 rows and insert into a local database. Then user editing happens. Then they get uploaded and re-inserted into the master.
The way I wrote the code, I decided against XML or JSON. Instead, an http POST request for data from the client is turned into an INSERT statement by the middleware ...
For example: one screen lets you search by last name, and if you enter the name "Smith" the middleware returns this string (in UTF-8):

INSERT INTO `customer` (`customerkey`,`firstname`,`middlename`,`lastname`,`address1`) VALUES ('124', 'Byrl', 'L.', 'Smith', '607 W. Spring St.'), ('2854', 'Bonnie', 'L.', 'Smith', '560 N. Nixon'), ('3501', 'Ben', '', 'Smith', '2578 Shawnee Rd.'), ('4113', 'Beverly', 'J', 'Smith', '1737 Leland Ave'), ('4891', 'Barbara', 'Ilene', 'Smith', '851 E. High St.') ETC. ETC.

This gets piped straight into SQLite to make a local cursor (dataset) for further processing.

Now to get to the point, I was extremely interested in Erel's RDC webserver. I got it working with no problem which is amazing considering how dumb I am. It seems very fast processing requests, typically 2ms on a localhost database. But then you have a binary object which has to be read, and decoded, and looped and processed and interpreted, and inserted row by row... and all this takes time. So it seems to me that the php system is faster overall because as soon as it comes back from the middleware it just goes straight into the database with no massaging needed.

Now I have great faith in Erel (or I wouldn't be here) so I'm assuming I have some fundamental misunderstanding about the proper way to handle the object returned by RDC. There must be a quicker, easier way to get it into SQLite. What am I missing please? (PS yes I have read the tutorial. All 10 pages.) Thanks. Derek.

 

[JakeBullet70]

Have you looked into wrapping it all in a transaction?

 

[Erel]

Have you looked into wrapping it all in a transaction?

This is the correct answer. Use a single transaction to insert all the rows. It will be very fast.

 

[ezderek]

This is the correct answer. Use a single transaction to insert all the rows. It will be very fast.

My concern is that it is a lot of complex code to do what should be a simple task. And each different type of transaction you want to handle with the middleware will need its own unique bunch of client code to process the return. And every future update or addition to the webservers collection of queries involves a potential addition to the client code needed to handle it. So I see myself looking at a long development time, complex maintenance, and of course a greater possibility of bugs or glitches. I'm afraid in my ignorance, I don't see the upside here, compared to just downloading a SQL statement that does everything. (Told you I was naive).

 

[Erel]

If the complete data is small enough to be processed locally then it is a good solution to do it locally. The purpose of RDC is to allow you to process the data remotely.

RDC is very fast. It is faster than the solutions based on XML or JSON.