Android Tutorial Remote Database Connector (RDC) - Connect to any remote DB

Status
Not open for further replies.
jRDC version 2 is available here: https://www.b4x.com/android/forum/t...ation-of-rdc-remote-database-connector.61801/

This tutorial covers a new framework named Remote Database Connector (RDC). The purpose of RDC is to make it simple to develop Android applications that interact with remote database servers.

There are two components in this framework: a lightweight Java web server (the middleware) and a B4A class named DBRequestManager that interacts with this web server.

The Java web server can connect to any database platform that provides a JDBC driver.

This includes: MySQL, SQL Server, Oracle, Sybase, DB2, postgreSQL, Firebird and many others.

You can also use the web server without a database server by connecting to a local SQLite database file.

The Java web-server is a simple server that connects to the database server and to the Android clients.
As this is a Java app you can run it on Linux or Windows computers. It requires Java JRE 6 or 7.

This solution is much more powerful than the PHP (MySQL) and ASP.Net (MS SQL) solutions that are already available.

Main advantages over previous solutions:
  • Support for many types of database platforms.
  • Significantly better performance.
  • SQL statements are configured in the server (safer).
  • Support for all types of statements, including batch statements.
  • Support for BLOBs.

Server Configuration

JDBC is a Java API that provides a standard method to access any database. A database driver (jar file) is required for each type of database. You will need to download the driver for your database and put it in the jdbc_driver folder.

The Java server configuration is saved in a file named config.properties.
This file includes two sections: general configuration and a list of SQL commands.

For example:

SS-2013-08-04_16.10.20.png


Note that the configuration fields are case sensitive.

DriverClass / JdbcUrl - The values of these two fields are specific to each database platform. You will usually find the specification together with the required driver jar file.

User / Password - Database user and password.

ServerPort - The Java server will listen to this provided port.

Debug - If Debug is set to true then the SQL commands list will be loaded on every request. This is useful during development as it allows you to modify the commands without restarting the server process.

SQL Commands - A list of commands. Each command starts with 'sql.'. The commands can include question marks (parameterised queries). Question marks will be replaced with the values provided by the Android clients. Note that the command name is case sensitive and it doesn't include the 'sql.' prefix.

Client Code
The client sends requests to the server. There are two types of requests: query requests (usually SELECT statements) and batch requests (any other statement).

Note that both the client and the server can manage multiple requests in parallel.
Usually you will only need a single DBRequestManager object.

Each request is made of a command (or a list of commands) and a tag. The tag can be any object you like. You can use it later when the result is ready. The tag is not sent to the server.

A command is an object of type DBCommand:
B4X:
Type DBCommand (Name As String, Parameters() As Object)
Name - The case sensitive command name as configured in the server configuration (without sql.).
Parameters - An array of objects that will be sent to the server and will replace the question marks in the command.

For example to send a SELECT request:
B4X:
Dim cmd As DBCommand
cmd.Initialize
cmd.Name = "select_animal"
cmd.Parameters = Array As Object("cat 1")
reqManager.ExecuteQuery(cmd, 0, "cat 1")
ExecuteQuery expects three parameters: the command, maximum number of rows to return or 0 if there is no limit and the tag value.

Under the hood DBRequestManager creates a HttpJob for each request. The job name is always "DBRequest".

You should handle the JobDone event in your activity or service:
B4X:
Sub JobDone(Job As HttpJob)
   If Job.Success = False Then
     Log("Error: " & Job.ErrorMessage)
   Else
     If Job.JobName = "DBRequest" Then
       Dim result As DBResult = reqManager.HandleJob(Job)
       If result.Tag = "cat 1" Then 'query tag
         For Each records() As Object In result.Rows
           Dim name As String = records(0) 'or records(result.Columns.Get("name"))
           Log(name)
         Next
       End If
     End If
   End If
   Job.Release
End Sub

As you can see in the above code, DBRequestManager.HandleJob method takes the Job and returns a DBResult object:
B4X:
Type DBResult (Tag As Object, Columns As Map, Rows As List)
Tag - The request tag.
Columns - A Map with the columns names as keys and the columns ordinals as values.
Rows - A list that holds an array of objects for each row in the result set.

Non-select commands are sent with ExecuteCommand (single command) or ExecuteBatch (any number of commands). It is significantly faster to send one request with multiple commands over multiple requests. Batch statements are executed in a single transaction. If one of the commands fail all the batch commands will be cancelled (roll-backed).

Note that for non-select requests, Rows field in the results will hold an array with a single int for each command. The int value is the number of affected rows (for each command separately).

Initializing DBRequestManager:
B4X:
Sub Activity_Create(FirstTime As Boolean)
   If FirstTime Then
     reqManager.Initialize(Me, "http://192.168.0.100:17178")
   End If
End Sub
The first parameter is the module that will handle the JobDone event.
The second parameter is the link to the Java server (with the port number).

DBRequestManager provides the following helper methods for common tasks related to BLOB fields: FileToBytes, ImageToBytes and BytesToImage.It also includes a method named PrintTable that prints DBTable objects to the logs.

Framework Setup
  1. Unpack the server zip file.
  2. You will need to download the driver for your database platform and copy the jar file to jdbc_driver folder.
  3. Edit config.properties as discussed above.
  4. Edit RunRLC.bat and set the path to java.exe. If you are running it on Linux then you need to create a similar script with the path to java. On Linux you should change the ';' separator to ':'.
  5. Run the server :)
    You should see something like:

    SS-2013-08-04_17.05.16.png


    Note that the path to config.properties is printed in the second line.
  6. Add an exception for the server port in your firewall.
  7. Try to access the server from the browser:

    SS-2013-08-04_17.06.17.png


    You will probably not see the above message on the first run :(. Instead you will need to check the server output and read the error message.
    If you are able to call the server from the local computer and not from other devices then it is a firewall issue.

Tips

  • MySQL driver is available here: http://dev.mysql.com/downloads/connector/j/
  • Google for <your database> JDBC Driver to find the required driver. For SQL Server you can use this open source project: http://jtds.sourceforge.net/
  • The server uses an open source project named c3p0 to manage the database connection pool. It can be configured if needed by modifying the c3p0.properties file.
  • Use a text editor such as Notepad++ to edit the properties file.
  • If you are running the server on Linux then you can run it with nohup to prevent the OS from killing the process when you log out.
  • Java doesn't need to be installed. You can just unpack the files.
The server is based on the two following open source projects:
Jetty - http://www.eclipse.org/jetty/
c3p0 - http://www.mchange.com/projects/c3p0/index.html
http://www.mchange.com/projects/c3p0/index.html
 
Last edited:

LorenzoTRANSFEREX

Member
Licensed User
Longtime User
Hi, I get this error when I execute any SQL update in config.properties: "sql.update_albaranorigen=UPDATE Servicios SET HInicio=? WHERE Id=?"
The data update works fine, but the server returns ever this error and I don't know how ignore it in B4A:

com.microsoft.sqlserver.jdbc.SQLServerException: La instrucci¾n no devolvi¾ un c
onjunto de resultados.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(S
QLServerException.java:190)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePrep
aredStatement(SQLServerPreparedStatement.java:408)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecC
md.doExecute(SQLServerPreparedStatement.java:350)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLSe
rverConnection.java:1715)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLSer
verStatement.java:180)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLS
erverStatement.java:155)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeQuery(
SQLServerPreparedStatement.java:285)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewPr
oxyPreparedStatement.java:116)
at anywheresoftware.b4a.remotedatabase.Servlet.executeQuery(Servlet.java
:141)
at anywheresoftware.b4a.remotedatabase.Servlet.doGet(Servlet.java:78)
at anywheresoftware.b4a.remotedatabase.Servlet.doPost(Servlet.java:52)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:538
)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java
:478)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandl
er.java:937)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:
406)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandle
r.java:871)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j
ava:117)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper
.java:110)
at org.eclipse.jetty.server.Server.handle(Server.java:346)
at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.
java:589)
at org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpCo
nnection.java:1065)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:823)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:220)

at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:41
1)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEn
dPoint.java:535)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEnd
Point.java:40)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool
.java:529)
at java.lang.Thread.run(Thread.java:744)
 

MrKim

Well-Known Member
Licensed User
Longtime User
Yes. You can set a Null parameter. Please post the code that doesn't work.
It was late and I was tired. I was setting a string variable to Null. But that brings up the question - what type of variable to dim that can be either a string or Null - the VBA Variant equivalent? Object?
 

LorenzoTRANSFEREX

Member
Licensed User
Longtime User
@LorenzoTRANSFEREX seems like you are calling ExecuteQuery instead of ExecuteCommand. ExecuteQuery should only be used for SELECT requests.

I get it, thanks Erel.

Another issue I found is that in some smartphones like Lenovo S660 I must put the slash '/' in the next sentence in sub ExecuteQuery, if not I got a http error.

j.PostBytes(link & "/?method=query", ms.ToBytesArray)

Thanks and regards
 

johnB

Active Member
Licensed User
Longtime User
Hi, would appreciate some help

I'm having a problem with the "IN" statement in config.properties in RDC.

The following works -
sql.select_dividends_in3=SELECT * FROM DividendsPayable WHERE Stock IN ('ANZ.AX','CBA.AX','NBA.AX','WBC.AX')

I can't get the following to work
sql.select_dividends_in1=SELECT * FROM DividendsPayable WHERE Stock IN ?
sql.select_dividends_in2=SELECT * FROM DividendsPayable WHERE Stock IN (?)
sql.select_dividends_received_in=SELECT Stock, Company, ExDate, PayDate, DividendAmt, Franking, LastPrice, Yield FROM DividendsPayable WHERE Stock IN (?) AND PayDate > ?

The last one is the one I want but I can't get the first 2 to work.
If sent the parameters as a String, List and Object - eg

Dim params As Object = "'ANZ.AX','CBA.AX'" or
Dim params As Object = "('ANZ.AX','CBA.AX')"- set up as either Object, String or List as per Sub(s) below
SelectStock_IN(params)

'Sub SelectStock_IN(Name As List)
or
'Sub SelectStock_IN(Name As String)
or
Sub SelectStock_IN(Name As Object)
Dim cmd As DBCommand
cmd.Initialize
cmd.Name = "select_dividends_in1"
cmd.Parameters = Array As Object(Name)
reqManager.ExecuteQuery(cmd, 0, Null)
End Sub

If I put the () around the NAME parameter, the sql.select_dividends_in1 gives a database error, so I guess the () should not be included in the params

sql.select_dividends_in2=SELECT * FROM DividendsPayable WHERE Stock IN (?) returns the the number of columns but no records

My sincere appreciation in advance for any help
 

Erel

B4X founder
Staff member
Licensed User
Longtime User
The question marks are not simply replaced with the strings you supply. These are bound parameters.
Not all clauses can be parameterized. In this case each value is a parameter.

Assuming that the possible numbers of parameters is not too large then you can do something like:
B4X:
sql.select_dividends_in1=SELECT * FROM DividendsPayable WHERE Stock IN (?)
sql.select_dividends_in2=SELECT * FROM DividendsPayable WHERE Stock IN (?, ?)
sql.select_dividends_in3=SELECT * FROM DividendsPayable WHERE Stock IN (?, ?, ?)
...
 

Roberto P.

Well-Known Member
Licensed User
Longtime User
Sources are available to rdc server? Thank you very much. Greetings
 

johnB

Active Member
Licensed User
Longtime User
Hi, would appreciate some help

I'm having a problem with the "IN" statement in config.properties in RDC.

The following works -
sql.select_dividends_in3=SELECT * FROM DividendsPayable WHERE Stock IN ('ANZ.AX','CBA.AX','NBA.AX','WBC.AX')

I can't get the following to work
sql.select_dividends_in1=SELECT * FROM DividendsPayable WHERE Stock IN ?
sql.select_dividends_in2=SELECT * FROM DividendsPayable WHERE Stock IN (?)
sql.select_dividends_received_in=SELECT Stock, Company, ExDate, PayDate, DividendAmt, Franking, LastPrice, Yield FROM DividendsPayable WHERE Stock IN (?) AND PayDate > ?

The last one is the one I want but I can't get the first 2 to work.
If sent the parameters as a String, List and Object - eg

Dim params As Object = "'ANZ.AX','CBA.AX'" or
Dim params As Object = "('ANZ.AX','CBA.AX')"- set up as either Object, String or List as per Sub(s) below
SelectStock_IN(params)

'Sub SelectStock_IN(Name As List)
or
'Sub SelectStock_IN(Name As String)
or
Sub SelectStock_IN(Name As Object)
Dim cmd As DBCommand
cmd.Initialize
cmd.Name = "select_dividends_in1"
cmd.Parameters = Array As Object(Name)
reqManager.ExecuteQuery(cmd, 0, Null)
End Sub

If I put the () around the NAME parameter, the sql.select_dividends_in1 gives a database error, so I guess the () should not be included in the params

sql.select_dividends_in2=SELECT * FROM DividendsPayable WHERE Stock IN (?) returns the the number of columns but no records

My sincere appreciation in advance for any help
Thanks Erel
Unfortunately I have a variable number of parameters so I have solved it by selecting all the relevant records then doing the "variable" selection in the app.
I'm going to experiment with setting up variable cmd.parameters but the way I have it there is now only one call to RDC and from past experience I think that this is faster than multiple calls which as you pointed out before can come back in random sequence
 

wmardian

Member
Licensed User
Longtime User
Dear Erel,

Is it possible to configure RDC on shared hosting environment?
Where is the location we can unzip the rdc?

thanks in advance

best regards
 

dibesw

Active Member
Licensed User
Longtime User
Hi Erel,
I use RDC (with ucanaccess) wery well from 1 year.
My only problem is this:
if I run e.g. an insert into table and I run subsequently a query, I found this record;
but if I run an insert from access or another application, if I run the same query from RDC, I do not found this record.
If I close and open RDC (close and reopen RunRCL.bat) I found it!
The same happens if I do update.
I tried to run ucanaccess console. From this all works well.
What could be the reason?
Could be due to anywheresoftware.b4a.remotedatabase.RemoteServer.
What is this anywheresoftware.b4a.remotedatabase.RemoteServer?
Can you help me?
Thanks
 

johnB

Active Member
Licensed User
Longtime User
The question marks are not simply replaced with the strings you supply. These are bound parameters.
Not all clauses can be parameterized. In this case each value is a parameter.

Assuming that the possible numbers of parameters is not too large then you can do something like:
B4X:
sql.select_dividends_in1=SELECT * FROM DividendsPayable WHERE Stock IN (?)
sql.select_dividends_in2=SELECT * FROM DividendsPayable WHERE Stock IN (?, ?)
sql.select_dividends_in3=SELECT * FROM DividendsPayable WHERE Stock IN (?, ?, ?)
...
Only posting for the sake of completing this part of the post
I got the IN statement working by formatting as

ql.select_dividends_in1=SELECT * FROM DividendsPayable WHERE Stock IN (?)
sql.select_dividends_in2=SELECT * FROM DividendsPayable WHERE Stock IN (?,?)
sql.select_dividends_in3=SELECT * FROM DividendsPayable WHERE Stock IN (?,?,?)

It didn't like the spaces after the commas

Thanks again Erel for your help
 

incendio

Well-Known Member
Licensed User
Longtime User
I wonder about this parameter in file Run.bat
"C:\Program Files\Java\jdk1.6.0_37\jre\bin\java" -Xmx256m -cp .;libs\*;jdbc_driver\* anywheresoftware.b4a.remotedatabase.RemoteServer


1) is -xmx256m means that it will required 256MB RAM?
2) if it is what is the effect to increase or decrease RAM usage, such as -Xmx64m or -Xmx1024m

Thanks
 

incendio

Well-Known Member
Licensed User
Longtime User
Only posting for the sake of completing this part of the post
I got the IN statement working by formatting as

ql.select_dividends_in1=SELECT * FROM DividendsPayable WHERE Stock IN (?)
sql.select_dividends_in2=SELECT * FROM DividendsPayable WHERE Stock IN (?,?)
sql.select_dividends_in3=SELECT * FROM DividendsPayable WHERE Stock IN (?,?,?)

It didn't like the spaces after the commas

Thanks again Erel for your help
You should make store procedured that accept single string as a parameter.
This parameter string is something like this "ANZ.AX,CBA.AX", parsed this string in your store procedure before execute select statement.
 

Pravin Shah

Member
Licensed User
Longtime User
Hi Erel,

I am not able to connect the database using RDC server. I have tried connecting with MS SQL and MySQL server but I am not getting the error message. I have followed the instructions provided in your tutorial and listing down the steps which I have followed. This may help to identify the issue.
1. Extracted content of RDC-Client.zip and RDC-Serer.zip under appropriate directory
2. Downloaded the MySQL driver from http://dev.mysql.com/downloads/connector/j/ and installed it my machine. After that I have copied
mysql-connector-java-5.1.33-bin.jar file under jdbc_driver directory.


3. Modified the config.properties file and here are the contents of my file
DriverClass=com.mysql.jdbc.Driver
JdbcUrl=jdbc:mysql://localhost/test?characterEncoding=utf8
User=root
Password=
ServerPort=17178

4. After that I run the RunRLC.bat file and it runs properly and shows .....STARTING message as per your screenshot

5. After that I tried running following url 127.0.0.1:17178/?method=test to test database connection
but it gave following error.
RemoteServer is running (Tue Oct 14 13:16:43 IST 2014)
java.sql.SQLException: An attempt by a client to checkout a Connection has timed out.

I tried same process for MS SQL Server as well but got the same error. The JDBC server throws lot of errors and I am not able to identify what exactly is the problem. I am stuck up at database connection since yesterday. I appreciate your help. Thanks
 
Last edited:

Straker

Active Member
Licensed User
Longtime User
The message says that RDC is running but it cannot finde the database.
Are you sure the database is in localhost? Try with IP.
Are you shour you have a 'test' database?
 
Status
Not open for further replies.
Top