B4J Question [RESOLVED] Server Out of Memory Error

Jmu5667 · Apr 23, 2018

Hello

Have a server that is started with a batch file and the following errors have been thrown

B4X:

C:\Isle Systems Java\Aspect Hello>java -version
java version "1.8.0_152"
Java(TM) SE Runtime Environment (build 1.8.0_152-b16)
Java HotSpot(TM) Client VM (build 25.152-b16, mixed mode, sharing)

C:\Isle Systems Java\Aspect Hello>java -XX:+AggressiveHeap -Xms64m -Xmx64m -Xboo
tclasspath/p:alpn-boot-8.1.11.v20170118.jar  -jar ies_svr_aspect_hello.jar
Java HotSpot(TM) Client VM warning: MaxNewSize (268800k) is equal to or greater
than the entire heap (65536k).  A new max generation size of 65280k will be used
.

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler i
n thread "Thread-10"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler i
n thread "C3P0PooledConnectionPoolManager[identityToken->1hge0w29u1qz6etl1rbsuex
|aa7bc2]-HelperThread-#2"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler i
n thread "Thread-11"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler i
n thread "qtp13097502-17"
Unexpected error (113) returned by AddToSystemClassLoaderSearch
Unable to add C:\Program Files (x86)\Java\jre1.8.0_152\lib\management-agent.jar
to system class path - not supported by system class loader or configuration err
or!

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler i
n thread "C3P0PooledConnectionPoolManager[identityToken->1hge0w29u1qz6etl1rbsuex
|aa7bc2]-AdminTaskTimer"

From my log file for the server I have the following:

B4X:

2018-04-20 19:43:45.269:AC7452A84E28EB6DAFA24427D74C5CF4:IES_CLASS:ping:Login_Credentials
2018-04-20 19:43:45.441:AC7452A84E28EB6DAFA24427D74C5CF4:IES_CLASS:ping:WebSocket_Disconnected, Processing Time 328 ms
2018-04-20 19:44:04.019:AC7452A84E28EB6DAFA24427D74C5CF4:IES_CLASS:login:Logoff_Credentials
2018-04-20 19:44:04.300:AC7452A84E28EB6DAFA24427D74C5CF4:IES_CLASS:login:WebSocket_Disconnected, Processing Time 406 ms
Apr 22, 2018 9:06:20 PM com.mchange.v2.async.ThreadPoolAsynchronousRunner
WARNING: An Error forced the closing of Thread[C3P0PooledConnectionPoolManager[identityToken->1hge0w29u1qz6etl1rbsuex|aa7bc2]-HelperThread-#1,5,main]. Will attempt to reconstruct, but this might mean that something bad is happening.
java.lang.OutOfMemoryError: Java heap space
   at sun.security.ssl.InputRecord.<init>(Unknown Source)
   at sun.security.ssl.SSLSocketImpl.waitForClose(Unknown Source)
   at sun.security.ssl.SSLSocketImpl.closeSocket(Unknown Source)
   at sun.security.ssl.SSLSocketImpl.closeInternal(Unknown Source)
   at sun.security.ssl.SSLSocketImpl.close(Unknown Source)
   at com.microsoft.sqlserver.jdbc.TDSChannel.disableSSL(IOBuffer.java:705)
   at com.microsoft.sqlserver.jdbc.TDSChannel.close(IOBuffer.java:2026)
   at com.microsoft.sqlserver.jdbc.SQLServerConnection.close(SQLServerConnection.java:2760)
   at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:1559)
   at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:866)
   at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:569)
   at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
   at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
   at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
   at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
   at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
   at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
   at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
   at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
   at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)

java.lang.OutOfMemoryError: Java heap space
   at sun.security.ssl.InputRecord.<init>(Unknown Source)
   at sun.security.ssl.SSLSocketImpl.waitForClose(Unknown Source)
   at sun.security.ssl.SSLSocketImpl.closeSocket(Unknown Source)
   at sun.security.ssl.SSLSocketImpl.closeInternal(Unknown Source)
   at sun.security.ssl.SSLSocketImpl.close(Unknown Source)
   at com.microsoft.sqlserver.jdbc.TDSChannel.disableSSL(IOBuffer.java:705)
   at com.microsoft.sqlserver.jdbc.TDSChannel.close(IOBuffer.java:2026)
   at com.microsoft.sqlserver.jdbc.SQLServerConnection.close(SQLServerConnection.java:2760)
   at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:1559)
   at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:866)
   at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:569)
   at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
   at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
   at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
   at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
   at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
   at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
   at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
   at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
   at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)

java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
   at java.lang.reflect.Method.invoke(Unknown Source)
   at anywheresoftware.b4j.object.JServlet.createInstance(JServlet.java:62)
   at anywheresoftware.b4j.object.BackgroundWorkersManager$1.run(BackgroundWorkersManager.java:21)
   at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
   at anywheresoftware.b4a.BA.raiseEvent2(BA.java:119)
   at anywheresoftware.b4a.objects.Timer$TickTack$1.run(Timer.java:118)
   at anywheresoftware.b4a.keywords.SimpleMessageLoop.runMessageLoop(SimpleMessageLoop.java:30)
   at anywheresoftware.b4a.StandardBA.startMessageLoop(StandardBA.java:26)
   at anywheresoftware.b4a.keywords.Common.StartMessageLoop(Common.java:148)
   at ies.svr.aspect.hello.hello_helper._initialize(hello_helper.java:77)
   ... 7 more
Caused by: java.lang.OutOfMemoryError: Java heap space
   at java.util.jar.Manifest$FastInputStream.<init>(Unknown Source)
   at java.util.jar.Manifest$FastInputStream.<init>(Unknown Source)
   at java.util.jar.Manifest.read(Unknown Source)
   at java.util.jar.Manifest.<init>(Unknown Source)
   at java.util.jar.JarFile.getManifestFromReference(Unknown Source)
   at java.util.jar.JarFile.getManifest(Unknown Source)
   at sun.misc.URLClassPath$JarLoader$2.getManifest(Unknown Source)
   at java.net.URLClassLoader.defineClass(Unknown Source)
   at java.net.URLClassLoader.access$100(Unknown Source)
   at java.net.URLClassLoader$1.run(Unknown Source)
   at java.net.URLClassLoader$1.run(Unknown Source)
   at java.security.AccessController.doPrivileged(Native Method)
   at java.net.URLClassLoader.findClass(Unknown Source)
   at java.lang.ClassLoader.loadClass(Unknown Source)
   at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
Apr 22, 2018 9:13:48 PM com.mchange.v2.async.ThreadPoolAsynchronousRunner
WARNING: An Error forced the closing of Thread[C3P0PooledConnectionPoolManager[identityToken->1hge0w29u1qz6etl1rbsuex|aa7bc2]-HelperThread-#2,5,main]. Will attempt to reconstruct, but this might mean that something bad is happening.
java.lang.OutOfMemoryError: Java heap space

The version of java is "1.8.0_152" the server appStart code is:

B4X:

Sub AppStart (Args() As String)
  
   Dim res As Int
      
   Try
       appPath = mod_settings.get_app_path
       logPath = mod_settings.get_log_path
      
       mod_settings.load_settings
      
       RedirectOutput(logPath, "ies_svr_aspect_hello.log")
       writelog("AppStart")
      
       ' // setup the SQL server
       res = ConfigureSQL
       If res > 0 Then
           ExitApplication2(res)
       End If
      
      
       ' // configure websocket      
       srvr.Initialize("")
       srvr.AddWebSocket("/login", "login")
       srvr.AddWebSocket("/ping", "ping")
       srvr.AddWebSocket("/core", "core")
       srvr.AddWebSocket("/addressbook", "addressbook")
       srvr.AddBackgroundWorker("hello_helper")
       res = ConfigureSSL(SettingsMap.Get("ssl_port"))
       If res > 0 Then
           ExitApplication2(res)
       End If      
       srvr.Http2Enabled = True
       srvr.Start  
      
       ' // configure console port
       svrConsole.Initialize(SettingsMap.Get("console_port"),"svrConsole")
       If Not(console_listen) Then
           ExitApplication2(3)
       End If
      
       ' // start the message loop
       StartMessageLoop
      
   Catch
       writelog("Error " & LastException.Message)
       ExitApplication2(999)
   End Try  
  
  
  
End Sub

The app has been running for over 67 hours and is still running but non contactable from the B4A client and no activity is shown on the server when a connection request from the client is made, the client times out correctly. Tthese are the settings for the server:

heartbeat=1
sql_pool=100
console_port=5001
ttl=20
sql_host=//127.0.0.1\\VM_PPS_TEST
sql_usr=mlEFnB+TGY6JgC0BhBGVl/DU9yJJQXncpQxvJm5DhgcVhBFBBa8SMg\=\=
keystore=C\:\\Isle Systems Data\\Certs\\jetty.keystore
sql_pwd=yVn0gVUBZkMuutsmwN4BZI4DKIw5DrVHL+bIGVwF6LzrUVNLyDvN8Q\=\=
ssl_port=5000

Any advice or guidelines on settings memory etc would be appreciated, also trapping these kind of errors. Here is the ping class, all class's are pretty much the sames format, I have included this in case I am doing something incorrect with the thread pool:

B4X:

'WebSocket class
Sub Class_Globals
  
   Private sql                   As SQL
   Private ws                    As WebSocket  
   Private session_id_client   As String
   Private startTime            As Long = DateTime.Now
   Private tmrKill               As Timer
  
  
End Sub

Public Sub Initialize
  
   DateTime.DateFormat = "yyyy-MM-dd"
   DateTime.TimeFormat = "HH:mm:ss.SSS"
   tmrKill.Initialize("tmrKill",30000)
  
End Sub

Private Sub WebSocket_Connected (WebSocket1 As WebSocket)
  
   Try
       ws = WebSocket1
       ws.RunFunction("ServerReady", Array As Object("READY"))
       ws.Flush
       tmrKill.Enabled = True
   Catch
       writelog("WebSocket_Connected, error - " & LastException)
       close_socket
   End Try

End Sub

Sub tmrKill_Tick
  
   writelog("tmrKill_Tick, Killing Connection")
   tmrKill.Enabled = False
   close_socket
  
End Sub

Sub getSqlConnection
  
  
   Try
       ' // get a connection to the system db
       sql = Main.DB_SQL.GetConnection      
   Catch
       writelog("getSqlConnection, error - " & LastException)
       close_socket
   End Try
  
  
End Sub

Sub closeSqlConnection
  
   Try
       ' // get a connection to the system db
       sql.Close      
   Catch
       writelog("closeSqlConnection, error - " & LastException)
       close_socket
   End Try
  
  
End Sub

Sub close_socket
  
   closeSqlConnection
   ws.Close
  
End Sub


Sub update_ttl
  
      
   Dim rs As ResultSet, sSQL As String
   Dim valid As Boolean
  
   valid = True
   ' // get a SQL connection
   getSqlConnection
   ' // do we have a connection
   If sql.IsInitialized Then
       Try
           ' // check f this user is online
           rs = sql.ExecQuery("SELECT public_id_device FROM [IS_PRESENCE].dbo.presence_hello " & _
                               "WHERE session_id = '" & session_id_client & "'")
          
           If rs.NextRow Then
               ' // update the TTL for this device
               Dim ttl As Long
               ' // TTL =  heatbeat freq * 3 + now
               ttl = DateTime.Now + ((Main.SettingsMap.Get("heartbeat") * DateTime.TicksPerMinute)*3) + DateTime.TicksPerMinute
               sql.BeginTransaction
               sSQL = "UPDATE [IS_PRESENCE].dbo.presence_hello " & _
                   "SET ttl = '" & DateTime.Date(ttl) & " " & DateTime.time(ttl) & "' " & _
                   "WHERE session_id = '" & session_id_client & "'"
                  
               sql.ExecNonQuery(sSQL)
               sql.TransactionSuccessful
               valid= True
           Else
               writelog("update_ttl() session is no longer valid " & session_id_client)
               valid= False
           End If
           rs.close
          
       Catch
           valid= False
           writelog("update_ttl() error = " & LastException.Message)
       End Try

   Else
       writelog("update_ttl() sql.IsInitialized = " & sql.IsInitialized)
       valid= False
   End If
  
   ' // close connection to sql
   closeSqlConnection
  
   If Not(valid) Then
       ' // send
       ws.RunFunction("ServerPing", Array As Object("FAILED"))
   Else
       ' // send
       ws.RunFunction("ServerPing", Array As Object("PINGOK"))
   End If
   ws.Flush

  
End Sub


Sub Login_Credentials(Params As Map)
  
   session_id_client = Params.Get("session_id")
   writelog("Login_Credentials")
   If session_id_client.Length > 0  Then
       CallSubDelayed(Me,"update_ttl")
   Else
       close_socket
   End If
  
  
End Sub

Private Sub WebSocket_Disconnected
  
   tmrKill.Enabled = False
   writelog("WebSocket_Disconnected, Processing Time " & (DateTime.Now - startTime) & " ms")
  
End Sub


Sub writelog(pData As String)
  
   DateTime.DateFormat = "yyyy-MM-dd"
   DateTime.TimeFormat = "HH:mm:ss.SSS"
  
   Log(DateTime.date(DateTime.Now) & " " & DateTime.Time(DateTime.Now) & ":" & session_id_client & ":IES_CLASS:ping:" & pData)
  
End Sub

Thanks in advance

John.

OliverA · Apr 24, 2018

Jmu5667 said:
Ah man, the string values change so a reference to the same string is not good ? Or am I missing somethin

Your string changes constantly. Each call to AddNonQueryBatch has different public_id and public_device_id. You see, the string is created with the different Id's before it is passed to the underlying routine. The routine does not get the rs.GetString methods, rather those methods are executed and a new string created before the new string is passed to the underlying routine of AddNonQueryBatch. Therefore, each call to AddNonQueryBatch has a different string. I, on the other hand, use the same string. Nothing changes, only the parameters that are passed as a different argument.

Wish: https://www.b4x.com/android/forum/t...nt-addnonquery2args-execnonquery2batch.92290/

BTW: Funny how both of us were thinking about the underlying code of AddNonQueryBatch.

Note 2: And that's why code optimization at times may be done in the wrong place. SQL calls are very expensive, so the small changes done here have a minuscule impact in comparison, and therefore a may be considered a nit-pick (or waste of time).

Jmu5667 · Apr 24, 2018

OliverA said:
Your string changes constantly. Each call to AddNonQueryBatch has different public_id and public_device_id. You see, the string is created with the different Id's before it is passed to the underlying routine. The routine does not get the rs.GetString methods, rather those methods are executed and a new string created before the new string is passed to the underlying routine of AddNonQueryBatch. Therefore, each call to AddNonQueryBatch has a different string. I, on the other hand, use the same string. Nothing changes, only the parameters that are passed as a different argument.

Wish: https://www.b4x.com/android/forum/t...nt-addnonquery2args-execnonquery2batch.92290/

BTW: Funny how both of us were thinking about the underlying code of AddNonQueryBatch.

Note 2: And that's why code optimization at times may be done in the wrong place. SQL calls are very expensive, so the small changes done here have a minuscule impact in comparison, and therefore a may be considered a nit-pick (or waste of time).

Cool, get it now !

BTW: Great minds think alike, and fools, well we know the answer to that one

EnriqueGonzalez · Apr 24, 2018

to put my grain.

this is not the correct way to write text in B4x:

B4X:

    Dim sSQL1 As String = "UPDATE [IS_SYSTEM].dbo.devices_hello " & _
                   "SET " & _
                   "active_user = '', " & _
                   "active = 0, " & _
                   "last_user = ?, " & _
                   "last_user_exitcode = 2 " & _
                   "WHERE device_id = ?"

use smartstrings

B4X:

    Dim sSQL1 As String = $"
        UPDATE[IS_SYSTEM].dbo.devices_hello
            SET 
                ACTIVE_user = ''
              , Active = 0
              , lastuser = ?
              , last_user_exitcode = 2
            WHERE device_id = ? 
    "$

no it will not add any performance, but will be much easier to pass queries between the IDE and the SQL ide.

Jmu5667 · May 4, 2018

Update: @OliverA

1. Server App Running for 161 hours, contant client connections.

2. java -XX:+UseG1GC -XX:NewSize=64m -XX:MaxNewSize=64m -Xbootclasspath/p:alpn-boot-8.1.11.v20170118.jar -jar ies_svr_aspect_hello.jar

3. Task Manager memory usage seems to have reached 172mb and hovers around this.

I hope this has solved any issues I had, and hope someone else finds it useful.

Regards

John.

B4J Question [RESOLVED] Server Out of Memory Error

Jmu5667

Well-Known Member

OliverA

Expert

Jmu5667

Well-Known Member

EnriqueGonzalez

Well-Known Member

Jmu5667

Well-Known Member

Similar Threads