B4J Question [SOLVED] What exactly is FTP TimeoutMs?

[JackKirk]

I'm having some issues with FTP uploads over a low speed link.

If I set FTP TimeoutMS to 30000 does this mean that a file uploaded via FTP UploadFile must complete the entire upload in 30 seconds?

Or does FTP TimeoutMS have a different meaning? - googling "java ftp client timeout" suggests there are several possible timeouts.

Thanks...

 

[Erel]

If I set FTP TimeoutMS to 30000 does this mean that a file uploaded via FTP UploadFile must complete the entire upload in 30 seconds?

No.

As long as data is received, the timeout will never fire. The timeout defines the time that the socket will wait until connected or while reading data, until data is received.

 

[JackKirk]

Thanks Erel, are there any timeouts hidden under the hood that could affect a slow upload?

 

[Erel]

Which problem are you trying to solve?

 

[JackKirk]

I have a B4J FTP client application running on a headless/legless mini PC in a forest in a fairly remote area.

The mini PC acts as a buffer point between a series of cameras in trees and a fairly low speed wireless Internet connection.

I have had this running for about 2 years and it has been quite stable.

All of a sudden I am having problems - the B4J FTP client application is throwing lots of this:

org.apache.commons.net.io.CopyStreamException: IOException caught while copying.
at org.apache.commons.net.io.Util.copyStream(Util.java:130)
at org.apache.commons.net.ftp.FTPClient.__storeFile(FTPClient.java:574)
at org.apache.commons.net.ftp.FTPClient.storeFile(FTPClient.java:1716)
at anywheresoftware.b4a.net.FTPWrapper$2.run(FTPWrapper.java:239)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)

I initially assumed that I had an equipment failure but a bit of jugging of equipment seems to have dispelled this.

I am currently working on the premise that the link has deteriorated to an extent that some timeout is being triggered.

Hence my questions re timeouts.

BTW I also use a modified form of the B4J FTP server on the other end and I see there is at least 1 timeout visible there.

Any thoughts are gladly accepted - e.g. is there any way to find out what is triggering the rather uninformative low level message I'm getting?

Thanks...

 

[Erel]

You should expect network problems to occur. Especially in the conditions you describe.

The best way to deal with it is by initializing the FTP object again and then retry to upload the file after X seconds. It is quite simple to implement such algorithms with a loop + Sleep.

 

[JackKirk]

Erel, thanks for the confirmation of the approach I have already developed in my B4J FTP client application - any upload job in it that bombs is automatically returned to the upload queue for another go - 5 goes and it is put aside.

Recently the requeuing has been of such a frequency that virtually no job was getting uploaded - it was almost literally spinning its digital wheels.

I am currently pursuing some testing of blowing out the timeout in the FTP_Client module of the B4J FTP server code which looks like it is working - more testing and some hardware jiggling yet to be done.

Is there any low level code that could be added to determine the cause of the log messages I am getting - if this confirmed the error message was due to the server terminating the job then it would help somewhat.

Thanks...

 

[JackKirk]

I am currently pursuing some testing of blowing out the timeout in the FTP_Client module of the B4J FTP server code which looks like it is working - more testing and some hardware jiggling yet to be done.

OK it turns out that the said mentioned timeout in the FTP_Client module of the B4J FTP server code is the culprit - if it is set large there are no problems, set small and there are problems.

It appears there was a short term problem at the other end of the link that was causing a backlog of upload jobs to be uploaded which meant that the number of simultaneous upload jobs (limited to 16 in my B4J FTP client application, but rarely approached) was causing all jobs to be more slowly uploaded than normal and some were running foul of the timeout.

I would still be keen on knowing of any low level code that could be added to determine the cause of the log messages I was getting.

 

[Erel]

I would still be keen on knowing of any low level code that could be added to determine the cause of the log messages I was getting.

Might be possible if you modify the library source code.

 

[JackKirk]

Might be possible if you modify the library source code.

Not in my repertoire unfortunately...