Android Question [Solved] HttpJob error with unexpected end of stream

max123

Well-Known Member
Licensed User
Longtime User
Hi all,

I started on my PC an http server to serve a directory.
In Windows prompt I write this command: (for convenience I use python command line)
PS C:\Users\Massimo\Downloads\three.js-master> python -m http.server
and python reply this:
Serving HTTP on :: port 8000 (http://[::]:8000/) ...
started to serving a three.js-master directory on localhost:8000 as default port.

I just used python command line to test, but next I will use B4J with http server.

Now I want download a file from B4A with OkHttpUtils2 and HttpJob.

Because I will test on Android Emulator, initially I had troubles because I've used 'localhost' or 127.0.0.1 in my requests,
and this cannot work because it refer to the same android device and not to a localhost of host machine.

I works a lot with networks, but I learned this on this thread thanks to @aeric tip.
https://www.b4x.com/android/forum/threads/httpjob-not-connecting-to-localhost.145737/#post-989752
Now that I remember I had the same issue on VirtualBox in past.

As expected this don't worked because different loopbacks and python server on host machine always refused to connect.

Now, searching on the web, I found a tip to use 10.0.2.2 as IP loopback to connect the emulator to the host machine.
This works now, and the emulator can connect to the python server started on the host machine.

Here a code:
B4X:
Private Sub Button1_Click
    DownloadAndSaveFile ("http://10.0.2.2:8000/examples/models/gcode/benchy.gcode")
End Sub

Sub DownloadAndSaveFile (Link As String)
    Dim j As HttpJob
    j.Initialize("", Me)
    j.Download(Link)
 
'    j.GetRequest.SetHeader("Access-Control-Allow-Origin", "true")
'    j.GetRequest.SetHeader ("ContentType", "text/plain")
    j.GetRequest.SetHeader ("ContentType", "application/octet-stream")
'    j.GetRequest.SetHeader ("Host", "exampleproject.com")
'    j.GetRequest.SetHeader ("Connection", "close")
 
    Wait For (j) JobDone(j As HttpJob)
 
    Dim map As Map = j.Response.GetHeaders
    For i = 0 To map.Size - 1
        Log("HEADER: " & map.GetKeyAt(i) & ": " & map.GetValueAt(i))
    Next
 
    Log(" ")
    Log("Success: " & j.Success)
    Log("StatusCode: " & j.Response.StatusCode)
    Log("ErrorResponse: " & j.Response.ErrorResponse)
    Log(" ")
    
    If j.Success Then
        Dim out As OutputStream = File.OpenOutput(File.DirInternal, "benchy.gcode", False)
        File.Copy2(j.GetInputStream, out)
        out.Close '<------ very important
        Log("DONE")
    Else
        Log("ERROR: " & j.ErrorMessage)
    End If
    j.Release
 
    For Each f As String In File.ListFiles(File.DirInternal)
        Log("File: " & f)
    Next
End Sub
Now it can connect to the server and I see the log on the command prompt that show served file:
::ffff:127.0.0.1 - - [24/May/2024 20:56:28] "GET /examples/models/gcode/benchy.gcode HTTP/1.1" 200 -
::ffff:127.0.0.1 - - [24/May/2024 21:22:17] "GET /examples/models/gcode/benchy.gcode HTTP/1.1" 200 -
::ffff:127.0.0.1 - - [24/May/2024 21:24:16] "GET /examples/models/gcode/benchy.gcode HTTP/1.1" 200 -
::ffff:127.0.0.1 - - [24/May/2024 21:25:22] "GET /examples/models/gcode/benchy.gcode HTTP/1.1" 200 -
::ffff:127.0.0.1 - - [25/May/2024 09:55:52] "GET /examples/models/gcode/benchy.gcode HTTP/1.1" 200 -
And this is the B4A log, here the http code is 200, it even have content length:
** Activity (main) Pause, UserClosed = false **
** Activity (main) Create (first time) **
** Activity (main) Resume **
*** Receiver (httputils2service) Receive (first time) ***
HEADER: content-length: [1940664]
HEADER: content-type: [application/octet-stream]
HEADER: date: [Sat, 25 May 2024 11:14:24 GMT]
HEADER: last-modified: [Fri, 26 Apr 2024 07:52:13 GMT]
HEADER: server: [SimpleHTTP/0.6 Python/3.11.5]

Success: false
StatusCode: 200
ErrorResponse:

ERROR: java.net.ProtocolException: unexpected end of stream
File: virtual_assets
The problem now is that the HttpJob fails with this error:
ERROR: java.net.ProtocolException: unexpected end of stream
I've tried to manage headers but the result not changed.
Note that I placed in the manifest this:
B4X:
' 28 - Non-ssl (non-https) communication is not permitted by default.
' It can be enabled in B4A v9+ by adding this line to the manifest editor:
CreateResourceFromFile(Macro, Core.NetworkClearText)

Please, can someone help me to know what happen here and because this error appear ?

Thanks
 
Last edited:

aeric

Expert
Licensed User
Longtime User
I found a tip to use 10.0.2.2 as IP loopback to connect the emulator to the host machine.
My emulator has different IP (10.0.2.16). 10.0.2.2 is the gateway or localhost of PC. If you run a webserver in PC, it can reach the page.

1717253847827.png
1717255069144.png


Your example works even on Emulator, just put http://1.0.2.16:5566 in the emulator browser.
Yes, it works with the emulator IP.
 
Last edited:
Upvote 0

max123

Well-Known Member
Licensed User
Longtime User
Now your code looks very good. It works for benchy.gcode.
So the server is still serving the files, probably it should work with Threejs distribution like that.

The actual test I do that are working for me, is when I launch the B4J server, then access to the page with browser, then access from here the /examples pages, here there are html files that just works and any required file is served from server.

So now B4A can serve it, please let me experiment with a full threejs distribution (removing big and unused files) and by loading some html files (after I start HttpServer) to see if files are correctly served. With python I see served files in the Windows command prompt when I load html files in the browser.

Thanks for your help and interest, I will experiment it.
 
Last edited:
Upvote 0

max123

Well-Known Member
Licensed User
Longtime User
My emulator has different IP (10.0.2.16). 10.0.2.2 is the gateway.

View attachment 154243


Yes, it works with the emulator IP.
I tried it and do not worked with 10.0.2.2. I've read that this should be the loopback to the host machine, so if you point 10.0.2.2, it point to 127.0.0.1, or 'localhost' from host pc machine. This worked to me when I launched Python or B4J server on PC and accessed from emulator.
I have a question for you now that you posted emulator screen.... how did you get android emulator frame? My emulator don't have a frame around it, yhis is a 10" tablet, but even with 4" do not worked.
Did you downloaded some special files from SDK manager ? Or may is the emulator with Playstore and Google Services that do not support io ?
** Activity (main) Create (first time) **
2024-06-01 15:04:09.040:INFO::jetty-7.x.y-SNAPSHOT
2024-06-01 15:04:09.123:INFO::started o.e.j.s.ServletContextHandler{/,null}
2024-06-01 15:04:09.266:INFO::Started BlockingChannelConnector@0.0.0.0:8000 STARTING
org.eclipse.jetty.server.Server@369b0d8
Server started
Ip address: 10.0.2.16
** Activity (main) Resume **
ready
DELETED
Start download...
*** Receiver (httputils2service) Receive (first time) ***
ResponseError. Reason: java.net.SocketTimeoutException: failed to connect to /10.0.2.2 (port 8000) from /10.0.2.16 (port 51822) after 30000ms, Response:
ERROR: java.net.SocketTimeoutException: failed to connect to /10.0.2.2 (port 8000) from /10.0.2.16 (port 51822) after 30000ms
1717255430064.png

We have same settings, 10.0.2.2 do not works for me, I've to use 10.0.2.16. Now I'm curious, it works for you ?
 
Last edited:
Upvote 0

aeric

Expert
Licensed User
Longtime User
how did you get android emulator frame?
You should search the forum or post a new thread. :)

You can use Emulator skin.
I have a github repo forked from someone else.

You can also check a tutorial by mcqueccu

video:

My emulator don't have a frame around it, yhis is a 10" tablet, but even with 4" do not worked.
Edit: I usually use 6" device.
 
Upvote 0

max123

Well-Known Member
Licensed User
Longtime User
I confirm both these works to download file from emulator:
B4X:
DownloadAndSaveFile ("http://localhost:8000/examples/models/gcode/benchy.gcode")
DownloadAndSaveFile ("http://127.0.0.1:8000/examples/models/gcode/benchy.gcode")
So no need to use 10.0.2.16 or put the real IP address on real device.
This is if you want access a B4J server started on host machine from emulator.
B4X:
DownloadAndSaveFile ("http://10.0.2.2:8000/examples/models/gcode/benchy.gcode")
 
Upvote 0

max123

Well-Known Member
Licensed User
Longtime User
Hi @aeric , and all other members, I tried to port jServer to B4A.

I searched to adapt on Android, the most I changed is a Runnable that looks different.
Now the library project under Eclipse is near to be compiled, just I have to solve small things, one of this is that the code manage StartMessageLoop, inexistent on Android.

Here a relevant part of code that I do not know how to change:
Java:
        private void runDisconnect(boolean messageLoopIsRunning) {
            if (ba != null) {
                if (messageLoopIsRunning) {
      
//                    ba.postRunnable(new Runnable() {  // B4J
//                        @Override
//                        public void run() {
//                            if (ba != null) {
//                                if (disconnectRun == false) {
//                                    ba.raiseEvent(null, "websocket_disconnected");
//                                    if (parentServlet.singleThread == false)
//                                        Common.StopMessageLoop(ba);
//                                    disconnectRun = true;
//                                }
//                                ba = null;
//                            }
//                        }
//                    });
      
                    Runnable r = new Runnable() {   // B4A    //// CHANGED ////
                         @Override
                         public void run() {
                            if (ba != null) {
                                if (disconnectRun == false) {
                                    ba.raiseEvent(null, "websocket_disconnected");
                                    if (parentServlet.singleThread == false)
                                        Common.StopMessageLoop(ba);  // <<<<<<<<<<<<<<<<<<
                                    disconnectRun = true;
                                }
                                ba = null;
                            }
                         }           
                      };
                      BA.submitRunnable(r, this, 0);
      
                } else {
                    if (disconnectRun == false) {
                        ba.raiseEvent(null, "websocket_disconnected");
                        disconnectRun = true;
                        ba = null;
                    }
                }
            }
        }
        public class ThreadHandler implements Runnable {
            @Override
            public void run() {
                try {
                    B4AClass handler = JServlet.createInstance(parentServlet.handlerClass, parentServlet.initializeMethod);
                    ba = handler.getBA();
                    ws = new WebSocket();
                    if (parentServlet.singleThread == false)
                        ba.cleanMessageLoop();
                    ws.adapter = Adapter.this;
                    ws.session = getSession();
                    ws.singleThread = parentServlet.singleThread;
                    ws.setEvents(ba);
                    ws.setJQElements(handler);
                    ba.raiseEvent(null, "websocket_connected", ws);
                    ws.Flush();
                    if (parentServlet.singleThread == false) {
                        ba.startMessageLoop();  // <<<<<<<<<<<<<<<
                    }
                } catch (Exception e) {
                    e.printStackTrace();
                    try {
                        runDisconnect(false);
                        if (getSession() != null && getSession().isOpen()) {
                            getSession().close();
                        }
                    } catch (Exception ee) {
                        ee.printStackTrace();
                    }
                }
            }
        }
Here I tried to adapt a Runnable too.... I used this on other positions in the code.

I even have this that do not have a counterpart on Android:
Java:
   @Override
        public void onWebSocketConnect(Session sess) {
            super.onWebSocketConnect(sess);
            ((JettyWebSocketRemoteEndpoint)sess.getRemote()).setBatchMode(BatchMode.ON);

            handler = new ThreadHandler();

            if (parentServlet.singleThread) {
                BA.firstInstance.postRunnable(handler);  // <<<<<<<< I do not found a counterpart for it in the B4A Core
            }  else {
                Servlet.pool.submit(handler);
            }
And this because different SQL implementation:
Java:
    /**
     * Retrieves a connection from the pool. Make sure to close the connection when you are done with it.
     */
    public SQL GetConnection() throws SQLException {
        SQL s = new SQL();
        s.connection = getObject().getConnection();        
        return s;
    }
    /**
     * Asynchronously gets a SQL connection. This method is useful in UI applications as it prevents the main thread from freezing until the connection is available.
     *The ConnectionReady event will be raised when the connection is ready.
     */
    public void GetConnectionAsync(final BA ba, final String EventName) {
        BA.runAsync(ba, this, EventName.toLowerCase(BA.cul) + "_connectionready", new Object[] {false, null}, new Callable<Object[]>() {
            @Override
            public Object[] call() throws Exception {
                return new Object[] {true, GetConnection()};
            }            
        });
    }
    }
After fixing these things the code should compile and tested on B4A.

I started from original jServer code on github, imported all relevant classes for B4X implementation and even imported all jar files placed inside a /B4J/Libraries/jserver folder. This fixed all dependencies from Jetty server. Here, as I wrote, even SQL looks different, I imported the B4A SQL class inline but I'm not pratical using SQL.

Started from here:
https://github.com/AnywhereSoftware/B4J/tree/master/Libs_Server/src/anywheresoftware/b4j/object
 
Last edited:
Upvote 0

aeric

Expert
Licensed User
Longtime User
Good effort but I am not sure is it worth it to further explore this. Maybe Erel can advise. You may also talk to Star-Dust, he has created his own httpserver.

On the SQL part, I don't think jServer should have this dependency. However, I can't confirm this because I don't see the code in github. I am wondering why don't you use the SQL library for B4A?

 
Upvote 0

max123

Well-Known Member
Licensed User
Longtime User
Yes, I used the B4A SQL library, just created an inline class.
I don't need this for my use case, I had to change the code to compile, probably I can remove it.

The only thing I need is the automatic static files service, just in background without code handlers to manage manually.

I tried your last code using HttpServer and a real threejs zip file, this don't worked for me and I cannot longer download the file.

The only things I changed are:
- replaced www.zip with theejs.zip
- extracted it from DirAssets to DirInternal, here I changed the 'www' folder name to 'three' (probably this is the issue and server just expets www ?)
- the benchy.gcode is inside the same folder like in your HttpServer code, just I changed the root folder name as explained

At this time it fails to connect and download a file.
Note that I try to download a file just to test it is really served at exact location, at the end I don't need to manage it manually, just load the html page and any file should be served by a server.
B4X:
    '    Dim url As String = xui.FileUri(File.Combine(ThreeJsPath, "/examples"), "index.html") // file scheme
    Dim url As String = "http://localhost:8000/examples/index.html" ' We switch from file:// scheme to http:// scheme
    Log("Loading web page. Url: " & url)
    WebView1.LoadUrl(url)
This should load the main page that is inside examples folder.

I don't know how to point the HttpServer to serve DirInternal/files/three folder, in the jServer there is StaticFilesFolder method to set it, but in HttpServer I don't see it.
 
Last edited:
Upvote 0

aeric

Expert
Licensed User
Longtime User
I just use folder name "www" as an example. I think you can use other name.
If you use "files" then it should be fine.

Use "For loop" to check the file list for the folder hierarchy.

Make sure the unzip folder from threejs.zip is "three" and not "threejs".
 
Upvote 0

max123

Well-Known Member
Licensed User
Longtime User
'files' is very confusing here, because the path from DirInternal always already have it as last folder, See my first screenshot.

Here you can see that misteriously without change any code now I've succeded to download the full benchy.gcode, about 2 MB with HttpJob.

Now finally I can even load a threejs index.html main page and requested files are served from HttpServer and it's Handler. See second screenshot.
..... but strangely this do not worked for 2 days.

I do not changed anything in the name of folder, it's name is 'three' instead of 'www'.

Here remote address always is [::1] with this line:
B4X:
Log("CLIENT: [" & Request.RemoteAddress & "] - [" & Request.Method & "] => " & Request.RequestURI)
as result:
Request.RequestURI: files/RobotoMono-Regular.woff2
Server request file path exists ??? => true
Serve request file: /data/user/0/b4a.http.server.threejs/files/three/files/RobotoMono-Regular.woff2
PAGE FINISHED: http://localhost:8000/examples/index.html
CLIENT: [::1] - [GET] => /examples/tags.json
ContentType: application/octet-stream
Request.RequestURI: examples/tags.json
Server request file path exists ??? => true
Serve request file: /data/user/0/b4a.http.server.threejs/files/three/examples/tags.json
CLIENT: [::1] - [GET] => /files/favicon.ico
ContentType: image/vnd.microsoft.icon
Request.RequestURI: files/favicon.ico
Server request file path exists ??? => true
Serve request file: /data/user/0/b4a.http.server.threejs/files/three/files/favicon.ico
CLIENT: [::1] - [GET] => /examples/screenshots/webgl_animation_keyframes.jpg
ContentType: image/jpeg
Request.RequestURI: examples/screenshots/webgl_animation_keyframes.jpg
Server request file path exists ??? => true
Serve request file: /data/user/0/b4a.http.server.threejs/files/three/examples/screenshots/webgl_animation_keyframes.jpg
CLIENT: [::1] - [GET] => /examples/screenshots/webgl_animation_skinning_blending.jpg
ContentType: image/jpeg
Request.RequestURI: examples/screenshots/webgl_animation_skinning_blending.jpg
 

Attachments

  • Screenshot 2024-06-04 161503.png
    Screenshot 2024-06-04 161503.png
    175.7 KB · Views: 84
  • Screenshot 2024-06-04 162554.png
    Screenshot 2024-06-04 162554.png
    219 KB · Views: 90
Last edited:
Upvote 0

aeric

Expert
Licensed User
Longtime User
'files' is very confusing here, because the path from DirInternal always already have it as last folder, See my first screenshot.

Here you can see that misteriously without change any code now I've succeded to download the full benchy.gcode, about 2 MB with HttpJob.

Now finally I can even load a threejs index.html main page and requested files are served from HttpServer and it's Handler. See second screenshot.
..... but strangely this do not worked for 2 days.

I do not changed anything in the name of folder, it's name is 'three' instead of 'www'.

Here remote address always is [::1] with this line:
B4X:
Log("CLIENT: [" & Request.RemoteAddress & "] - [" & Request.Method & "] => " & Request.RequestURI)
as result:
What can I say? ?
So you tell me it's magic?
 
Upvote 0

max123

Well-Known Member
Licensed User
Longtime User
No, it's not magic at all, the fact is not for a 2 mb file I've downloaded, but that succesfully serves all files, so the full threejs library is working now.

I searched this for a long time, unless I changed the threejs main code FileLoader.js that is a responsible class to manage all loaders to not use XmlHttpRequest but the old file scheme. This worked but may it is not allowed by Google policy because CORS.

With this now I'm able to develop my B4A and B4J threejs libraries. I've managed with this and is something you don't expect. Please try some demos here.

Finally it works, many thanks.
 
Upvote 0

max123

Well-Known Member
Licensed User
Longtime User
Yes, I close it. But the main problem is never changed, just I had the necessity to have files served.
Hi all,

I started on my PC an http server to serve a directory.
In Windows prompt I write this command: (for convenience I use python command line)
 
Upvote 0

aeric

Expert
Licensed User
Longtime User
Yes, I close it. But the main problem is never changed too mutch, just I had the necessity to have files served.
What is the main problem?
CORS?
I am skeptical to Android as a http server.
In my opinion, it has some limitations as a web server for production use. Maybe you want to consider something else.
 
Upvote 0

max123

Well-Known Member
Licensed User
Longtime User
Threejs do not to have to do something of particular, just need some files served (js import files, texture files, model files and so) when a page is load, so I don't know what you mean.
I don't have to make a public file sharing server or something other....
In past it worked with file:// scheme, now it only works with files served from a web server.

An option is to use a new distribution and hack it to use file scheme. I worked on it and this worked for a long time, or just use an old distribution.
Currently 165 is latest and from 138 it only support XmlHttpRequests. But may it is not supported by Google policy because do not respect CORS policy.

I do not need other from this.
This permit me not only to load files, but even to save them, eg I can manipulate 3D objects, then save in a lots of 3D formats.
 
Last edited:
Upvote 0

aeric

Expert
Licensed User
Longtime User
Threejs do not to have to do something of particular, just need some files served (js import files, texture files, model files and so) when a page is load, so I don't know what you mean.
I don't have to make a public file sharing server or something other....
In past it worked with file:// scheme, now it only works with files served from a web server.

An option is to use a new distribution and hack it to use file scheme. I worked on it and this worked for a long time, or just use an old distribution.
Currently 165 is latest and from 138 it only support XmlHttpRequests. But may it is not supported by Google policy because do not respect CORS policy.

I do not need other from this.
I think you should ask in a new thread.
 
Upvote 0

max123

Well-Known Member
Licensed User
Longtime User
This is not a question, it is an affermation. For sure I have to hack original threejs.js file to do it without serve files over http.
 
Upvote 0

aeric

Expert
Licensed User
Longtime User
An option is to use a new distribution and hack it to use file scheme. I worked on it and this worked for a long time, or just use an old distribution.
Currently 165 is latest and from 138 it only support XmlHttpRequests. But may it is not supported by Google policy because do not respect CORS policy.
I see. I don't follow threejs development.
This is probably problem with CORS in Chrome on desktop. I can't remember in which version it started the restriction.

You can try disable it.

Windows​

Just do follow steps:
  1. Open Start window
  2. Search Run and open it or press Window+ R
  3. Paste chrome.exe --user-data-dir="C://Chrome dev session" --disable-web-security and execute it
This will open a new browser with web security disabled.
You can now access your project in this browser without worrying about the CORS errors.
B4X:
chrome.exe --user-data-dir="C://Chrome dev session" --disable-web-security
https:// medium.com/@beligh.hamdi/run-chrome-browser-without-cors-872747142c61

1717520081054.png


For local server, just make sure you have at least these 3 directories.
1717520163195.png


This is not a question, it is an affermation. For sure I have to hack original threejs.js file to do it without serve files over http.
You may need to check the JS file if you are familiar with JS.
 
Upvote 0

max123

Well-Known Member
Licensed User
Longtime User
@DonManfred , please read before...... noone forced you to read this post.
This is not a question, it is an affermation. For sure I have to hack original threejs.js file to do it without serve files over http.

@aeric Many Thanks, you are more constructive than @DonManfred .
You may need to check the JS file if you are familiar with JS.
I already do it and works, I replaced FileLoader class in a new distribution (inside the main threejs.js) with FileLoader of an older distribution, the last that is working with file:// fetch. For me this works without the server, just pointing to files in DirInternal inside the same 'three' distribution folder that I serve with HttpServer. But this hacking threejs as I explained.

This works perfectly, but just I said that may the Google policy refuses it, and is not a good thing hacking a third part library without permissions if not for personal use.

I really appreciate your help, but may I'm not explained well. I don't have to load theejs page on Chrome, nor in any other external browsers, I have to load it in the B4X WebView.

The problem with CORS on external browser only occours if file:// scheme is used, they are not more permitted.

This is probably problem with CORS in Chrome on desktop. I can't remember in which version it started the restriction.
Any modern browser now works this way with XmlHttpRequest, that is more sicure and permits to update the page and any resource in Async way (internally with ajax) without reload a full page. It is the client side that need to be adapted.
 
Last edited:
Upvote 0
Top