Upload Progress with Counting Asyncstreams

thedesolatesoul · Jan 28, 2012

I've picked up a project after a long time (I really dont have time for programming but b4a keeps pulling me back)
I am trying to detect the upload progress but I am not entirely sure how.

This is where the request goes out:

B4X:

   Dim in As InputStream
   in = File.OpenInput(Dir, FileName)
   UploadLength = in.BytesAvailable 
   PostInputStream.Initialize(in)
   Log("PostStreamCount:" & PostInputStream.Count)
   File.Copy2(in, stream) 'read the file and write it to the stream
   Log("PostStreamCount:" & PostInputStream.Count)
   b = EOL.GetBytes("UTF8")
   stream.WriteBytes(b, 0, b.Length)
   
   b = (EOL & "--" & boundary & "--" & EOL).GetBytes("UTF8")
   stream.WriteBytes(b, 0, b.Length)
   b = stream.ToBytesArray
   
   UpdateFileTransfers(0)
   'PostInputStream.Initialize(stream)
   
   tmrProgress.Enabled = True
   
   Dim req As HttpRequest
   req.InitializePost2(link, b)
   req.SetContentType("multipart/form-data; boundary=" & boundary)
   req.SetContentEncoding("UTF8")
   req.SetHeader("Authorization", authHeader)
   Return req

So I have an input stream opened from a file, and redirected to the output stream which will be posted using File.Copy2

Since File.Copy2 copies all the data immediately (I think!), am I probably better monitoring the output stream as it is posted?

Unfortunately I can do neither and the count always return 0.

Erel · Jan 29, 2012

You should use something like:

B4X:

   Dim req As HttpRequest
   Dim in As InputStream
   in.InitializeFromBytesArray(b, 0, b.Length)
   Dim countInputStream As CountingInputStream
   countInputStream.Initialize(in)
   req.InitializePost(link, countInputStream, b.Length)

thedesolatesoul · Jan 29, 2012

Hi Erel,
This code is taken from your modified HttpUtils for Dropbox.
Since the stream posted, contains additional data, I think I might need an output stream to pipe it all out?

In short here is the code:

B4X:

   Dim stream As OutputStream
   Dim size_i As Int 
   stream.InitializeToBytesArray(20)
   PostInputStream.Initialize(stream)
   Log("PostStreamCount:" & PostInputStream.Count)

   Dim EOL As String
   EOL = Chr(13) & Chr(10) 'CRLF constant matches Android end of line character which is chr(10).
   Dim b() As Byte
   b = ("--" & boundary & EOL & "Content-Disposition: form-data; name=" _ 
      & QUOTE & "file" & QUOTE & "; filename=" & QUOTE & FileName & QUOTE _
      & EOL & "Content-Type: " & "application/octet-stream" & EOL & _ 
      "Content-Transfer-Encoding: binary" & EOL & EOL).GetBytes("UTF8")
   'stream.WriteBytes(b, 0, b.Length)
   size_i = b.Length 
   PostInputStream.WriteBytes(b,0,b.Length)
   Log("PostStreamCount:" & PostInputStream.Count)
   Dim in As InputStream
   in = File.OpenInput(Dir, FileName)
   UploadLength = in.BytesAvailable 
   'PostInputStream.Initialize(in)
   'File.Copy2(in, stream) 'read the file and write it to the stream
   File.Copy2(in, PostInputStream)
   Log("PostStreamCount:" & PostInputStream.Count)
   
   b = EOL.GetBytes("UTF8")
   'stream.WriteBytes(b, 0, b.Length)
   PostInputStream.WriteBytes(b, 0, b.Length)
   size_i = size_i + b.Length 
   b = (EOL & "--" & boundary & "--" & EOL).GetBytes("UTF8")
   'stream.WriteBytes(b, 0, b.Length)
   PostInputStream.WriteBytes(b, 0, b.Length)
   size_i = size_i + b.Length
   'b = stream.ToBytesArray
   'b= PostInputStream.ToBytesArray 
   UploadLength = UploadLength + size_i

   tmrProgress.Enabled = True
   
   Dim req As HttpRequest
   req.InitializePost2(link, b)
   'req.InitializePost(link, PostInputStream,UploadLength)
   req.SetContentType("multipart/form-data; boundary=" & boundary)
   req.SetContentEncoding("UTF8")
   req.SetHeader("Authorization", authHeader)
   Return req

So it is the OutputStream 'stream' that is actually posted and consists of some other data apart from the file.
But I do not know how to post the output stream.
InitializePost only allows an input stream.
If I use InitializePost2, it requires a byte array and when I do:
PostInputStream.ToBytesArray
it gives me an error that: java.lang.RuntimeException: ToBytes can only be called after InitializeToBytesArray.

Maybe I can append the extra information in files, create a temporary file, and read that as an input stream.

Erel · Jan 30, 2012

The InputStream is posted. You should wrap the input stream.

thedesolatesoul · Jan 30, 2012

Hi Erel,
Currently, HTTPUtils posts using a ByteArray, you are saying to post using just the InputStream.
Does that mean I do not need to POST all this metadata (Content-Type, Filename, EOL etc)?
I will try it when I get home.

thedesolatesoul · Jan 30, 2012

If I just post the inputstream, the dropbox servers complain:
Error. Url=/mnt/sdcard/2011-06-15 09.53.53.jpg Message=Bad Request
Error: StatusCode=400, file parameter value 'None' is invalid

As they expect a parameter 'file' as part of the multipart upload.

Is there some way I can get the OutputStream (with all the extra parameters) to become an InputStream?

thedesolatesoul · Jan 30, 2012

Okay, if I write out the original stream to file, and then read it back in (just in order to get an InputStream object) it seems to work.
However this is not an ideal solution for me, since uploading large files will have overhead and space issues.
Also, I noticed that downloading the same file takes much longer than uploading? Maybe the InputStream is quick to flush data out to the network and then sits back waiting for a response.

Erel · Jan 31, 2012

You shouldn't write it to a file. You should do something like what I posted in post #2.

thedesolatesoul · Jan 31, 2012

Erel said:
You shouldn't write it to a file. You should do something like what I posted in post #2.

I apologise for not trying that before :sign0013:. I didnt understand that code at first, but now I see what you are trying to say! :BangHead:
I will try this today.
Thanks a lot!!!

thedesolatesoul · Jan 31, 2012

Thanks Erel, that worked
Do I need to close the inputstream later?

Erel · Feb 1, 2012

Do I need to close the inputstream later?

It always safer to explicitly close the stream.

thedesolatesoul · Feb 1, 2012

Thank you

Upload Progress with Counting Asyncstreams

thedesolatesoul

Expert

Erel

B4X founder

thedesolatesoul

Expert

Erel

B4X founder

thedesolatesoul

Expert

thedesolatesoul

Expert

thedesolatesoul

Expert

Erel

B4X founder

thedesolatesoul

Expert

thedesolatesoul

Expert

Erel

B4X founder

thedesolatesoul

Expert

Upload Progress with Counting Asyncstreams

Expert

B4X founder

Expert

B4X founder

Expert

Expert

Expert

B4X founder

Expert

Expert

B4X founder

Expert

Privacy & Transparency

Privacy & Transparency