Upload Progress with Counting Asyncstreams

[thedesolatesoul]

I've picked up a project after a long time (I really dont have time for programming but b4a keeps pulling me back)
I am trying to detect the upload progress but I am not entirely sure how.

This is where the request goes out:

   Dim in As InputStream
   in = File.OpenInput(Dir, FileName)
   UploadLength = in.BytesAvailable 
   PostInputStream.Initialize(in)
   Log("PostStreamCount:" & PostInputStream.Count)
   File.Copy2(in, stream) 'read the file and write it to the stream
   Log("PostStreamCount:" & PostInputStream.Count)
   b = EOL.GetBytes("UTF8")
   stream.WriteBytes(b, 0, b.Length)
   
   b = (EOL & "--" & boundary & "--" & EOL).GetBytes("UTF8")
   stream.WriteBytes(b, 0, b.Length)
   b = stream.ToBytesArray
   
   UpdateFileTransfers(0)
   'PostInputStream.Initialize(stream)
   
   tmrProgress.Enabled = True
   
   Dim req As HttpRequest
   req.InitializePost2(link, b)
   req.SetContentType("multipart/form-data; boundary=" & boundary)
   req.SetContentEncoding("UTF8")
   req.SetHeader("Authorization", authHeader)
   Return req

So I have an input stream opened from a file, and redirected to the output stream which will be posted using File.Copy2

Since File.Copy2 copies all the data immediately (I think!), am I probably better monitoring the output stream as it is posted?

Unfortunately I can do neither and the count always return 0.

 

[Erel]

You should use something like:

   Dim req As HttpRequest
   Dim in As InputStream
   in.InitializeFromBytesArray(b, 0, b.Length)
   Dim countInputStream As CountingInputStream
   countInputStream.Initialize(in)
   req.InitializePost(link, countInputStream, b.Length)

 

[thedesolatesoul]

Hi Erel,
This code is taken from your modified HttpUtils for Dropbox.
Since the stream posted, contains additional data, I think I might need an output stream to pipe it all out?

In short here is the code:

   Dim stream As OutputStream
   Dim size_i As Int 
   stream.InitializeToBytesArray(20)
   PostInputStream.Initialize(stream)
   Log("PostStreamCount:" & PostInputStream.Count)

   Dim EOL As String
   EOL = Chr(13) & Chr(10) 'CRLF constant matches Android end of line character which is chr(10).
   Dim b() As Byte
   b = ("--" & boundary & EOL & "Content-Disposition: form-data; name=" _ 
      & QUOTE & "file" & QUOTE & "; filename=" & QUOTE & FileName & QUOTE _
      & EOL & "Content-Type: " & "application/octet-stream" & EOL & _ 
      "Content-Transfer-Encoding: binary" & EOL & EOL).GetBytes("UTF8")
   'stream.WriteBytes(b, 0, b.Length)
   size_i = b.Length 
   PostInputStream.WriteBytes(b,0,b.Length)
   Log("PostStreamCount:" & PostInputStream.Count)
   Dim in As InputStream
   in = File.OpenInput(Dir, FileName)
   UploadLength = in.BytesAvailable 
   'PostInputStream.Initialize(in)
   'File.Copy2(in, stream) 'read the file and write it to the stream
   File.Copy2(in, PostInputStream)
   Log("PostStreamCount:" & PostInputStream.Count)
   
   b = EOL.GetBytes("UTF8")
   'stream.WriteBytes(b, 0, b.Length)
   PostInputStream.WriteBytes(b, 0, b.Length)
   size_i = size_i + b.Length 
   b = (EOL & "--" & boundary & "--" & EOL).GetBytes("UTF8")
   'stream.WriteBytes(b, 0, b.Length)
   PostInputStream.WriteBytes(b, 0, b.Length)
   size_i = size_i + b.Length
   'b = stream.ToBytesArray
   'b= PostInputStream.ToBytesArray 
   UploadLength = UploadLength + size_i

   tmrProgress.Enabled = True
   
   Dim req As HttpRequest
   req.InitializePost2(link, b)
   'req.InitializePost(link, PostInputStream,UploadLength)
   req.SetContentType("multipart/form-data; boundary=" & boundary)
   req.SetContentEncoding("UTF8")
   req.SetHeader("Authorization", authHeader)
   Return req

So it is the OutputStream 'stream' that is actually posted and consists of some other data apart from the file.
But I do not know how to post the output stream.
InitializePost only allows an input stream.
If I use InitializePost2, it requires a byte array and when I do:
PostInputStream.ToBytesArray
it gives me an error that: java.lang.RuntimeException: ToBytes can only be called after InitializeToBytesArray.

Maybe I can append the extra information in files, create a temporary file, and read that as an input stream.

 

[Erel]

The InputStream is posted. You should wrap the input stream.

 

[thedesolatesoul]

Hi Erel,
Currently, HTTPUtils posts using a ByteArray, you are saying to post using just the InputStream.
Does that mean I do not need to POST all this metadata (Content-Type, Filename, EOL etc)?
I will try it when I get home.

 

[thedesolatesoul]

If I just post the inputstream, the dropbox servers complain:
Error. Url=/mnt/sdcard/2011-06-15 09.53.53.jpg Message=Bad Request
Error: StatusCode=400, file parameter value 'None' is invalid

As they expect a parameter 'file' as part of the multipart upload.

Is there some way I can get the OutputStream (with all the extra parameters) to become an InputStream?

 

[thedesolatesoul]

Okay, if I write out the original stream to file, and then read it back in (just in order to get an InputStream object) it seems to work.
However this is not an ideal solution for me, since uploading large files will have overhead and space issues.
Also, I noticed that downloading the same file takes much longer than uploading? Maybe the InputStream is quick to flush data out to the network and then sits back waiting for a response.

 

[Erel]

You shouldn't write it to a file. You should do something like what I posted in post #2.

 

[thedesolatesoul]

You shouldn't write it to a file. You should do something like what I posted in post #2.

I apologise for not trying that before :sign0013:. I didnt understand that code at first, but now I see what you are trying to say! :BangHead:
I will try this today.
Thanks a lot!!!

 

[thedesolatesoul]

Thanks Erel, that worked
Do I need to close the inputstream later?

 

[Erel]

Do I need to close the inputstream later?

It always safer to explicitly close the stream.

 

[thedesolatesoul]

Thank you