GZip issues

[joneden]

Hi,
I'm using GZip to compress some XML in a SOAP webservice and push down to an Android app. I can get the data to work from a VB app but cannot get it to work when decoding in Android. Code is below. In the VB code it seems that you need to include the length of the orginal byte array of the string, is that needed in Android? I've tried with and without it and no luck

The error that I seem to get is about a magic number not existing...

WebService code:

<WebMethod(Description:="Get various data tables - down only")> _
    Public Function GetGZippedDataTEST() As String
        Dim _encodedString As Byte() = Encoding.UTF8.GetBytes("Test text")
        Dim _memoryStream As MemoryStream = New MemoryStream()
        Dim _gZipStream As GZipStream = New GZipStream(_memoryStream, CompressionMode.Compress, True)
        _gZipStream.Write(_encodedString, 0, _encodedString.Length)
        _memoryStream.Position = 0

        Dim _compressedData As Byte() = New Byte(_memoryStream.Length - 1) {}
        ' Fill the compressed data byte array
        _memoryStream.Read(_compressedData, 0, _compressedData.Length)

        ' Add the length of the original byte array to the start of the output byte array
        Dim _amendedByteArray As Byte() = New Byte(_compressedData.Length + 3) {}
        System.Buffer.BlockCopy(_compressedData, 0, _amendedByteArray, 4, _compressedData.Length)
        System.Buffer.BlockCopy(BitConverter.GetBytes(_encodedString.Length), 0, _amendedByteArray, 0, 4)
        Return Convert.ToBase64String(_amendedByteArray)
    End Function

Android code (code triggers as expected then fails on marked line)

Sub ReactiveDataParser_EndElement (uri As String, name As String, text As StringBuilder)
   If name = "string" Then
      Dim objStringUtils As StringUtils
       Dim objInputByteArray() As Byte
       objInputByteArray = objStringUtils.DecodeBase64(text)
       Dim objInputStream As InputStream
       'objInputStream.InitializeFromBytesArray(objInputByteArray, 0, objInputByteArray.Length)
      objInputStream.InitializeFromBytesArray(objInputByteArray, 4, objInputByteArray.Length-4)

      Dim objCompressedStreams As CompressedStreams
      Dim objDecompressedByteArray() As Byte 
' Code fails on next line
      objDecompressedByteArray = objCompressedStreams.DecompressBytes(objInputByteArray ,"gzip")
      Dim objCompressedString As String 
      objCompressedString= BytesToString(objDecompressedByteArray,0, objDecompressedByteArray.Length, "UTF8")
   End If
End Sub

Thanks for any help

Jon

 

[agraham]

Note that the value you are seeing is -117 (negative) because bytes are signed values in Java. This is actually the value 0x8B which just happens to be 139 (128 + 11) when interpreted as an unsigned value but -117 (-128 + 11) when interpreted as a signed value.

 

[joneden]

Ah so there is at least some logic to why it's happening

How do I stop it from doing that then?

 

[agraham]

You don't need to "stop" anything. The bit pattern you sent has been received correctly. It is a matter of how that value is interpreted. http://www.b4x.com/forum/additional...ates/8942-ioio-board-library-5.html#post62516

 

[joneden]

OK I've got you.

Given that, how would the DecompressBytes GZip work then as it's going to be working from different numbers - or does it simply make the adjustment in type?

Assuming that DecompressBytes adjusts accordingly, I'm now still left with a string that doesn't decompress

 

[joneden]

arghhhhh stupid mistake alert!

The string that I was passing to the decryption routine still had the 4 bytes from the vb code with the string size. And to make it worse when I removed that as bug checking I still ended up looking at the wrong byte array.

All fixed now but many thanks for the help guys and agraham - thanks for the patience with java version of bytes

Cheers,

Jon

 

[sioconcept]

(Don't use it for the moment; see http://www.b4x.com/forum/basic4android-updates-questions/26634-compress-gzip-zlib.html#post154039)

You can use simply in PHP 5.4+ (change -String- of course):

echo gzencode(-String-, -1, FORCE_DEFLATE);

and in B4A

CompressedStreams.DecompressBytes(-String-.GetBytes("UTF8"), "zlib")

 

[aalekizoglou]

HTTPUtils2 JobDone with compressed data

I am trying to implement a call to MySQL tunnel that returns compressed data with gzencode($result, -1, FORCE_DEFLATE);

Then I have a problem decompressing data in B4A JobDone. Here is the code:

Sub JobDone(Job As HttpJob)
   ProgressDialogHide
   If Job.Success Then
      Dim strResult As String
      
      strResult = Job.GetString   
      If MySQLConnection.Compress = 1 Then
         Dim cs As CompressedStreams      
         Dim decompressed() As Byte
         decompressed = cs.DecompressBytes(strResult.GetBytes("UTF8"), "zlib")
         strResult = BytesToString(decompressed,0, decompressed.Length, "UTF8")
      Else
         strResult = Job.GetString
      End If
      Log("Response from server: " & strResult)
      JSONDecodeResult(strResult)
   Else
        Log("HTTP Request Error: " & Job.ErrorMessage)
   End If
   Job.Release
End Sub

Line decompressed = cs.DecompressBytes(strResult.GetBytes("UTF8"), "zlib") raises an Java.IO.Exception

I understand something is wrong in between calling Job.GetString which does a UTF8 decode and strResult.GetBytes("UTF8"), but cannot find the error.

 

[Erel]

If the stream is compressed then you should not call Job.GetString.

You should instead work with the input stream:

decompressed = cs.DecompressBytes(Job.GetInputStream, "zlib")

 

[aalekizoglou]

If the stream is compressed then you should not call Job.GetString.

You should instead work with the input stream:

decompressed = cs.DecompressBytes(Job.GetInputStream, "zlib")

Erel,

since cs.DecompressBytes gets CompressedData as Byte() i've tried

In = cs.WrapInputStream(Job.GetInputStream, "zlib")
reader.Initialize(In)         
strResult = reader.ReadAll

But also gets a IOexception in reader.ReadAll

 

[Erel]

Your code is correct. See this thread: http://www.b4x.com/forum/basic4android-updates-questions/26634-compress-gzip-zlib.html#post154124

 

[aalekizoglou]

Great.

Found my error. I was using gzencode in php and CS.DecompressBytes(Tools, "zlib") in B4A.

Changed that to gzcompress in php and is working great.

Thanks

 

[mike.1313]

You should skip the first 4 bytes as they are not part of the gzip stream.

hi erel,
how many bytes will should to skip for zlib compression?

best regards

 

[Erel]

Normally you shouldn't skip any byte. My answer was only for this specific stream that for some reason started with 4 bytes that are not part of the GZIP protocol.

 

[mike.1313]

thanks Erel,

I changed zlib by gzip and works perfect