Java Question Raw2BMP

XverhelstX

Well-Known Member
Licensed User
Longtime User
Hey everyone,

I'm making an app that is going very well, but I have a problem uploading the images to my webserver. The way I work is as the following:

The code repeatedly takes a snapshot from the camera preview using setOneShotPreviewCallback() to call onPreviewFrame(). The frame is delivered in YUV format so raw2jpg() converts it into 32 bit ARGB for the jpeg encoder.

getPicture() is called, presumably by the application, and produces the jpeg data for the image in the private byte array mCurrentFrame and returns that array. Next I upload the picture to my index.php file with public void uploadFrame().

I use the following code in my java library:

B4X:
public synchronized void getPicture(int Width, int Height){
    FrameWidth = Width;
    FrameHeight = Height;
    try {
                Log.i("B4A", "Try Mode.");
        while (!isPreviewOn) wait();
                Log.i("B4A", "IsPreviewOn is True");
        isDecoding = true;
                Log.i("B4A", "Decoding is true.");
        c.setOneShotPreviewCallback(this);
                Log.i("B4A", "Callback Done.");             
    } catch (Exception e) {
                Log.i("B4A", "An error occured.");
    }       
}

@Override
public synchronized void onPreviewFrame(byte[] data, Camera camera) {
        Log.i("B4A", "In onPreviewFrame");
    int width = FrameWidth;
    int height = FrameHeight;

    int[] temp = new int[width * height];
    OutputStream out = new ByteArrayOutputStream();
    Bitmap bm = null;
        Log.i("B4A", "Initialize raw2jpg");
    raw2jpg(temp, data, width, height);
    bm = Bitmap.createBitmap(temp, width, height, Bitmap.Config.RGB_565);
    bm.compress(CompressFormat.JPEG, 100, out);
    mCurrentFrame = ((ByteArrayOutputStream) out).toByteArray();
    isDecoding = false;
        Log.i("B4A", "isDecoding false");
    ba.raiseEvent(this, eventName + "_previewtaken", new Object[] { mCurrentFrame } );
}

/**
 * Encodes a bytearray frame with Base64 and uploads it to a web client that supports PHP.
 * @param WebClient a string referring to your base.php file (http://IP:Port/base.php)
 * e.g. "http://192.168.1.100/android/base.php, "http://www.mywebsite.com/index.php
 * 
 */
public void uploadFrame() {
    Log.i("B4A", "UploadFrame Called");
    String ba1=Base64.encodeBytes(mCurrentFrame);
    ArrayList<NameValuePair> nameValuePairs = new
    ArrayList<NameValuePair>();
       nameValuePairs.add(new BasicNameValuePair("image",ba1));

       try{

           HttpClient httpclient = new DefaultHttpClient();
           HttpPost httppost = new
           HttpPost("http://www.streamnation.netai.net/index.php");
           httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
           HttpResponse response = httpclient.execute(httppost);
           HttpEntity entity = response.getEntity();
           is = entity.getContent();
     }catch(Exception e){
           Log.e("log_tag", "Error in http connection "+e.toString());
     }
}

/**
 * Converts raw data to jpg.
 * 
 */
public void raw2jpg(int[] rgb, byte[] raw, int width, int height) {

        Log.i("B4A", "In raw2jpg.");
    final int frameSize = width * height;

    for (int j = 0, yp = 0; j < height; j++) {
        int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
        for (int i = 0; i < width; i++, yp++) {
            int y = 0;
            if (yp < raw.length) {
                y = (0xff & ((int) raw[yp])) - 16;
            }
            // int y = (0xff & ((int) raw[yp])) - 16;
            if (y < 0)
                y = 0;
            if ((i & 1) == 0) {
                if (uvp < raw.length) {
                    v = (0xff & raw[uvp++]) - 128;
                    u = (0xff & raw[uvp++]) - 128;
                }
            }

            int y1192 = 1192 * y;
            int r = (y1192 + 1634 * v);
            int g = (y1192 - 833 * v - 400 * u);
            int b = (y1192 + 2066 * u);

            if (r < 0)
                r = 0;
            else if (r > 262143)
                r = 262143;
            if (g < 0)
                g = 0;
            else if (g > 262143)
                g = 262143;
            if (b < 0)
                b = 0;
            else if (b > 262143)
                b = 262143;

            rgb[yp] = 0xff000000 | ((r << 6) & 0xff0000)
                    | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);

        }

    }
    Log.i("B4A", "Done with raw2jpg");
}

My code uploadFrame is the following: Android: Upload image to server with the public void uploadFrame. but because in there, they use a bitmap and not a jpg file, I think that this is the problem that it doesn't display on my webserver.

This is the php server side:

Got an error when I try to upload with my php code.

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, webmaster@basic4ppc.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.

Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.

See this here for my php code: site


So is there any code that will convert my raw data to a bmp file instead of a jpg file (= just like raw2jpg(temp, data, width, height); but then raw2bmp(temp, data, width, height);) or is there anything else wrong with the code? It has worked one time with this code, but the picture shown on my webserver was completely green and blurry. This is an example of how it sometimes look like:

zVkli.jpg
 
Last edited:

agraham

Expert
Licensed User
Longtime User
The picture that you linked to shows an astonishingly wasteful and incredibly stupid way to send an image over a network - as a Base64 encoded string representing a byte array. Even a small 320 x 256 picture is going to occupy more than 300Kbytes of data.

Luckily that is not what is actually happening when you examine the code. It is compressing the bitmap to a jpeg coded byte array and sending that Base64 encoded to the server which writes it to a file and opens it as a jpg image. I don't know why it's not working at the server end but I think you are sending the correct data.
 

XverhelstX

Well-Known Member
Licensed User
Longtime User
The picture that you linked to shows an astonishingly wasteful and incredibly stupid way to send an image over a network - as a Base64 encoded string representing a byte array. Even a small 320 x 256 picture is going to occupy more than 300Kbytes of data.

So do you think this is a bad method to send the file over a network and upload it to my website?

Problem apparently solved:

Did the following on my previewTaken sub:

B4X:
Sub Camera1_PreviewTaken (PictureFrame() As Byte)
   CurrentJPEG = PictureFrame
   
   Dim out As OutputStream
   out = File.OpenOutput(File.DirRootExternal, "abcdefghij.jpg", False)
   out.WriteBytes(PictureFrame, 0, PictureFrame.Length)
   out.Close
   ToastMessageShow("Image saved: " & File.Combine(File.DirRootExternal, "abcdefghij.jpg"), True)

End Sub

and apparantly, I receive again a cyan/green blurry picture, so it must be related to the code over there. Strange, as I copied the code like it says...

EDIT: Ow man, the picture still shows the crappy image even after changing the code. :s

One problem can be the following:

B4X:
raw2jpg(temp, data, width, height);
bm = Bitmap.createBitmap(temp, width, height, Bitmap.Config.RGB_565);
bm.compress(CompressFormat.JPEG, quality, out);

That raw2jpg is called and while it is decoding, bitmap.createbitmap is immediatly called that affects the raw2jpg?:)s) just an assuming. (remember the annoying wait() and notify())

EDIT: Ok, now I'm almost sure the problem is related to the data:
Executing this (for API 8 (2.2)) I also receive a green blurry picture:
B4X:
// API 8 and above
      YuvImage yuvi = new YuvImage(data, ImageFormat.NV21 , width, height, null);
      Rect rect = new Rect(0,0,yuvi.getWidth() ,yuvi.getHeight() );
      OutputStream out = new ByteArrayOutputStream();
      yuvi.compressToJpeg(rect, quality, out);     
      mCurrentFrame = ((ByteArrayOutputStream) out).toByteArray();

So the problem is related to the data captured by the onpreviewFrame I think :s
 
Last edited:

agraham

Expert
Licensed User
Longtime User
So do you think this is a bad method to send the file over a network and upload it to my website?
If you were doing what that diagram indicates - yes. But as the diagram omits the jpeg compression and decompression stages it is wrong and does not represent what is actually happening which is a much better way of doing it.
That raw2jpg is called and while it is decoding, bitmap.createbitmap is immediately called that affects the raw2jpg
No, that's not happening. It may not be being run on the main thread but the other thread won't interrupt itself (I could make a joke about that but it's rude and non-native English speakers probably wouldn't understand it :)).

You have done what I was going to suggest last night but had to go and cook tea before I could post it. That is decode the jpeg data locally as an image and see what it looks like. If the local image looks like the remote one at least the networky bits are OK and you only have an image coding problem.
 
Last edited:

XverhelstX

Well-Known Member
Licensed User
Longtime User
If you were doing what that diagram indicates - yes. But as the diagram omits the jpeg compression and decompression stages it is wrong and does not represent what is actually happening which is a much better way of doing it.

Ok, thanks for your clarification :D

No, that's not happening. It may not be being run on the main thread but the other thread won't interrupt itself (I could make a joke about that but it's rude and non-native English probably speakers probably wouldn't understand it :)).

I prob won't get it. :sign0161:

Have you looked at Camera.getPreviewFormat to check what the image format actually is?

I did this now and my output integer was: 17 what means it uses the NV21 encoding format, the default format for camera preview images.
Do I have to change this in somewhat like 4? (RGB_565) as it compresses the data with RGB_565. I will look in the code if they use SetPreviewFormat somewhere.

EDIT: nope, the only setPreview data they use are:
- setPreviewDisplay(holder);
- setPreviewSize(w, h);
- setPreviewFrameRate(fps);

XverhelstX
 
Last edited:

agraham

Expert
Licensed User
Longtime User
I've just looked at the version of ACL that I used to show you the event. This line looks in onPreviewFrame looks odd.
B4X:
bm = Bitmap.createBitmap(temp, width, height, Bitmap.Config.RGB_565);
I don't understand why it makes a lower resolution bitmap than the data which is is ARGB_8888 and then compresses it to a jpeg which is an 8bit per colour format. I suppose it might make the final file a bit smaller.

EDIT:- Actually I forgot that the camera preview data is already colour range limited so it doesn't matter. The following comment still applies though.

Also it doesn't make sense to pass a quality setting of 100, the highest quality, to the jpeg compressor when you have just thrown away quality by using a limited range colour format bitmap. Once you get it working I would experiment with smaller values. I guess you could halve the data with more compression with very little visual effect.
B4X:
bm.compress(CompressFormat.JPEG, 100, out);

However neither of these comments explain why you are getting that crappy image.
 
Last edited:

XverhelstX

Well-Known Member
Licensed User
Longtime User
Also it doesn't make sense to pass a quality setting of 100, the highest quality, to the jpeg compressor when you have just thrown away quality by using a limited range colour format bitmap. Once you get it working I would experiment with smaller values. I guess you could halve the data with more compression with very little visual effect.
B4X:
bm.compress(CompressFormat.JPEG, 100, out);
However neither of these comments explain why you are getting that crappy image.

In the current ACL Library now, I have already changed the code:

B4X:
public void getPicture(int Width, int Height, int Quality){
FrameWidth = Width;
FrameHeight = Height;
FrameQuality = Quality;
...
...
yuvi.compressToJpeg(rect, FrameQuality, out);

to use this in B4A:

B4X:
Sub mnuPreviewMode_Click
   camera1.getPicture(176, 144, 40)
End Sub

Now there might be one solution left for the crappy image.
According to some issue's with camdroid, some guy also has the same problem but his picture is less 'crapier' than mine. The problem here could be that the resolution was to high.

Now, I opened Camdroid on my smartphone, took the lowest resolution that it works with(176*144) (camera1.getPicture(176, 144, 40)), changed my Panel size to the same resolution, tested it and had the same green blurry effect.
Although, in their code, they use another way to set their resolution I think:

B4X:
public void setResolution(int w, int h)
    {
        LayoutParams layoutParams;
        Camera.Parameters camParams;
        int x,y;
        double camRatio = 0.98;
        
        // Setting up the Resolution of the CameraView
        x = (int)(camRatio*(double)mDisplay.getWidth());
        y = (int)(camRatio*(double)mDisplay.getHeight());
        layoutParams = this.calcResolution(x, y, w, h);
        
        layoutParams.gravity = Gravity.CENTER;
        this.setLayoutParams(layoutParams);
        
        // Setting up the Resolution of the overlaying OSD
        // Taking the Resolution of the CameraView
        mOSD.setLayoutParams(layoutParams);
        
        layoutParams.gravity = Gravity.CENTER;
        mBorder.setLayoutParams(layoutParams);
        
        // Setting up the Resolution of the underlying BorderFrame
        // Taking the Resolution of the CameraView adding only the
        // height-difference to get an equal border around the CameraView
        x += (int)((1.0-camRatio)*(double)mDisplay.getHeight());
        y += (int)((1.0-camRatio)*(double)mDisplay.getHeight());
        layoutParams =  this.calcResolution(x, y, w, h);
        
        layoutParams.gravity = Gravity.CENTER;
        mBorder.setLayoutParams(layoutParams);
        
        // Setting up the Resolution of resulting Image
        camParams = mCamera.getParameters();
        camParams.setPreviewSize(w, h);
        mCamera.setParameters(camParams);
    }
 //Here is the code of the onPreviewFrame where they set the width and the height
public synchronized void onPreviewFrame(byte[] data, Camera camera)
        {
        int width = mSettings.PictureW() ;
        int height = mSettings.PictureH();

//Here is the code where they set the resolution:
public void surfaceChanged(SurfaceHolder holder, int format, int w, int h)
    {
        // Now that the size is known, set up the camera parameters and begin
        // the preview.
        setResolution(mSettings.PictureW(), mSettings.PictureH());
        setFrameRate(mSettings.FrameRate());
        startPreview();
    }


Now I will add a setResolution for setPreviewSize in the ACL code and check if it changes anything. Note that I don't understand everything from the code "setResolution"

XverhelstX

EDIT:

added:

B4X:
public void setResolution(int width, int height) {
      Camera.Parameters params = c.getParameters();
      params.setPreviewSize(width, height);
      c.setParameters(params);
   }

Still shows the crappy image, but i'm sure I did something else wrong with the resolution. Apparently something with the camRatio and that you maybe have to call setPreviewSize at surfaceChanged(SurfaceHolder holder, int format, int w, int h) ?

B4X:
Sub mnuIndex_click
   
   camera1.setResolution(176, 144)

End Sub
 
Last edited:

agraham

Expert
Licensed User
Longtime User
Where are you setting the camera preview size? You pass an arbitrary width and height into getPicture but there is nowhere that I can see that you communicate this to the camera. The size of the picture in raw[] needs to match the size you pass to raw2jpg otherwise it won't decode properly.
 

XverhelstX

Well-Known Member
Licensed User
Longtime User
Now, I don't work with raw2jpg, but with the version supported by 2.2. (see API level 8)

B4X:
/**
    * Sets the dimensions for preview pictures. The sides of width and height are based on camera orientation. 
    * That is, the preview size is the size before it is rotated by display orientation. So applications need 
    * to consider the display orientation while setting preview size. For example, suppose the camera supports 
    * both 480x320 and 320x480 preview sizes. The application wants a 3:2 preview ratio. If the display orientation 
    * is set to 0 or 180, preview size should be set to 480x320. If the display orientation is set to 90 or 270, 
    * preview size should be set to 320x480. The display orientation should also be considered while setting picture 
    * size and thumbnail size.
    */
   public void setResolution(int width, int height) {
      Camera.Parameters params = c.getParameters();
      params.setPreviewSize(width, height);
      c.setParameters(params);
   }
   
   /**
    * Gets the picture in the current frame.
    * Quality is the quality of the jpeg image.
    * Calls PreviewTaken when finished.
    * 
    */
   public void getPicture(int Width, int Height, int Quality){
      FrameWidth = Width;
      FrameHeight = Height;
      FrameQuality = Quality;
      try {
               Log.i("B4A", "Try Mode.");
         
               Log.i("B4A", "IsPreviewOn is True");
         isDecoding = true;
               Log.i("B4A", "Decoding is true.");
         c.setOneShotPreviewCallback(this);
               Log.i("B4A", "Callback Done.");            
      } catch (Exception e) {
               Log.i("B4A", "An error occured.");
      }      
   }

   @Override
   public void onPreviewFrame(byte[] data, Camera camera) {
         Log.i("B4A", "In onPreviewFrame");
      int width = FrameWidth;
      int height = FrameHeight;
      int quality = FrameQuality;

      // API 8 and above
      YuvImage yuvi = new YuvImage(data, ImageFormat.NV21 , width, height, null);
      Rect rect = new Rect(0,0,yuvi.getWidth() ,yuvi.getHeight() );
      OutputStream out = new ByteArrayOutputStream();
      yuvi.compressToJpeg(rect, quality, out);     
      mCurrentFrame = ((ByteArrayOutputStream) out).toByteArray();
      isDecoding = false;
         Log.i("B4A", "isDecoding false");
       ba.raiseEvent(this, eventName + "_previewtaken", new Object[] { mCurrentFrame } );
}
   
   /**
    * Encodes a bytearray frame with Base64 and uploads it to a web client that supports PHP.
    * @param WebClient a string referring to your base.php file (http://IP:Port/base.php)
    * e.g. "http://192.168.1.100/android/base.php, "http://www.mywebsite.com/index.php
    * 
    */
   public void uploadFrame(byte[] Frame, String WebClient) {
      Log.i("B4A", "UploadFrame Called");
      String ba1=Base64.encodeBytes(Frame);
        ArrayList<NameValuePair> nameValuePairs = new
        ArrayList<NameValuePair>();
           nameValuePairs.add(new BasicNameValuePair("image",ba1));
          
           try{
           
               HttpClient httpclient = new DefaultHttpClient();
               HttpPost httppost = new
               HttpPost(WebClient);
               httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
               HttpResponse response = httpclient.execute(httppost);
               HttpEntity entity = response.getEntity();
               is = entity.getContent();
         }catch(Exception e){
               Log.e("log_tag", "Error in http connection "+e.toString());
         }
   }

So what I do is first execute setResolution in mnuResolution_click.

B4X:
Sub mnuResolution_click
   
   camera1.setResolution(176, 144)

End Sub
(everything is already loaded then, startPreview etc)

next, I call getPicture in mnuPreviewMode

B4X:
Sub mnuPreviewMode_Click
   camera1.getPicture(176, 144,40)
End Sub

and in onPreviewFrame:

B4X:
Sub Camera1_PreviewTaken (PictureFrame() As Byte)
   CurrentJPEG = PictureFrame
   
   Dim out As OutputStream
   out = File.OpenOutput(File.DirRootExternal, "abcdefghij.jpg", False)
   out.WriteBytes(PictureFrame, 0, PictureFrame.Length)
   out.Close
   ToastMessageShow("Image saved: " & File.Combine(File.DirRootExternal, "abcdefghij.jpg"), True)

End Sub

Those are the only codes used to retrieve that blurry image.
XverhelstX

EDIT:

I was just googling a bit around and came across this here:

http://stackoverflow.com/questions/1893072/getting-frames-from-video-image-in-android

If you scroll down to the Answer with 6 votes up, you can see that someone has the exact same results like me:
Hey Alexander, are you doing any other processing or voodoo magic in the background? I've tried this and end up with an image like this: dl.dropbox.com/u/6620976/image.jpg. Any insight is appreciated. – Brian D Feb 28 at 19:26


Nevermind! I just needed to define my surface's width and height and it all works great. Thanks for posting this up! – Brian D Feb 28 at 20:02


@Brian D: I am facing the same problem, can you please show me your surface code? – Mudassir May 23 at 5:12
1

@mudassir: have at it. bit.ly/mHLf5p – Brian D May 27 at 4:04
upvote
flag
@Brian D: Thanks. :) – Mudassir Jun 8 at 11:56
 
Last edited:

XverhelstX

Well-Known Member
Licensed User
Longtime User
:eek:? xD how...

Changing this:

B4X:
@Override
         public void surfaceChanged(SurfaceHolder holder, int format,
               int width, int height) {
            
            Camera.Parameters p = c.getParameters();

                p.setPreviewSize(width, height);
                p.setPreviewSize(800, 480);
                c.setParameters(p);

         }

Closed my app unexpectedly.

B4X:
I've got it working

Would you mind to share it? :D

XverhelstX
 

agraham

Expert
Licensed User
Longtime User
The problem is what I said in the post above.
Where are you setting the camera preview size? ...The size of the picture in raw[] needs to match the size you pass to raw2jpg otherwise it won't decode properly.
There seems to be a list of valid Preview sizes in the camera parameters but I just left it at the default and changed the entry to onPreviewFrame to ensure that raw2jpg was using the same width and height as the camera. I used the pre-2.2 algorithm as my phone is only 2.1 and it works fine.

B4X:
      Camera.Parameters cp = c.getParameters();
      Size sz = cp.getPreviewSize();
      int width = sz.width; //FrameWidth;
      int height = sz.height; //FrameHeight;
 

agraham

Expert
Licensed User
Longtime User
Still I don't understand why you had to enter a size for it and couldn't just give in 800 and 480 for example.
You probably can if you do it right. But the data the camera returns is of a specific width and height and you must decode it using that width and height otherwise you don't get the proper image back.
 

agraham

Expert
Licensed User
Longtime User
Why is it I can work this stuff out but you can't. :confused:
Like I said above, there is a list available of valid preview sizes from camera parameters getSupportedPreviewSizes(). If you set an invalid size then the camera will adopt some default value. Note that just setting the camera parameter properties is not enough - you need to set the parameters again on the camera to get the new size to stick. If you wanted you could check the provided size against the list and throw an exception for an invalid size.
B4X:
public synchronized void getPicture(int Width, int Height){
   Camera.Parameters cp = c.getParameters();      
   cp.setPreviewSize(Width, Height);
   c.setParameters(cp);
   ...
 
Top