How can I manipulate the camera preview frames?

mitsusdev

Member
Licensed User
Longtime User
Hi,
i'm working on an application for live camera streaming analisys.

I would like to take a lot of frames from camera and analyze them (color transition, ecc).
I've seen about Advanced Camera Library, but i don't understand if is it possible to take frames without taking a picture (it takes too long to take a picture).

Can someone help me to find a solution (if this is possible using basic4android?)

Thanks and Regards
 

mitsusdev

Member
Licensed User
Longtime User
It is possible to add a preview event to Camera library. The data received will be formatted with YCbCr_420_SP. Analyzing the images in real-time will probably be too slow and will make problems to the camera service.

Do you think that it will be useful?

Thanks a lot Erel for your response.

My idea is the same of your explanation!

Yes, i've seen about preview event (onPreviewFrame....i think), but i'm not able to use it;
It start a callback after take a preview?

Can you post a brief sample about it please?

Best Regards
 
Upvote 0

mitsusdev

Member
Licensed User
Longtime User
Download the attached camera library and put it in the internal libraries folder.

It adds a preview event:
B4X:
Sub Camera1_Preview (Data() As Byte)
   
End Sub

Great work Erel,
this is what I was looking. Now i test it, but in the first instance it seems to work well.

Thanks a lot
 
Upvote 0

mitsusdev

Member
Licensed User
Longtime User
Download the attached camera library and put it in the internal libraries folder.

It adds a preview event:
B4X:
Sub Camera1_Preview (Data() As Byte)
   
End Sub

Hi Erel,
please can you share source of this Camera Library with _Preview method? I want add other features.

Regards
 
Upvote 0

Erel

B4X founder
Staff member
Licensed User
Longtime User
Here:
B4X:
package anywheresoftware.b4a.objects;

import java.io.IOException;
import java.util.concurrent.atomic.AtomicInteger;

import android.hardware.Camera;
import android.view.SurfaceHolder;
import android.view.SurfaceView;
import android.view.ViewGroup;
import anywheresoftware.b4a.BA;
import anywheresoftware.b4a.BA.ActivityObject;
import anywheresoftware.b4a.BA.Events;
import anywheresoftware.b4a.BA.Permissions;
import anywheresoftware.b4a.BA.ShortName;
import anywheresoftware.b4a.BA.Version;

/**
 * The camera object allows you to use the device back facing camera to take pictures and show preview images.
 *Currently the camera orientation is always landscape. Usually you will want to force the application to also be in landscape mode (Project - Supported Orientations).
 *Only one process can access the camera at any time. Therefore it is highly recommended to initialize the camera object in Activity_Resume and release it in Activity_Pause.
 *A working example with explanations is available <link>here|http://www.b4x.com/forum/basic4android-getting-started-tutorials/6891-take-pictures-internal-camera.html</link>.
 */
@ActivityObject
@Permissions(values={"android.permission.CAMERA"})
@ShortName("Camera")
@Version(1.10f)
@Events(values={"Ready (Success As Boolean)", "PictureTaken (Data() As Byte)",
      "Preview (Data() As Byte)"})
public class CameraW {
   private static Camera c;
   private SurfaceView sv;
   private String eventName;
   private BA ba;
   private AtomicInteger readyCount = new AtomicInteger();
   /**
    * Initializes the camera.
    *Panel - The preview images will be displayed on the panel.
    *EventName - Events subs prefix.
    *The Ready event will be raised when the camera has finished opening.
    */
   public void Initialize(final BA ba, ViewGroup Panel, String EventName) {
      this.ba = ba;
      readyCount.set(0);
      this.eventName = EventName.toLowerCase(BA.cul);
      sv = new SurfaceView(ba.context);
      anywheresoftware.b4a.BALayout.LayoutParams lp = new anywheresoftware.b4a.BALayout.LayoutParams(0, 0,
            Panel.getLayoutParams().width, Panel.getLayoutParams().height);
      Panel.addView(sv, lp);
      if (c != null) {
         readyCount.set(1);
      }
      sv.getHolder().setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
      sv.getHolder().setFixedSize(Panel.getLayoutParams().width, Panel.getLayoutParams().height);
      sv.getHolder().addCallback(new SurfaceHolder.Callback() {

         @Override
         public void surfaceChanged(SurfaceHolder holder, int format,
               int width, int height) {
         }

         @Override
         public void surfaceCreated(SurfaceHolder holder) {
            if (readyCount.addAndGet(1) == 2) {
               ba.raiseEvent(null, eventName + "_ready", true);
            }
            
         }

         @Override
         public void surfaceDestroyed(SurfaceHolder holder) {

         }

      });
      if (c == null) {
         ba.submitRunnable(new Runnable() {

            @Override
            public void run() {
               try {
                  c = Camera.open();

                  if (readyCount.addAndGet(1) == 2) {
                     ba.raiseEventFromDifferentThread(null, CameraW.this, -1,eventName +  "_ready", false, new Object[] {true});
                  }
                  if (ba.subExists(eventName + "_preview")) {
                     c.setPreviewCallback(new Camera.PreviewCallback() {

                        @Override
                        public void onPreviewFrame(byte[] data,
                              Camera camera) {
                           ba.raiseEvent(null, eventName + "_preview", data);
                        }
                        
                     });
                  }
               }
               catch (Exception e) {
                  e.printStackTrace();
                  Release();
                  ba.raiseEventFromDifferentThread(null, CameraW.this, -1,eventName +  "_ready", false, new Object[] {false});
               }
            }

         }, this, -1);
      }
   }
   /**
    * Starts displaying the preview images.
    */
   public void StartPreview() throws IOException {
      c.setPreviewDisplay(sv.getHolder());
      c.startPreview();
   }
   /**
    * Stops displaying the preview images.
    */
   public void StopPreview() {
      if (c != null)
         c.stopPreview();
   }
   /**
    * Releases the camera object and allows other processes to access the camera.
    */
   public void Release() {
      if (sv != null) {
         ViewGroup vg = (ViewGroup) sv.getParent();
         vg.removeView(sv);
         sv = null;
      }
      if (c != null) {
         c.release();
         c = null;
      }
   }
   /**
    * Takes a picture. When the picture is ready, the PictureTaken event will be raised.
    *You should not call TakePicture while another picture is currently taken.
    *The preview images are stopped after calling this method. You can call StartPreview to restart the preview images.
    */
   public void TakePicture() {
      c.takePicture(null , null, new Camera.PictureCallback() {

         @Override
         public void onPictureTaken(byte[] data, Camera camera) {
            ba.raiseEvent(null, eventName + "_picturetaken", data);
         }
         
      });
   }



}
 
Upvote 0

Gsquared

Member
Licensed User
Longtime User
working with camera Preview event

Hi Erel,

I am not able to get the Preview event to work from your updated camera library. I used the following sub:

B4X:
Sub Camera1_Preview (PreviewPic() As Byte)
   Dim in As InputStream
   in.InitializeFromBytesArray(PreviewPic, 0, PreviewPic.Length)
   Dim bmp As Bitmap
   bmp.Initialize2(in)
   Dim PixelColor As Int
   PixelColor = bmp.GetPixel(100,100)
   Dim bmppix(bmp.Width,bmp.Height) As PixColor
   bmppix(100,100) = GetARGB(bmp.GetPixel(100, 100))
   Log("PixelColor = A "&bmppix(100,100).res(0)&" R "&bmppix(100,100).res(1)&" G "&bmppix(100,100).res(2)&" B "&bmppix(100,100).res(3))
End Sub

I get an error on the "bmp.Initialize2(in)" line. Here's the first couple of lines of the log:
java.lang.RuntimeException: Error loading bitmap.
at anywheresoftware.b4a.objects.drawable.CanvasWrapper$BitmapWrapper.Initialize2(CanvasWrapper.java:500)

I used essentially the same code with the PictureTaken event and it worked. But, I was only able to take 2 pictures/sec with this. I need to acquires images faster than this. That's why I'm trying to use the Preview event.

Thanks,

G^2
 
Upvote 0

Erel

B4X founder
Staff member
Licensed User
Longtime User
Seems like the first few frames are empty frames. The buffer is full of zeroes. This is what the camera returns. I recommend you to use Try Catch and just ignore these errors.

Creating an image from each frame might not be a good idea. It may hog the main thread. You should save the last image time and then process the next image after x milliseconds.
 
Upvote 0

irda

Member
Licensed User
Longtime User
What is exactly the question?

If it is possible to manipulate the data in the preview event of a quick way. Can you extract a thumbnail?
I read that "Data ()" is YCbCr_420_SP format, can it pass to RGB, modify a pixel and save the image or display it on a panel?

One last question, is where the definition of PixColor? thanks
 
Upvote 0

francoisg

Active Member
Licensed User
Longtime User
How can I get the preview image data as a bitmap (jpg) when I can't use the CameraEx lib (using Android 2.2)?

Is there example code of how to convert the YCbCr_420_SP format byte array to something I can load into an ImageView / Bitmap etc. ???

(I got something to almost work (using NV21util library) but I can't seem to find the preview image size which makes it impossible to use the "proceed" method in order to get the data back as an image :confused::cool:
 
Upvote 0

francoisg

Active Member
Licensed User
Longtime User
Sorry - I got a bit desperate after struggling so much!

Actually got it working if anyone wants sample code ;)
 
Upvote 0

MaFu

Well-Known Member
Licensed User
Longtime User
If it is possible to manipulate the data in the preview event of a quick way. Can you extract a thumbnail?
I read that "Data ()" is YCbCr_420_SP format, can it pass to RGB, modify a pixel and save the image or display it on a panel?

One last question, is where the definition of PixColor? thanks
Try my litte demo if it's fast enough on your device.
 
Upvote 0
Top