Android Question OpenCV help

[jdonth]

I need to add motion detection to my kiosk app. Nothing fancy like facial recognition - just trigger an event when someone walks into the room (but is not sensitive enough to false trigger on sun-angle). I've been looking at the examples (cars, javacamera, facial, etc.) but can't seem to "get it". An example would be perfect or hints on where to find the specific resources would be appreciated.

 

[MarkusR]

a team mate said you need subtract two time-displaced images and the difference in sum is a movement.
then you need a threshold value.

 

[JordiCP]

The attached example displays the detected movement on top of the original image. It is similar to the car detection modified example, but with the camera preview

If you hold your camera still and move your hand in front of it, or simply move your camera a bit, the white area will be the detected movement.

The initialization parameters of OCVBackgroundSubtractorMOG2 (search for backgroundSubstractorMOG2) are related to the time-window history length and threshold value, which can be changed.

The way to decide if there is movement or not is quite straightforward but will also need your tuning --> you will decide that there is movement when the detection area is big enough --> how to do it? --> detect the biggest blob of the mask image, calculate its area, and decide that there is "movement" if this area is bigger than a certain threshold.

 

[jdonth]

The attached example displays the detected movement on top of the original image. It is similar to the car detection modified example, but with the camera preview

If you hold your camera still and move your hand in front of it, or simply move your camera a bit, the white area will be the detected movement.

The initialization parameters of OCVBackgroundSubtractorMOG2 (search for backgroundSubstractorMOG2) are related to the time-window history length and threshold value, which can be changed.

The way to decide if there is movement or not is quite straightforward but will also need your tuning --> you will decide that there is movement when the detection area is big enough --> how to do it? --> detect the biggest blob of the mask image, calculate its area, and decide that there is "movement" if this area is bigger than a certain threshold.

JordiCP,
Thank you for the sample code. I now understand until the last part:

The way to decide if there is movement or not is quite straightforward but will also need your tuning --> you will decide that there is movement when the detection area is big enough --> how to do it? --> detect the biggest blob of the mask image, calculate its area, and decide that there is "movement" if this area is bigger than a certain threshold.​

In the _newFrame sub, how do I determine the number of white pixels?

​

 

[JordiCP]

Add these lines at the end of frameprocessor_newframe(..)

Dim meanScalar as OCVScalar = mCore.mean1(fgMaskMOG2)   'values in fgMaskMOG2 will be 0 or 255
Dim percent as Double = 100.0*meanScalar.val(0)/255.0
Log("Percent of 'moving' pixels is: "&percent)

It will give you the percentage of "moving" pixels (according to the init settings) in the whole frame. Perhaps it will be enough for your needs.
Additionally, you can add a criteria that the measured value is higher than a threshold during a certain time, for instance.
Another good idea is to first filter the "noisy" pixels with erode/dilate functions

 

[jdonth]

Add these lines at the end of frameprocessor_newframe(..)

Dim meanScalar as OCVScalar = mCore.mean1(fgMaskMOG2)   'values in fgMaskMOG2 will be 0 or 255
Dim percent as Double = 100.0*meanScalar.val(0)/255.0
Log("Percent of 'moving' pixels is: "&percent)

It will give you the percentage of "moving" pixels (according to the init settings) in the whole frame. Perhaps it will be enough for your needs.
Additionally, you can add a criteria that the measured value is higher than a threshold during a certain time, for instance.
Another good idea is to first filter the "noisy" pixels with erode/dilate functions

PERFECT. THANK YOU VERY MUCH. A reading of over 1.5% seems to work great.

Is there a way to add a "cooling off" period to _newFrame when I get a "trigger"?

 

[JordiCP]

PERFECT. THANK YOU VERY MUCH. A reading of over 1.5% seems to work great.

Is there a way to add a "cooling off" period to _newFrame when I get a "trigger"?

Not sure to understand correctly: by "cooling off" do you mean a way not to get more triggers after a given one?
If so, I'd simply get the timestamp after a 'successful' trigger and, whenever there is a new one, only accept it as good if the elapsed time is larger than a given threshold (and get its timestamp again).

 

[jdonth]

Not sure to understand correctly: by "cooling off" do you mean a way not to get more triggers after a given one?
If so, I'd simply get the timestamp after a 'successful' trigger and, whenever there is a new one, only accept it as good if the elapsed time is larger than a given threshold (and get its timestamp again).

Of course. Thank you again!

 

[jdonth]

JordiCP,

Your example code works great, but when I intergrated the CV code into my app, I get the following:

My Resume code is:

Sub Activity_Resume
Log("connectCamera " & PCV.Width & ":" & PCV.height)
mOpenCvCameraView.connectCamera(PCV.Width,PCV.Height)
Log("Done")
End Sub​

Result:

*** Service (starter) Create ***
** Service (starter) Start **
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **connectCamera 1280:752
JavaCameraViewInitialize java camera
JavaCameraViewTrying to open front camera
JavaCameraViewTrying to open camera with new open(1)
JavaCameraViewCamera #1failed to open: Fail to connect to camera service
mCamera=null
Done​

The only difference between your demo code is that my app uses a Starter service. Might there be some interaction that I don't understand?

Thank you,
Joe

**** FYI: Demo code Results *****
Sub Activity_Resume
mOpenCvCameraView.connectCamera(PCV.Width,PCV.Height)
Log("connectCamera")
End Sub
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
JavaCameraViewInitialize java camera
JavaCameraViewTrying to open front camera
JavaCameraViewTrying to open camera with new open(1)
mCamera=android.hardware.Camera@21e401f0
JavaCameraViewgetSupportedPreviewSizes()
JavaCameraViewSet preview size to 640x480
JavaCameraViewstartPreview
connectCamera

 

[JordiCP]

is the camera related code in Activity_Create and Activity_Pause also the same?

 

[jdonth]

They look the same to me (the only difference is I have the PCV panel declared in the "main" Layout file) but I'm including copies of each:
Kiosk uses two timers, a Starter service, MediaPlayer???

Kiosk app:
Sub Globals
'These global variables will be redeclared each time the activity is created.
'These variables can only be accessed from this module.
Dim ocl As OCVOpenCVLoader 'ignore Needs to be here before declaring Mat objects
Dim mOpenCvCameraView As OCVJavaCameraView
Dim PREVIEW_WIDTH As Int = 640 'Set it to a resolution that your camera supports. 640x480 is more than enough to detect color blobs in most cases.
Dim PREVIEW_HEIGHT As Int = 480
'OCV
Dim mUtils As OCVUtils
Dim mImgProc As OCVImgproc
Dim mCore As OCVCore
'Substractor
Dim mVideo As OCVVideo
Dim mOCVBackgroundSubtractor As OCVBackgroundSubtractorMOG2 = mVideo.createBackgroundSubtractorMOG2(80,50,False)
Dim fgMaskMOG2,fgMaskMOG3 As OCVMat
...
----------------------------------------------------------------------------
Your Demo (with my changes):
Sub Globals
Dim ocl As OCVOpenCVLoader 'ignore Needs to be here before declaring Mat objects
Private PCV As Panel
Dim mOpenCvCameraView As OCVJavaCameraView
Dim PREVIEW_WIDTH As Int = 640 'Set it to a resolution that your camera supports. 640x480 is more than enough to detect color blobs in most cases.
Dim PREVIEW_HEIGHT As Int = 480
'OCV
'Dim mUtils As OCVUtils
Dim mImgProc As OCVImgproc
Dim mCore As OCVCore
'Substractor
Dim mVideo As OCVVideo
Dim mOCVBackgroundSubtractor As OCVBackgroundSubtractorMOG2 = mVideo.createBackgroundSubtractorMOG2(80,50,False)
Dim fgMaskMOG2,fgMaskMOG3 As OCVMat
'Dim tmp As OCVMat
...
===============================================================================
Kiosk:
Sub Activity_Create(FirstTime As Boolean)
LoadPanel(vm,"main")
mOpenCvCameraView.Initialize("frameprocessor",PCV,mOpenCvCameraView.CAMERA_ID_FRONT)'.CAMERA_ID_BACK) .CAMERA_ID_ANY)
mOpenCvCameraView.setMaxFrameSize(PREVIEW_WIDTH,PREVIEW_HEIGHT)
PCV.Visible = False
...
End Sub
----------------------------------------------------------------------------
Demo:
TMR.Initialize("TMR",5000) ' Change this after testing to more than 5 seconds
PCV.Initialize("") ' The panel where the camera surfaceview will be placed
Activity.AddView(PCV,0,0,100%X,100%Y)
PCV.Visible = False
mOpenCvCameraView.Initialize("frameprocessor",PCV,mOpenCvCameraView.CAMERA_ID_FRONT)'.CAMERA_ID_BACK)
mOpenCvCameraView.setMaxFrameSize(PREVIEW_WIDTH,PREVIEW_HEIGHT)
End Sub
===============================================================================
Kiosk:
Sub Activity_Pause (UserClosed As Boolean)
mOpenCvCameraView.disconnectCamera
Minute.Enabled = False
End Sub
----------------------------------------------------------------------------
Demo:
Sub Activity_Pause (UserClosed As Boolean)
mOpenCvCameraView.disconnectCamera
Log("disconnectCamera")
End Sub
===============================================================================

 

[jdonth]

I just "recoded" the app without a starter service, eliminated one timer and the mediaplayer. Same result.

 

[JordiCP]

Did you add the camera permission in the manifest?

 

[jdonth]

THANK YOU, thank you, thank you.

 

[jdonth]

My app keeps "dieing" without error or warning so I thought I'd isolate the motion detection code into a service and am running into the requirement of a Panel:
mOpenCvCameraView.Initialize("frameprocessor",pcvpanel,mOpenCvCameraView.CAMERA_ID_FRONT)'.CAMERA_ID_BACK) .CAMERA_ID_ANY)

I don't need to view the frames (via a panel) I just need to "see" the frames in the _newframe subroutine. Is there another way to open the camera without the requirement of a panel (or any other activity-level views)?

 

[DonManfred]

I guess no. The Camera needs an Activity-Context. It has nothing to do with OpenCV.

Additional:

Please use [CODE]code here...[/CODE] tags when posting code.

 

[jdonth]

Thanks.