Part 3 - The B4R ESP32CAM App!
The source code for the ESP32CAM Demo app is enclosed here in this post. This is my initial version and attempt at interfacing with the camera board, so I am sure there's lots of room for improvement. I've tried to self document the code as much a possible so you can follow along. I'll describe a bit about how it works.
The app will:
- Attempt to initialize the Camera module (which calls an inline c function "init", this setups up the camera with default such as frame size, image type (JPEG,RGB, etc.), and other sensor options)
- Connect to your access point (using the hard coded ssid and password)
- Start a Web Server on port 80. The web server will look for a /pic (which takes a still pic) and /live which starts a live stream
The web server (at the moment) only allows one connection at a time, to avoid contention for the camera.
Sending an Image
When the web server sees the /pic url it calls
SendPic:
The /pic URL fetches a JPEG image from the camera. The default image size that will be returned is set to SVGA (800x600) resolution. You can change this in code (the inline c), just look in the module ESP32CAM which contains the inline c and look for:
config.frame_size = FRAMESIZE_SVGA; // FRAMESIZE_ + QVGA|CIF|VGA|SVGA|XGA|SXGA|UXGA
Private Sub SendPic
ESP32CAM.TakePic
If ESP32CAM.PictureAvailable Then
Log("SendPic: Picture Available")
If StreamGood Then Astream.Write("HTTP/1.1 200 OK").Write(CRLF)
If StreamGood Then SendPicToStream
'CallSubPlus("CloseConnection", 100, 0)
ESP32CAM.Release
Else
'handle picture not available message, write now nothing displays at client side if error
End If
End Sub
This is the easiest to follow as it grabs a picture from the camera and sends it to our client. Now if you look at the code, Send Pic calls another B4R sub called TakePic which in turns calls the inline c function
take_pic_only. This is also a good place to start as you will basically understand how image capture works with the camera board (it's actually not to difficult). Basically:
- The esp_camera_fb_get() function is called, this uses the configured default values for the camera that were set up in the inline c function "init" (as I described above) and stores the results in a global frame buffer point fb:
//get image from camera and put in a frame buffer (a global point fb) - this is all managed by the camera driver and needs to be paired up with esp_camera_fb_return(fb)
//when done with the frame buffer. Called the B4R Release Sub to execute that function when done with the image, i.e. after sending to the client
fb = esp_camera_fb_get();
if(!fb) {
Serial.println("Camera capture failed");
b4r_esp32cam::_pictureavailable = false;
return;
}
The returned data is actually a single JPEG encode image (you could actually save the data "as-is" to a file and you have a valid image.
Camera boards that have PSRAM, the frame buffer will be mapped into that memory space, if not, it will be mapped into flash (the camera driver takes care of all of that). You'll see in the code that I utilize B4R variables quite a bit such as picture available (in c it's b4r_esp32cam::_pictureavailable ), this let's me know the image was capture successfully or not.
Obviously capturing the image is useless if we can't access the data from B4R! Since the frame buffer is totally managed by the camera driver and all we have is a pointer (fb), I used an empty B4R byte array (which effectively is just a pointer to nothing). this is called buffer() and is defined as so in ESP32CAM module:
We don't initialize it in B4R instead we manually set buffer() to *initially* point to the beginning of the frame buffer. This is done at the end of the take_pic_only inline c function:
//update the B4R buffer() variable. In B4R this is defined as a byte array. This will be sent to the requestor of http using ASTREAM.Write/Write2 calls
//Every time we update picture, we need to update the buffer pointer and length.
//On initial capture, the B4R buffer points to the beginning of the image, as we send chunks of data, the buffer pointer is updated by the following function
//move_frame_buffer_ptr. This was necessary as B4R arrays (i.e. our buffer byte array) can only be indexed up to 32768. If our image is larger than this
//then the image will be cutoff on the receiving end.
b4r_esp32cam::_buffer->data = fb->buf;
b4r_esp32cam::_buffer->length = frame_byte_length;
So now we have our B4R buffer() pointer pointing the raw image captured by the camera - great, kinda
. So "frame_byte_length" is the size in bytes of the image. The problem here is that our B4R length member of the array object is defined as an "int", which can only be 32768 max. So if our image is greater than that, we're out of luck. The way around this is that we will "chunk" the frame buffer up as we need it (i.e. sending over the network). In the Process Globals section of ESP32CAM I define a global variable:
Public G_CHUNK_SIZE As Int = 1024 'used for splitting image into chunks so can be sent over network
This variable defines the max amount of the frame buffer we'll use (i.e. like queuing to send over the network) at any time. So after we've captured our image, and returned from the inline function take_pic_only, the next logical thing to consider is how do we move where the B4R variable buffer() points to. This is where the the sub:
If StreamGood Then SendPicToStream
comes in. This sub does all the heavy lifting of "chunking" the frame buffer and sending each chunk to the client. I won't post the entire sub here (you can see in the code for yourself), but I'll summarize what it does. it computes the number of "chunks" to send, and moves the B4R buffer() pointer accordingly so that it points to the position in the frame buffer that we want to send. The heart of SendPicToStream is this:
For i = 0 To (num_chunks - 1)
ESP32CAM.byte_offset = i * chunk_size
ESP32CAM.MoveFrameBufferPtr
'Log("Idx: ")
'Log(i)
'Log("Offset:")
'Log(ESP32CAM.byte_offset)
'Log("Buffer Len:")
'Log(ESP32CAM.buffer.Length)
If StreamGood Then
'note not worried about stack leakage here since were using an already allocated memory buffer
Astream.Write2(ESP32CAM.buffer,0,ESP32CAM.buffer.Length) 'explicitly using write2 even though write could probably work since the length variable is updated properly
Else
Log("Stream disconnected")
Return
End If
Next
The default chunk size is 1024 bytes (which is set by G _CHUNK_SIZE as mentioned earlier). So if my image is 64K for instance, you'll have 64 chunks to send. You can play around with the chunk size (presumably you can set your chunk to 32768 bytes if you like which is the max). The sub:
ESP32CAM.MoveFrameBufferPtr
Calls an inline c function call "move_frame_buffer_ptr". This function handles adjust the B4R buffer() variable to point to the start of the chunk and updates the length member of buffer() to reflect the current chunk. One thing to note is that the last chunk may be less than
G _CHUNK_SIZE so we need to handle this and set the length properly for the last chunk - this is all handled in "move_frame_buffer_ptr". Now when we call:
Astream.Write2(ESP32CAM.buffer,0,ESP32CAM.buffer.Length)
Astream.Write2 is pointing to the correct chunk of data to queue to the network *and* the length of buffer is set properly (so boundary checking won't bite us). As a side note, I toyed with turn off #CheckArrayBounds: True, but I actually like the feature for everywhere else in the program, so the method of chunking the data gives us the best of both worlds.
Once we've looped through all the chunks, the image is sent, and the web server can go back and serve another request.
Sending Stream Video
Sending streaming video is actually just sending a bunch of discrete pictures one after the other over HTTP. They are wrapped in a multipart message that is sent to the client. You can see further discussion on what the client sees
here. From the server perspective in the code, we're just in a loop, fetching images from the camera and sending them to the client (taken from Sub StreamVideo):
'now capture each frame from camera and send it to stream, note SendPicToStream is used here as well
Do While StreamGood
'take pic
'Log("Take pic...")
ESP32CAM.TakePic
If ESP32CAM.PictureAvailable Then
'Log("Have Picture")
'send the mime frame boundary
If StreamGood Then aws("--esp32camframe") Else Return
'now send the pic
If StreamGood Then SendPicToStream
If StreamGood Then
awscrlf
awscrlf
'aws(CRLF)'.Write(CRLF)
Else
Log("Connection broken...")
Exit
End If
ESP32CAM.Release
'Log("++++looping in video...")
Delay(10) '1 frame per sec
Else
Log("---NO PICTURE!")
End If
If G_FRAME_RATE_DELAY_MSEC > 0 Then 'if 0 then send as fast as possible
'might be better off using a timer vs delay for efficiency
'with SVGA and a frame rate of 0, and close to the access point the camera is connected to, the stream is pretty good
Delay(G_FRAME_RATE_DELAY_MSEC )
End If
Loop
That's an overview of how it all works!
