ESP32-CAM: Can someone explain how the camera frame buffer pointer works

martinwaltertan
Posts: 3
Joined: Mon Apr 05, 2021 8:41 am

ESP32-CAM: Can someone explain how the camera frame buffer pointer works

Postby martinwaltertan » Mon Apr 05, 2021 9:02 am

I'm a bit confused about how the frame buffer pointer from esp_camera_fb_get() works. According to the description it says "Obtain pointer to a frame buffer." so does this mean the camera frame was already captured beforehand?

There doesn't seem to be a function for actually triggering a "capture" in the camera so I keep thinking that "getting the frame buffer" means I'm getting a pointer to the image that was already captured by the camera. And if it is already being captured then I also assume the camera is continuously taking stills in the background after I called esp_camera_init().

The main reason I'm asking this question is because I set-up a video stream using ESP-IDF HTTP Server and I also wanted to obtain the camera frames myself in the Arduino loop function to process video frames independently of the server. So I have two separate calls to esp_camera_fb_get() and I want to know if there's some kind of performance consequence of it. I also encounter this error or warning when I run my code with these two calls "[E][camera.c:1474] esp_camera_fb_get(): Failed to get the frame on time!"

Here is my code where I call esp_camera_fb_get() twice in my project:

Code: Select all

void loop() {
  fb = esp_camera_fb_get();
  esp_camera_fb_return(fb);
  fb = NULL;
}

Code: Select all

static esp_err_t stream_handler(httpd_req_t *req) {
    esp_err_t res = ESP_OK;
    char * part_buf[64];
    camera_fb_t * fb = NULL;

    res = httpd_resp_set_type(req, s_STREAM_CONTENT_TYPE);
    if (res != ESP_OK) return res;
    httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");

    while (true) {
        fb = esp_camera_fb_get();
        if (!fb) {
            Serial.println("Camera get frame failed");
            break;
        }

        res = httpd_resp_send_chunk(req, s_STREAM_BOUNDARY, strlen(s_STREAM_BOUNDARY));
        if (res != ESP_OK) {
            Serial.print("Multipart/x-mixed-replace HTTPD not ESP_OK:");
            Serial.println(res);
            break;
        }
        
        size_t hlen = snprintf((char *)part_buf, 64, s_STREAM_PART, fb->len);
        res = httpd_resp_send_chunk(req, (const char *)part_buf, hlen);
        if (res != ESP_OK) {
            Serial.println("Content-Type not ESP_OK");
            break;
        }

        res = httpd_resp_send_chunk(req, (const char *)fb->buf, fb->len);
        if (res != ESP_OK) {
            Serial.println("HTTPD send image buffer not ESP_OK");
            break;
        }

        esp_camera_fb_return(fb);
        fb = NULL;
    }
    return res;
}
Thank you for your help

alanesq
Posts: 84
Joined: Thu Dec 14, 2017 8:38 pm

Re: ESP32-CAM: Can someone explain how the camera frame buffer pointer works

Postby alanesq » Mon Apr 05, 2021 2:41 pm

Hi,

The command : fb = esp_camera_fb_get();
tells it to capture a image and store it in memory, the pointer "fb" then points to where this data is
Once you have finished with this image you tell it that this reserved memory can be released with the command: esp_camera_fb_return(fb);

BTW - You may like to have a look at my sketch which shows how I use the esp32cam modules (using the Arduino iDE), it shows how to capture an image and look at the raw data, stream live video etc.
here: https://github.com/alanesq/esp32cam-demo

martinwaltertan
Posts: 3
Joined: Mon Apr 05, 2021 8:41 am

Re: ESP32-CAM: Can someone explain how the camera frame buffer pointer works

Postby martinwaltertan » Mon Apr 05, 2021 6:22 pm

Thank you! Will have a look around

nataly
Posts: 12
Joined: Sat Oct 16, 2021 12:16 pm

Re: ESP32-CAM: Can someone explain how the camera frame buffer pointer works

Postby nataly » Mon Oct 18, 2021 8:26 am

alanesq wrote:
Mon Apr 05, 2021 2:41 pm
Hi,

The command : fb = esp_camera_fb_get();
tells it to capture a image and store it in memory, the pointer "fb" then points to where this data is
Once you have finished with this image you tell it that this reserved memory can be released with the command: esp_camera_fb_return(fb);

BTW - You may like to have a look at my sketch which shows how I use the esp32cam modules (using the Arduino iDE), it shows how to capture an image and look at the raw data, stream live video etc.
here: https://github.com/alanesq/esp32cam-demo

Hi @alanesq I really benefited from your code thank, but I have a question, please,
With your code, you have accessed RGB data. You converted PIXFORMAT_JPEG to RGB. Why did you do this? Is it possible to access the captured image data(the pixesl) before it is converted to any type like jpeg? can you explaine more please?

alanesq
Posts: 84
Joined: Thu Dec 14, 2017 8:38 pm

Re: ESP32-CAM: Can someone explain how the camera frame buffer pointer works

Postby alanesq » Mon Oct 18, 2021 8:37 pm

Hi,
I don't think it possible to access individual pixels when it is captured as a JPG as it is stored in this compressed format so it would need to be decoded first. You could capture the image as RGB or Greyscale directly and access the data rather than capture as JPG then convert if (not something I have tried myself).

nataly
Posts: 12
Joined: Sat Oct 16, 2021 12:16 pm

Re: ESP32-CAM: Can someone explain how the camera frame buffer pointer works

Postby nataly » Fri Oct 22, 2021 7:45 pm

alanesq wrote:
Mon Oct 18, 2021 8:37 pm
Hi,
I don't think it possible to access individual pixels when it is captured as a JPG as it is stored in this compressed format so it would need to be decoded first. You could capture the image as RGB or Greyscale directly and access the data rather than capture as JPG then convert if (not something I have tried myself).


Ok thanks, I used
config.pixel_format = PIXFORMAT_RGB888 ;
config.frame_size = FRAMESIZE_QQVGA; // 160*120
I copped just the value from serial monitor and paste in txt file but the size of file = ( 193,515 bytes) I thought the size of txt file should be near to 57600 byte ! WHY!!

Who is online

Users browsing this forum: No registered users and 135 guests