Esp-EYE Microphone stream to HTTP server

Dracon · Postby **Dracon** » Mon Aug 03, 2020 4:45 am

Hi All,
I'm currently working on making my ESP-EYE stream via its web server.
So far, I have working video stream! However, I am now trying to get an audio stream working, and having some trouble.
To clarify here, I dont need the audio to sync up with the video. It would be great to get that working, but I am okay with a separate page/stream for audio.

I have finally managed to get audio data from the boards mic, however I am at the stage of not knowing what to do with it, and without being able to verify it in some way, I dont actually know if my data is meaningful/correctly formatted. To get it working, I have been converting the examples from esp-who's speech-wakeup. However, the wakeup sends the data to its neural network, while I dont want to do that, so i dont know if im storing it correctly, or even if my chunk size is correct.

Current code: http://github.com/DraconInteractive/ESP-EYE-Server
Look at Tester 002 specifically.

So, to the actual question. How do I turn this data into a playable format? Someone told me I should use a PCM, which I somewhat understand, but have no idea how to implement. Also, are there any tutorials out there for how to set up a web-page to receive audio and play it?

I'm sorry for the vagueness, I'm sitting here with a bunch of data and a goal, and no idea how to bridge the two.

Thanks
Dracon

tandisbh · Postby **tandisbh** » Sat Apr 03, 2021 6:13 am

Hello. I have the same problem. I want to stream ADC data received by microphone on ESP32 to my web page. Did you succeed to stream?

Dracon · Postby **Dracon** » Tue Apr 13, 2021 1:37 pm

Unfortunately I did not @tandisbh. Let me know if you have any luck, its a problem im coming back to!

Jatis314 · Postby **Jatis314** » Mon Oct 11, 2021 1:13 pm

@Dracon @tandisbh
Hello,
For a project of mine I am trying to send video and audio from an ESP32-CAM to a Webserver. Video was no problem thanks to many tutorials but as you i am struggling with audio transmission.
I found this: https://www.youtube.com/watch?v=NWdemILPp3Y which has a different goal but was helpful in so far that I am now able to send a constant frequency to my Webserver. The mic i ordered is still on the way but using its I2S interface i hope i will be able to send life audio from it.
In the video he uses PCM from the Wave Format and streams it using the HTML5 audio feature. I used a lot of his code and implemented it in my code for the video server. With that i can now send a constant frequency but technically any integer value should work. So in theory you should just be able to replace the sin-function with e.g an I2S read-function (some format converting may be required).
The one problem i encountered so far is the conflict between my audio an video stream. I don't know enough about HTML to explain it but I think it do to the while(true) loop in the stream_handler. Maybe u can switch between them but I will test it in the future.
Anyways here iss the code (without the video stream, just audio).

Code: Select all

#include "esp_camera.h"
#include <WiFi.h>
#include "esp_timer.h"
#include "img_converters.h"
#include "Arduino.h"
#include "fb_gfx.h"
#include "soc/soc.h"             // disable brownout problems
#include "soc/rtc_cntl_reg.h"    // disable brownout problems
#include "esp_http_server.h"
#include <math.h>

// Replace with your network credentials
const char* ssid = "xxxxxxxxxxxxxxxxxxxxx";
const char* password = "xxxxxxxxxxxxxxxxxxxxxxxxxx";



//Wifi Reset Counter
int counter = 0;


httpd_handle_t camera_httpd = NULL;
httpd_handle_t audio_stream_httpd = NULL;

static const char PROGMEM INDEX_HTML[] = R"rawliteral(
<html>
  <head>
    <title>ESP32-CAM Robot</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <style>
      body { font-family: Arial; text-align: center; margin:0px auto; padding-top: 30px; background-color:#222222}
      
      td { padding: 8 px; }
      
    </style>
  </head>
  <body>
    <h1 style="color: white">ESP32-CAM Doorbell Audio</h1>

    <audio controls src="" type="audio/x-wav" id="audio-wav">
    
   <script>
   function toggleCheckbox(x) {
     var xhr = new XMLHttpRequest();
     xhr.open("GET", "/action?go=" + x, true);
     xhr.send();
   }
   window.onload = document.getElementById("audio-wav").src = window.location.href.slice(0, -1) + ":81/audio_stream";
  </script>
  </body>
</html>
)rawliteral";

//loads Webpage itself
static esp_err_t index_handler(httpd_req_t *req){
  httpd_resp_set_type(req, "text/html");
  return httpd_resp_send(req, (const char *)INDEX_HTML, strlen(INDEX_HTML));
}


//################################################




#define SCRATCH_BUFSIZE  8192

typedef struct {
    uint32_t chunk_id;               
    uint32_t chunk_size;             
    uint32_t format;              
}chunk_riff_t;

typedef struct {
    uint32_t chunk_id;              
    uint32_t chunk_size;            
    uint16_t audio_format;          
    uint16_t num_of_channels;        
    uint32_t samplerate;            
    uint32_t byterate;             
    uint16_t block_align;            
    uint16_t bits_per_sample;        
}chunk_fmt_t;

typedef struct {
    uint32_t chunk_id;                 
    uint32_t chunk_size;               
}chunk_data_t;


typedef struct {
    chunk_riff_t riff;                   
    chunk_fmt_t fmt;                      
    chunk_data_t data;                   
}wav_header_t;

struct streaming_wav_t{

  wav_header_t  hdr;
  int16_t     *buf;
  int       buf_size;
  int       cnt;

} ;

void streaming_wav_destroy( struct streaming_wav_t* wav ) {

  free(wav->buf);
}

int streaming_wav_factor( struct streaming_wav_t* wav ) {

  return (
      wav->hdr.fmt.num_of_channels *
      wav->hdr.fmt.bits_per_sample / 8 );
}

void streaming_wav_init( struct streaming_wav_t* wav, int buffer_size ) {

  int num_channels = 1;
  int bits_per_sample = 16;
  int sample_rate = 8000;

  wav->cnt = 0;

  streaming_wav_header( wav, num_channels, bits_per_sample, sample_rate );

  wav->buf = (int16_t*)malloc( buffer_size );
  wav->buf_size = buffer_size / streaming_wav_factor( wav );

}
void streaming_wav_header( struct streaming_wav_t* wav, int num_channels, int bits_per_sample, int sample_rate )
{
  wav_header_t* w = (wav_header_t*) &(wav->hdr);

  int len = 0xFFFFFFFF;

  w->riff.chunk_id = 0X46464952;      // "RIFF"
  w->riff.format = 0X45564157;      // "WAVE"

  w->riff.chunk_size = len;

  w->fmt.chunk_id = 0X20746D66;     // "fmt "
  w->fmt.audio_format = 1;
  w->fmt.bits_per_sample = bits_per_sample;
  w->fmt.block_align = num_channels * bits_per_sample/8;
  w->fmt.byterate = sample_rate * num_channels * bits_per_sample/8;
  w->fmt.chunk_size = 16;
  w->fmt.num_of_channels = num_channels;
  w->fmt.samplerate = sample_rate;

  w->data.chunk_id = 0X61746164;
  w->data.chunk_size = len;
}

void streaming_wav_play( struct streaming_wav_t* wav, float frequency ) {

  int sample_rate = wav->hdr.fmt.samplerate;
  int Loop = sample_rate / frequency * 1000;

    for( int j = 0 ; j < wav->buf_size ; j++ )
    {

      int i = wav->cnt++;
      if ( wav->cnt > Loop )
        wav->cnt = 0;

      double sin_float =  15000 * sinf( 2 * i * M_PI / ( sample_rate / frequency ) );  //15k for Amplitude
     

        int16_t lval = sin_float;
        int16_t rval = sin_float;

        if ( wav->hdr.fmt.num_of_channels == 2 ) {
          wav->buf[j*2] = lval;
          wav->buf[j*2+1] = rval;
        } else {
          wav->buf[j] = lval;
        }
    }
}



static esp_err_t audio_stream_handler(httpd_req_t *req)
{
    Serial.println("in audio_stream_handler");
    httpd_resp_set_type(req, "audio/x-wav");

    streaming_wav_t wav;

    Serial.println("sending header");  
    streaming_wav_init( &wav, SCRATCH_BUFSIZE );
    httpd_resp_send_chunk(req, (const char*)&(wav.hdr), sizeof(wav.hdr));

    size_t chunksize = SCRATCH_BUFSIZE;
    float frequency = 600;

    int cnt = 0;

    while(true) {                                          //Basically while(true)
      streaming_wav_play( &wav, frequency );

      if (httpd_resp_send_chunk(req, (const char*)wav.buf, chunksize) != ESP_OK) {       //Sending Audio Buffer
          Serial.println("File sending failed / Windows Closed");
          httpd_resp_sendstr_chunk(req, NULL);
          httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, "Failed to send file");
          streaming_wav_destroy( &wav );
          return ESP_FAIL;
      }

  }

    streaming_wav_destroy( &wav );
    httpd_resp_send_chunk(req, NULL, 0);

    return ESP_OK;
}

  
  //#########################################################################
  

void startCameraServer(){
  httpd_config_t config = HTTPD_DEFAULT_CONFIG();
  config.server_port = 80;
  httpd_uri_t index_uri = {
    .uri       = "/",
    .method    = HTTP_GET,
    .handler   = index_handler,
    .user_ctx  = NULL
  };


  httpd_uri_t audio_stream_uri = {
    .uri       = "/audio_stream",
    .method    = HTTP_GET,
    .handler   = audio_stream_handler,
    .user_ctx  = NULL
  };

  
  if (httpd_start(&camera_httpd, &config) == ESP_OK) {
    httpd_register_uri_handler(camera_httpd, &index_uri);
  }
  config.server_port += 1;
  config.ctrl_port += 1;
  if (httpd_start(&audio_stream_httpd, &config) == ESP_OK) {
    httpd_register_uri_handler(audio_stream_httpd, &audio_stream_uri);
  }
  
}




//######################################################################

void setup() {
  
  WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0); //disable brownout detector

  //Enable Serial Debug
  Serial.begin(115200);
  Serial.setDebugOutput(false);

  
  // Wi-Fi connection
  Serial.println("");
  Serial.print("Connecting to Wifi");
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
    counter++;
    if (counter == 10){
      Serial.println("");
      Serial.println("Could not Connect to Wifi - Restarting...");
      delay(1000);
      ESP.restart();
    }
  }
  Serial.println("");
  Serial.println("WiFi connected");
  
  Serial.print("Camera Stream Ready! Go to: http://");
  Serial.println(WiFi.localIP());


  
  // Start streaming web server
  startCameraServer();
 
}


//###################################################################

void loop() {
  
  
}

There is probably some stuff in there thats useless, but I'm still figuring out a lot of stuff myself.
Hope i could be of any help.

Esp-EYE Microphone stream to HTTP server

Esp-EYE Microphone stream to HTTP server

Re: Esp-EYE Microphone stream to HTTP server

Re: Esp-EYE Microphone stream to HTTP server

Re: Esp-EYE Microphone stream to HTTP server

Who is online

About Us

Extra

Information