ESP HTTPS Server Leaks Memory

TSchmidt
Posts: 3
Joined: Sat May 01, 2021 10:16 pm

ESP HTTPS Server Leaks Memory

Postby TSchmidt » Sun May 02, 2021 12:03 am

Hello,
I started a project with HTTPS and WebSocket and realized that something is loosing memory (by calling esp_get_free_heap_size()).
Most of the code is based on the examples (e.g. https_server/wss_server and more).

The web server uses the wildcard mechanism and loads the data from SPIFFS.

I stripped the code to more or less the minimum and isolated the static content HTTPS Webserver (no websocket). Each request is loosing memory. After connection timeout the memory is not recovered.
I went step by step trough my code and eliminated any own memory allocation and free(). Memory allocation is may only done internally by the used ESF-IDF libraries.
The memory loss occurs on any new socket connection (HTTP(S) clients may open multiple to deliver the web content). When only one connection is open at a time there is no loss while the connection is open. However after each socket close (and open a new one) memory is lost. While the connections are open (and repeating transfers with open connections) no memory is lost. Beside own global or user context (which I haven't set) is there anything which is in my responsibility to free() ?

I tried mbedTLS static Memory (seems even worse memory loss) and dynamic allocation.

I'm a bit lost how to proceed to debug this. Are there any hints what could cause the memory eating ?
My ESP-IDF version is v4.4-dev-744-g1cb31e5-dirty.

Thanks
Kind Regards
ThomasS
Attachments
TestWebServer.zip
(3.62 KiB) Downloaded 333 times

ESP_Minatel
Posts: 361
Joined: Mon Jan 04, 2021 2:06 pm

Re: ESP HTTPS Server Leaks Memory

Postby ESP_Minatel » Mon May 03, 2021 8:32 am

Hi,

Have you created this issue on GitHub?

https://github.com/espressif/esp-idf/issues

ESP_YJM
Posts: 300
Joined: Fri Feb 26, 2021 10:30 am

Re: ESP HTTPS Server Leaks Memory

Postby ESP_YJM » Thu May 06, 2021 3:12 am

I think it is a normal issue. Although the socket is closed by the server, and server only close TX and wait client send FIN to server. If client not send FIN to server, the socket in server will enter TIME_WAIT state, this socket allocated memory will not be freed immediately until the function tcp_tmr() automatic recycling it. You can add print on tcp_tw_pcbs and check whether the lost memory in here.

TSchmidt
Posts: 3
Joined: Sat May 01, 2021 10:16 pm

Re: ESP HTTPS Server Leaks Memory

Postby TSchmidt » Thu May 06, 2021 6:28 pm

Hello,
thanks so far for your answers. I tried to go further myself - I have left the device running with a looping task which displays every 30s the free heap. Also after 1h the memory is not released. If i register a close_fn in the HTTPD config and doing a esp_tls_conn_delete() in there:

Code: Select all

void onClose(httpd_handle_t hd, int sockfd) {
    esp_tls_t *tls = httpd_sess_get_transport_ctx(hd, sockfd);
    esp_tls_conn_delete(tls);
    ESP_LOGI(TAG, "Closing Socket %d, Heap:%d", sockfd, esp_get_free_heap_size());
}
I get most of the memory released - but not all. So Memory seems not to be freed automatically even not after a time >1h. I need may to clean up something manually but I don't know what.

Is that really expected ?
May should open an issue at GitHub ...

Thanks a lot for your hints !
BR
ThomasS

ESP_YJM
Posts: 300
Joined: Fri Feb 26, 2021 10:30 am

Re: ESP HTTPS Server Leaks Memory

Postby ESP_YJM » Fri May 07, 2021 3:12 am

Could you please modify code as follow to test. First need enable SO_LINGER function(idf.py menuconfig->component config->lwip->Enable SO_LINGER processing). This
  1. void onClose(httpd_handle_t hd, int sockfd) {
  2.     esp_tls_t *tls = httpd_sess_get_transport_ctx(hd, sockfd);
  3.     esp_tls_conn_delete(tls);
  4.     struct linger linger;
  5.     linger.l_onoff = 1;
  6.     linger.l_linger = 0;
  7.     if (setsockopt(sockfd, SOL_SOCKET, SO_LINGER, &linger, sizeof(linger)) == 0) {
  8.          ESP_LOGI(TAG, "Set SO_LINGER success");
  9.          close(sockfd); // Now call close will send RST to peer and free pcb immediately.
  10.     }
  11.  
  12.     ESP_LOGI(TAG, "Closing Socket %d, Heap:%d", sockfd, esp_get_free_heap_size());
  13. }
Last edited by ESP_YJM on Fri Oct 08, 2021 7:07 am, edited 1 time in total.

TSchmidt
Posts: 3
Joined: Sat May 01, 2021 10:16 pm

Re: ESP HTTPS Server Leaks Memory

Postby TSchmidt » Tue May 25, 2021 11:45 pm

Hello,
this looks pretty much stable now. Unfortunately it is not exactly returning back to the heap value it had before opening a connection to the server and close it again. Means the heap utilization got a bit "noisy". So it is hard to say it is really freeing the memory all correctly. I need to see what are the memory debugging options (like electric fence) in some of the next steps.
Additionally it was difficult that WebSocket Server part and static server part are not interfering and destroying their sockets mutually. Somehow I faced the situation that there was a WS socket closed by the static web server. I overcame the situation by checking the session context which is set for the WS and not set for the static web server.
However after a number of stress tests it recovers quite nicely - memory is stable - still after plenty of user agent access.

Thanks for the help
Kind Regards
Thomas S.

Baldhead
Posts: 433
Joined: Sun Mar 31, 2019 5:16 am

Re: ESP HTTPS Server Leaks Memory

Postby Baldhead » Mon Oct 04, 2021 5:38 pm

ESP_YJM wrote:
Fri May 07, 2021 3:12 am
Could you please modify code as follow to test. First need enable SO_LINGER function(idf.py menuconfig->component config->lwip->Enable SO_LINGER processing). This
  1. void onClose(httpd_handle_t hd, int sockfd) {
  2.     esp_tls_t *tls = httpd_sess_get_transport_ctx(hd, sockfd);
  3.     esp_tls_conn_delete(tls);
  4.     struct linger linger;
  5.     linger.l_onoff = 1;
  6.     linger.l_linger = 0;
  7.     if (setsockopt(sockfd, SOL_SOCKET, SO_LINGER, &linger, sizeof(linger) == 0) {
  8.          ESP_LOGI(TAG, "Set SO_LINGER success");
  9.          close(sockfd); // Now call close will send RST to peer and free pcb immediately.
  10.     }
  11.  
  12.     ESP_LOGI(TAG, "Closing Socket %d, Heap:%d", sockfd, esp_get_free_heap_size());
  13. }

Hi @ESP_YJM,

I think that this line of code are incorrect "if( setsockopt(sockfd, SOL_SOCKET, SO_LINGER, &linger, sizeof(linger) == 0 )".

Should be:
  1. if( setsockopt( sockfd, SOL_SOCKET, SO_LINGER, &linger, sizeof(linger) ) == 0 )
  2. or
  3. int ret = setsockopt(sockfd, SOL_SOCKET, SO_LINGER, &linger, sizeof(linger) );
  4. if( ret == 0 )
  5. .....

The strange thing is that neither the compiler nor the visual studio code showed the error.

ESP_YJM
Posts: 300
Joined: Fri Feb 26, 2021 10:30 am

Re: ESP HTTPS Server Leaks Memory

Postby ESP_YJM » Fri Oct 08, 2021 7:06 am

Thanks for pointing the error. I will fix the comments code.

Who is online

Users browsing this forum: Bing [Bot], Google [Bot] and 128 guests