DOS due to webserver sockets not closing (easy repro) [IDFGH-3437]

saschah
Posts: 1
Joined: Thu Jun 04, 2020 6:37 am

DOS due to webserver sockets not closing (easy repro) [IDFGH-3437]

Postby saschah » Thu Jun 04, 2020 10:39 pm

Hi,

I am working on an ESP32-based IOT node that runs a simple web interface. During the last few months it happened quite often that it became unavailable - i.e. the webserver would stop responding, but I could still ping the device.

After some experimentation I finally found an easy way to repro the issue. I can reproduce it with both the built-in httpd as well as mongoose, indicating that the problem is probably lower in the TCP/IP stack, probably LWIP? And I am not sure if it's a bug, or WAI and I am simply missing a flag.

Here is the repro:
  • 0. Setup: Connect the ESP32 to the wifi and start-up either httpd or mongoose. Have a phone ready, I used a Pixel phone with Chrome, on the same Wifi network.
  • 1. Make a request to the webserver running on the ESP32.
  • 2. Now disconnect the phone from wifi by disabling it. (This is crucial, since the socket will now remain open, as the client wasn't able to close it properly before losing connection to the network.
  • 3. Re-enable WIFI on your phone.
  • 4. GOTO 1 and repeat
The default max open socket count for httpd is 10, so after about 10 requests you will see that it is no longer responding. When using mongoose, you'll instead get a "CONNECTION RESET".

The problem is that browsers by default will keep the connection alive in order to make future requests faster. However, by disconnecting the device from the network before the client/browser is able to close the connection, yields in the socket staying open on the ESP and never closing. When the device re-enters the network and makes a new request, it will open up a new connection. And so they pile up until max socket count is reached.

My solution for now is to send "Connection: close" in the HTTP response header so that Chrome and other browsers close the connection immediately. But it bugs me that I am dependent on the client behaving well, and it thus being very easy to DOS the device.

Am I missing a timeout setting that force closes open sockets after they have been idle for a while?

Thank you kindly
Sascha

PS: This might be the root cause of https://github.com/espressif/esp-idf/issues/3851

ESP_Alvin
Posts: 195
Joined: Thu May 17, 2018 2:26 am

Re: DOS due to webserver sockets not closing (easy repro) [IDFGH-3437]

Postby ESP_Alvin » Fri Jun 05, 2020 12:49 am

Moderator's note: edit the topic title for issue tracking, thanks for reporting.

ESP_Shubham
Posts: 8
Joined: Mon Oct 07, 2019 6:09 am

Re: DOS due to webserver sockets not closing (easy repro) [IDFGH-3437]

Postby ESP_Shubham » Fri Jun 26, 2020 7:03 am

Hi Sascha,

Thanks for reporting and mentioning detailed steps to reproduce this issue.

We have created a fix and will be available on GitHub soon.
I have also attached a patch which fixes the issue.
You can apply the patch and set lru_purge_enable true.

Thanks,
Shubham
Attachments
fix_lru_counter.patch.zip
(1.11 KiB) Downloaded 506 times

Who is online

Users browsing this forum: No registered users and 248 guests