http_req_t passed as void* to FreeRTOS task causes Guru Meditation Error (LoadProhibited)

HohkeiV
Posts: 6
Joined: Sun Jan 16, 2022 6:39 pm

http_req_t passed as void* to FreeRTOS task causes Guru Meditation Error (LoadProhibited)

Postby HohkeiV » Sun Jan 16, 2022 8:35 pm

Take a simplest http server example, and instead of handling a httpd request "directly", delegate the handling to a task, through a pointer to httpd_req_t. In my case, this causes a stack overflow. Why and how to fix it?

After some digging, it turns out that void *aux, member of httpd_req_t is NULL when referenced from inside the task (but is non-NULL just before entering the task (see below). If it's an issue of allocation/copying something somewhere properly, then I'd like to know this by example, since aux is patently a private variable.

Here's the stack trace:
  1. Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.
  2.  
  3. Core  0 register dump:
  4. PC      : 0x40100e47  PS      : 0x00060d30  A0      : 0x800d7f4a  A1      : 0x3ffcfc40  
  5. 0x40100e47: httpd_resp_set_hdr at components/esp_http_server/src/httpd_txrx.c:176
  6.  
  7. A2      : 0x3ffb1dc4  A3      : 0x3f40452c  A4      : 0x3f404520  A5      : 0x3ffaf02c  
  8. A6      : 0x3f403910  A7      : 0x3f4045c0  A8      : 0x00000000  A9      : 0x00000000  
  9. A10     : 0x00000000  A11     : 0x3ffaf02c  A12     : 0x00000001  A13     : 0x00000013  
  10. A14     : 0x3ffaf02c  A15     : 0x3f4044fc  SAR     : 0x00000000  EXCCAUSE: 0x0000001c  
  11. EXCVADDR: 0x0000021c  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xfffffffc
Below is the offending example. (I put together debug_req_type to print out some address values).
  1. void indirect_task_fn (void*par)
  2. {
  3.     debug_req_type ("task", par);  /* prints some info about par, see below */
  4.     simple_handler(par);
  5.     vTaskDelete(NULL); /* Does not reach this point */
  6. }
  7. /* debug_req_type: ooops
  8. task : Debugging http_req_t
  9. task : pointer=0x3ffb1dc4
  10. task : aux=0x0
  11. ...
  12. */
  13.  
  14. esp_err_t indirect_handler(httpd_req_t *req)
  15. {
  16.     debug_req_type("indirect handler", req); /* see below */
  17.     BaseType_t status = xTaskCreate(indirect_task_fn,"indirect task",100,req,1,NULL);
  18.     if(status != pdTRUE) /* do something */
  19.     return ESP_OK;
  20. }
  21. /* So far so good
  22. indirect handler : pointer=0x3ffb1dc4
  23. indirect handler : aux=0x3ffb1fe8
  24. ...
  25. */
  26.  
  27. const httpd_uri_t indirect = {
  28.     .uri     = "/indirect",
  29.     .method  = HTTP_GET,
  30.     .handler = indirect_handler,
  31.     .user_ctx = NULL
  32. };
  33.  
  34. httpd_register_uri_handler(server, &indirect);
A similar working case, just for comparison.
  1. esp_err_t simple_handler(httpd_req_t *req)
  2. {
  3.     debug_req_type("simple handler", req);
  4.     httpd_resp_set_hdr(req, "Content-type", "text/html"); /* irrelevant, but fine */
  5.     httpd_resp_send(req, "<html><head></head><body><h1>debugging is fun</h1><div><p>(sometimes)</p></div></body></html>", HTTPD_RESP_USE_STRLEN); /* same here */
  6.     return ESP_OK;
  7. }
  8. /* All is fine
  9. simple handler : pointer=0x3ffb1dc4
  10. simple handler : aux=0x3ffb1fe8
  11. ...
  12. */
  13.  
  14. const httpd_uri_t simple = {
  15.     .uri     = "/simple",
  16.     .method  = HTTP_GET,
  17.     .handler = simple_handler,
  18.     .user_ctx = NULL
  19. };
  20.  
  21. httpd_register_uri_handler(server, &simple); /* So far so good. */
Last edited by HohkeiV on Mon Jan 17, 2022 9:12 am, edited 2 times in total.

ESP_Sprite
Posts: 6393
Joined: Thu Nov 26, 2015 4:08 am

Re: http_req_t passed as void* to FreeRTOS task causes Guru Meditation Error (LoadProhibited)

Postby ESP_Sprite » Mon Jan 17, 2022 1:34 am

First of all, I'd advise against starting a task from an user-initiated action... what if, for instance, they press refresh a bunch of times? The better option is to start a task on startup, then hand it work using a semaphore or queue or something.

Secondly, your issue is that the lifetime of the 'req' variable is only guaranteed during the execution time of the indirect_handler call. As soon as that call is done, the HTTP server is free to use that memory for whatever it wants. If you need to use the data anyway, you should make sure you copy it to some memory you own yourself before the function exits.

HohkeiV
Posts: 6
Joined: Sun Jan 16, 2022 6:39 pm

Re: http_req_t passed as void* to FreeRTOS task causes Guru Meditation Error (LoadProhibited)

Postby HohkeiV » Mon Jan 17, 2022 8:50 am

Your 1st point makes perfect sense. I am experimenting with the httpd component with the goal of using it in situations of shared access to a resource and your suggestion is certainly going in the right direction, but this was the simplest warm-up example I could think of. Concerning your second point, I get that the owner has the prerogative to free its resources, I just don't see how the "indirect_handler" could trigger that.

ESP_Sprite
Posts: 6393
Joined: Thu Nov 26, 2015 4:08 am

Re: http_req_t passed as void* to FreeRTOS task causes Guru Meditation Error (LoadProhibited)

Postby ESP_Sprite » Mon Jan 17, 2022 9:08 am

It's about memory ownership, as in who gets to decide what to do with the memory. In this case, the memory ownership of 'req' and all its members is with the http server: it allocated that memory and per general programming conventions, by default it gets to re-use it for whatever it wants or to stash whatever into it. The only guarantee you have about the memory (again per general programming conventions) is that while your indirect_handler function is running, the memory will stay consistent and will have whatever contents the API documents it should have. Before that call is made and after you return from that call, all bets are off. The webserver can dump the contents of the Bible xor-encrypted with the Anarchists Cookbook into that memory for all you know, and it would be within its right to do so.

The issue now is what you do within that function: you create a task and pass it that memory region. Note that when that task will actually run is undefined: it might start up the exact moment xTaskCreate is fired up, it might start up millions of cycles later. However, xTaskCreate itself does not care; it will immediately return; it *created* the (perhaps-not-directly-running) task, so its work is done. After that your function happily returns ESP_OK and the function exits. Note what I said about what happens if the function exits? The webserver is free to do anything, be it holy or unholy, to the memory of the req structure. Nuking the aux pointer is only one of the many options it has.

So a few cycles later, the OS decides it's time to perhaps wake up your newly-birthed task and have it do some work. It inspects the req structure and finds it mangled! However, is it the http servers fault? No, it is not: your function ended and that means control over the req memory is back with the webserver. It was free to re-use it, not knowing that you started something that would need it later, and in this particular case, it decided that nuking the aux member, for whatever reason, was the prudent thing to do.

HohkeiV
Posts: 6
Joined: Sun Jan 16, 2022 6:39 pm

Re: http_req_t passed as void* to FreeRTOS task causes Guru Meditation Error (LoadProhibited)

Postby HohkeiV » Wed Jan 19, 2022 12:44 pm

Thanks ESP_Sprite for a colorful illustration of what was wrong with my code. The take home message has reached the destination.

Which is: don't try to grab the memory that is not mine for taking.

On the other hand, I think we'll agree that such details as memory guarantees are poorly if at all documented in ESP-IDF? I still have to guess that, in this particular case, memory should stay consistent for the duration of the handler (a hint to this is contained in ESP_Sprite's sermon style post). So I can still make it work (hopefully following general programming conventions), provided that the handler could be forced to enter a blocked state until the task has finished reading the memory in question.

For anyone who might be interested, the easiest way to do it is by using binary semaphores: the task gives a semaphore, and the handler must take it before it can return. That's it. It even works with frequent reloads, thanks to the semaphore (whether that is a good thing to do in practice is not a concern for the purposes of this exercise).

Here is a tested 'conceptual' example, with only 4 lines added with respect to my original post.
  1. SemaphoreHandle_t binarySemaphore = NULL; /* L1 */
  2.  
  3. void indirect_task_fn (void*par)
  4. {
  5.     /* do some work */
  6.     xSemaphoreGive(binarySemaphore); /* L2: handler, take it! */
  7.     vTaskDelete(NULL);
  8. }
  9.      
  10. esp_err_t indirect_handler(httpd_req_t *req)
  11. {
  12.     BaseType_t status = xTaskCreate(indirect_task_fn,"indirect task",100,req,1,NULL);
  13.     if(status != pdTRUE) {/* handle errors */}
  14.     xSemaphoreTake(binarySemaphore,portMAX_DELAY); /* L3: die, sucker */
  15.     return ESP_OK;
  16. }
  17.      
  18. const httpd_uri_t indirect = {
  19.     .uri     = "/indirect",
  20.     .method  = HTTP_GET,
  21.     .handler = indirect_handler,
  22.     .user_ctx = NULL
  23. };
  24.  
  25. int app_main (void)
  26. {
  27.     /* other stuff */
  28.     httpd_register_uri_handler(server, &indirect);
  29.     binarySemaphore = xSemaphoreCreateBinary(); /* L4 */
  30.     if(binarySemaphore == NULL) { /* handle errors */ }
  31.     /* other stuff */
  32. }

chegewara
Posts: 1899
Joined: Wed Jun 14, 2017 9:00 pm

Re: http_req_t passed as void* to FreeRTOS task causes Guru Meditation Error (LoadProhibited)

Postby chegewara » Wed Jan 19, 2022 1:04 pm

HohkeiV wrote:
Wed Jan 19, 2022 12:44 pm
  1. SemaphoreHandle_t binarySemaphore = NULL; /* L1 */
  2.  
  3. void indirect_task_fn (void*par)
  4. {
  5.     /* do some work */
  6.     xSemaphoreGive(binarySemaphore); /* L2: handler, take it! */
  7.     vTaskDelete(NULL);
  8. }
  9.      
  10. esp_err_t indirect_handler(httpd_req_t *req)
  11. {
  12.     BaseType_t status = xTaskCreate(indirect_task_fn,"indirect task",100,req,1,NULL);
  13.     if(status != pdTRUE) {/* handle errors */}
  14.     xSemaphoreTake(binarySemaphore,portMAX_DELAY); /* L3: die, sucker */
  15.     return ESP_OK;
  16. }
  17.      
  18. const httpd_uri_t indirect = {
  19.     .uri     = "/indirect",
  20.     .method  = HTTP_GET,
  21.     .handler = indirect_handler,
  22.     .user_ctx = NULL
  23. };
  24.  
  25. int app_main (void)
  26. {
  27.     /* other stuff */
  28.     httpd_register_uri_handler(server, &indirect);
  29.     binarySemaphore = xSemaphoreCreateBinary(); /* L4 */
  30.     if(binarySemaphore == NULL) { /* handle errors */ }
  31.     /* other stuff */
  32. }
It is probably not bad code, but may bite you in the but when more than one client will try to connect at the same time.

HohkeiV
Posts: 6
Joined: Sun Jan 16, 2022 6:39 pm

Re: http_req_t passed as void* to FreeRTOS task causes Guru Meditation Error (LoadProhibited)

Postby HohkeiV » Wed Jan 19, 2022 1:20 pm

The semaphore makes sure only one task is running at any given time, the others wait for their turn (if by 'the same time' you mean 'while a task is running')

chegewara
Posts: 1899
Joined: Wed Jun 14, 2017 9:00 pm

Re: http_req_t passed as void* to FreeRTOS task causes Guru Meditation Error (LoadProhibited)

Postby chegewara » Wed Jan 19, 2022 1:26 pm

Yes, and semaphore makes sure that client wont get response before that task wont give semaphore back and wont let another client request, which will cause web browser to stuck for some time. How long depends how long the task will process data and keep the semaphore.
HohkeiV wrote:if by 'the same time' you mean 'while a task is running'
At the same time i mean when another request from browser will be issued before the task will give semaphore back.



But it may be a bit of topic here.

HohkeiV
Posts: 6
Joined: Sun Jan 16, 2022 6:39 pm

Re: http_req_t passed as void* to FreeRTOS task causes Guru Meditation Error (LoadProhibited)

Postby HohkeiV » Wed Jan 19, 2022 1:32 pm

Thank you very much for your feedback.

Who is online

Users browsing this forum: rtborg and 32 guests