esp_http_client_write returns -1 after a while

User avatar
gunar.kroeger
Posts: 143
Joined: Fri Jul 27, 2018 6:48 pm

esp_http_client_write returns -1 after a while

Postby gunar.kroeger » Tue Oct 30, 2018 2:54 pm

Hi everyone,

I'm uploading files via POST Multipart messages (see viewtopic.php?f=13&t=7801&p=33058#p33058)
Instead of using esp_http_client_perform, I had to use esp_http_client_write for writing the body, since the full files will not fit inside the memory.

the issue is that after about 2 seconds the esp_http_client_write returns -1
Output:

Code: Select all

I (5695) event: sta ip: 192.168.0.122, mask: 255.255.255.0, gw: 192.168.0.254
D (5695) WIFI: SYSTEM_EVENT_STA_GOT_IP
I (5695) WIFI: Connected to AP, begin http example
I (5695) WIFI: url = http://192.168.0.119:8080/api/log/upload
I (5705) WIFI: client initialized
I (5705) WIFI: sending file: LOG_181019_192602.zip
I (5705) SD: Opening file /sdcard/LOG_181019_192602.zip, rb
I (5725) WIFI: keyHeader: --MyBoundary

Content-Disposition: form-data; name="key"



${filename}


I (5725) WIFI: requestHead: --MyBoundary

Content-Disposition: form-data; name="file"; filename="LOG_181019_192602.zip"

Content-Type: application/zip




I (5745) WIFI: tail: 

--MyBoundary--




I (5745) WIFI: length: 312197
D (5905) WIFI: HTTP_EVENT_ON_CONNECTED
I (5905) WIFI: client connection open
I (5905) WIFI: keyHeader:	73
I (5905) WIFI: requestHead:	130
I (5915) WIFI: write file:	1000/1000
I (5915) WIFI: write file:	1000/1000
I (5925) WIFI: write file:	1000/1000
I (5925) WIFI: write file:	1000/1000
I (5935) WIFI: write file:	1000/1000
I (6005) WIFI: write file:	1000/1000
I (6015) WIFI: write file:	1000/1000
I (6025) WIFI: write file:	1000/1000
I (6025) WIFI: write file:	1000/1000
I (6035) WIFI: write file:	1000/1000
I (6035) WIFI: write file:	1000/1000
I (6045) WIFI: write file:	1000/1000
I (6205) WIFI: write file:	1000/1000
I (6215) WIFI: write file:	1000/1000
I (6215) WIFI: write file:	1000/1000
I (6225) WIFI: write file:	1000/1000
I (6235) WIFI: write file:	1000/1000
I (6235) WIFI: write file:	1000/1000
I (6315) WIFI: write file:	1000/1000
I (6315) WIFI: write file:	1000/1000
I (6325) WIFI: write file:	1000/1000
I (6325) WIFI: write file:	1000/1000
I (6335) WIFI: write file:	1000/1000
I (6345) WIFI: write file:	1000/1000
I (6345) WIFI: write file:	1000/1000
I (6415) WIFI: write file:	1000/1000
I (6415) WIFI: write file:	1000/1000
I (6425) WIFI: write file:	1000/1000
I (6435) WIFI: write file:	1000/1000
I (6435) WIFI: write file:	1000/1000
I (6515) WIFI: write file:	1000/1000
I (6515) WIFI: write file:	1000/1000
I (6525) WIFI: write file:	1000/1000
I (6535) WIFI: write file:	1000/1000
I (6535) WIFI: write file:	1000/1000
I (6545) WIFI: write file:	1000/1000
I (6545) WIFI: write file:	1000/1000
I (6625) WIFI: write file:	1000/1000
I (6625) WIFI: 38000/311974 bytes sent total 37.11 KiB/s
I (6625) WIFI: write file:	1000/1000
I (6625) WIFI: 39000/311974 bytes sent total 38.09 KiB/s
I (6635) WIFI: write file:	1000/1000
I (6635) WIFI: 40000/311974 bytes sent total 39.06 KiB/s
I (6645) WIFI: write file:	1000/1000
I (6645) WIFI: 41000/311974 bytes sent total 40.04 KiB/s
I (6655) WIFI: write file:	1000/1000
I (6655) WIFI: 42000/311974 bytes sent total 41.02 KiB/s
I (6715) WIFI: write file:	1000/1000
I (6715) WIFI: 43000/311974 bytes sent total 41.99 KiB/s
I (6725) WIFI: write file:	1000/1000
I (6725) WIFI: 44000/311974 bytes sent total 42.97 KiB/s
I (6735) WIFI: write file:	1000/1000
I (6735) WIFI: 45000/311974 bytes sent total 43.95 KiB/s
I (6735) WIFI: write file:	1000/1000
I (6735) WIFI: 46000/311974 bytes sent total 44.92 KiB/s
I (6745) WIFI: write file:	1000/1000
I (6745) WIFI: 47000/311974 bytes sent total 45.90 KiB/s
I (6755) WIFI: write file:	1000/1000
I (6755) WIFI: 48000/311974 bytes sent total 46.88 KiB/s
I (6765) WIFI: write file:	1000/1000
I (6765) WIFI: 49000/311974 bytes sent total 47.85 KiB/s
I (6825) WIFI: write file:	1000/1000
I (6825) WIFI: 50000/311974 bytes sent total 48.83 KiB/s
I (6825) WIFI: write file:	1000/1000
I (6835) WIFI: 51000/311974 bytes sent total 49.80 KiB/s
I (6835) WIFI: write file:	1000/1000
I (6835) WIFI: 52000/311974 bytes sent total 50.78 KiB/s
I (6845) WIFI: write file:	1000/1000
I (6845) WIFI: 53000/311974 bytes sent total 51.76 KiB/s
I (6855) WIFI: write file:	1000/1000
I (6855) WIFI: 54000/311974 bytes sent total 52.73 KiB/s
I (6855) WIFI: write file:	1000/1000
I (6855) WIFI: 55000/311974 bytes sent total 53.71 KiB/s
I (6865) WIFI: write file:	1000/1000
I (6865) WIFI: 56000/311974 bytes sent total 54.69 KiB/s
I (6925) WIFI: write file:	-1/1000
D (6925) WIFI: HTTP_EVENT_DISCONNECTED
Code:

Code: Select all

static bool sendFile(esp_http_client_handle_t client, const char *fileName)
{
	esp_err_t err;
	FILE *file;
    char fileNameWithDir[100];

    snprintf(fileNameWithDir, sizeof(fileNameWithDir), "/sdcard/%s", fileName);
    file = SD_open(fileNameWithDir, "rb");
	if(!file)
	{
		ESP_LOGW(TAG, "Could not open file to send through wifi");
		return false;
	}
	unsigned fileLenght = 0;
	struct stat st;
	if(stat(fileNameWithDir, &st) == 0)
		fileLenght = st.st_size;

	time_t fileTransferStart = 0;
	time(&fileTransferStart);

	char contentType[] = "application/zip";
    //=============================================================================================
	//key header
	char keyHeader[100] = "";
	strcat(keyHeader, "--");
	strcat(keyHeader, HTTP_BOUNDARY);
	strcat(keyHeader, "\r\n");

	strcat(keyHeader, "Content-Disposition: form-data; name=\"key\"\r\n\r\n");
	strcat(keyHeader, "${filename}\r\n");
	ESP_LOGI(TAG, "keyHeader: %s", keyHeader);

	//request header
	char requestHead[200] = "";
	strcat(requestHead, "--");
	strcat(requestHead, HTTP_BOUNDARY);
	strcat(requestHead, "\r\n");
	strcat(requestHead, "Content-Disposition: form-data; name=\"file\"; filename=\"");
	strcat(requestHead, fileName);
	strcat(requestHead, "\"\r\n");
	strcat(requestHead, "Content-Type: ");
	strcat(requestHead, contentType);
	strcat(requestHead, "\r\n\r\n");
	ESP_LOGI(TAG, "requestHead: %s", requestHead);

	//request tail
	char tail[50] = "";
	strcat(tail, "\r\n--");
	strcat(tail, HTTP_BOUNDARY);
	strcat(tail, "--\r\n\r\n");
	ESP_LOGI(TAG, "tail: %s", tail);

	//Set Content-Length
	int contentLength = strlen(keyHeader) + strlen(requestHead) + fileLenght + strlen(tail);
	ESP_LOGI(TAG, "length: %d", contentLength);
	char lengthStr[10];
	sprintf(lengthStr, "%i", contentLength);
    err = esp_http_client_set_header(client, "Content-Length", lengthStr);

    //=============================================================================================

    err = esp_http_client_open(client, HTTP_BUFFER_SIZE);
    if (err != ESP_OK) {
        ESP_LOGE(TAG, "Failed to open HTTP connection: %s", esp_err_to_name(err));
        esp_http_client_cleanup(client);
    }

	ESP_LOGI(TAG, "client connection open");

	ESP_LOGI(TAG, "keyHeader:\t%d", esp_http_client_write(client, keyHeader, strlen(keyHeader)));
	ESP_LOGI(TAG, "requestHead:\t%d", esp_http_client_write(client, requestHead, strlen(requestHead)));
	//Send file parts
	char buffer[HTTP_BUFFER_SIZE];

	unsigned fileProgress = 0;
	int length = 1;
	while(length)
	{
		length = fread(buffer, sizeof(char), HTTP_BUFFER_SIZE, file);
		if(length)
		{
			int wret = esp_http_client_write(client, buffer, length);
			ESP_LOGI(TAG, "write file:\t%d/%d", wret, length);
			if(wret < 0)
				 return false;

			fileProgress += wret;
			//ESP_LOGI(TAG, "%.*s", length, buffer);
			time_t now;
			time(&now);
			if((float)(now - fileTransferStart) / 1024 > 0)
				ESP_LOGI(TAG, "%u/%u bytes sent total %.02f KiB/s", fileProgress, fileLenght, fileProgress  / (float)(now - fileTransferStart) / 1024);
		}
	}
	SD_close(file);
	//finish Multipart request
	ESP_LOGI(TAG, "tail:\t%d", esp_http_client_write(client, tail, strlen(tail)));

	//Get response
	ESP_LOGI(TAG, "fetch_headers:\t%d", esp_http_client_fetch_headers(client));
	ESP_LOGI(TAG, "chunked:\t%d", esp_http_client_is_chunked_response(client));

	int responseLength = esp_http_client_get_content_length(client);
	ESP_LOGI(TAG, "responseLength:\t%d", responseLength);

	int status = esp_http_client_get_status_code(client);
	ESP_LOGI(TAG, "status:\t%d", status);

	if(responseLength)
	{
		if(esp_http_client_read(client, buffer, responseLength) == responseLength)
			ESP_LOGI(TAG, "Response: %.*s", responseLength, buffer);
	}

	err = esp_http_client_close(client);
    if (err != ESP_OK) {
        ESP_LOGE(TAG, "Failed to close HTTP connection: %s", esp_err_to_name(err));
        esp_http_client_cleanup(client);
    }
    return status == 200 || status == 409;
}
static void https_send_file()
{
	char url[200] = "http://192.168.0.119:8080/api/log/upload";
	ESP_LOGI(TAG, "url = %s", url);
	esp_err_t err;
	esp_http_client_config_t config = {
		.url = url,
		.event_handler = _http_event_handler,
		.cert_pem = howsmyssl_com_root_cert_pem_start,
	};
	esp_http_client_handle_t client = esp_http_client_init(&config);
	ESP_LOGI(TAG, "client initialized");

	err = esp_http_client_set_method(client, HTTP_METHOD_POST);
    if (err != ESP_OK) {
        ESP_LOGE(TAG, "Failed to set method: %s", esp_err_to_name(err));
        esp_http_client_cleanup(client);
    }

	//Set Multipart header
	char contentTypeStr[50] = "multipart/form-data; boundary=";
	strcat(contentTypeStr, HTTP_BOUNDARY);

    err = esp_http_client_set_header(client, "Content-Type", contentTypeStr);
    err = esp_http_client_set_header(client, "Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
    err = esp_http_client_set_header(client, "Accept-Encoding", "gzip,deflate");
    err = esp_http_client_set_header(client, "Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7");
    err = esp_http_client_set_header(client, "User-Agent", "Test");
    err = esp_http_client_set_header(client, "Keep-Alive", "300");
    err = esp_http_client_set_header(client, "Connection", "keep-alive");
    err = esp_http_client_set_header(client, "Accept-Language", "en-us");

    char fileName[] = "LOG_181019_192602.zip";
	ESP_LOGI(TAG, "sending file: %s", fileName);

	sendFile(client, fileName);

	err = esp_http_client_cleanup(client);
    if (err != ESP_OK) {
        ESP_LOGE(TAG, "Failed to clean HTTP connection: %s", esp_err_to_name(err));
        esp_http_client_cleanup(client);
    }

    vTaskDelete(NULL);
}
Digging inside idf code I found out that this return occurs when client->state < HTTP_STATE_REQ_COMPLETE_HEADER, but I couldn't advance from here. Any help would be apreciated
"Running was invented in 1612 by Thomas Running when he tried to walk twice at the same time."

chegewara
Posts: 2210
Joined: Wed Jun 14, 2017 9:00 pm

Re: esp_http_client_write returns -1 after a while

Postby chegewara » Wed Oct 31, 2018 7:28 am


User avatar
gunar.kroeger
Posts: 143
Joined: Fri Jul 27, 2018 6:48 pm

Re: esp_http_client_write returns -1 after a while

Postby gunar.kroeger » Wed Oct 31, 2018 5:10 pm

Ok, I am an idiot.

Looking at the documentation for esp_http_client_write:
@param[in] len This value must not be larger than the write_len parameter provided to esp_http_client_open()
I thought that that I should open the connection with the length of the maximum esp_http_client_write call.
Turns out it is pretty clear from:
**
* @brief This function will be open the connection, write all header strings and return
*
* @param[in] client The esp_http_client handle
* @param[in] write_len HTTP Content length need to write to the server
*
* @return
* - ESP_OK
* - ESP_FAIL
*/
esp_err_t esp_http_client_open(esp_http_client_handle_t client, int write_len);
that it should be equals to the sum of all esp_http_client_write calls...

Now I have everything working (:
"Running was invented in 1612 by Thomas Running when he tried to walk twice at the same time."

serg123
Posts: 1
Joined: Mon Apr 12, 2021 9:35 am

Re: esp_http_client_write returns -1 after a while

Postby serg123 » Mon Apr 12, 2021 11:20 am

Hi gunar.kroeger,
You mean that esp_http_client_open should be with length parammeter equal to ContentLength?
err = esp_http_client_open(client, contentLength); ?

Who is online

Users browsing this forum: lstpvtltd and 133 guests