Robust method to receive data from UART?

newsettler_AI
Posts: 121
Joined: Wed Apr 05, 2017 12:49 pm

Robust method to receive data from UART?

Postby newsettler_AI » Wed May 02, 2018 11:34 am

Hi!

I have connected ESP32 to other controller(STM32)and I need to send/receive data by UART, something like command-answer (esp32 sends command and stm32 sends responce). Also there is TCP server on ESP32 side, but I thinks its not the issue here.

I'm sendting 10..50 bytes from esp32 and expecting to receive 150...170 bytes from stm32.

Problem:

I'm receiving only half of stm's packet, i.e:

instead
01 02 03 [...] 168 169 170

I have
77 78 79 [...] 168 169 170

I have devided rx and tx to different tasks. In RX task, I'm using uart events.
Part of code:

Code: Select all

#define BUF_SIZE  (512)

static QueueHandle_t uart_queue;
void uart_init(void)
{
    uart_config_t uart_config = {
        .baud_rate = 115200,
        .data_bits = UART_DATA_8_BITS,
        .parity    = UART_PARITY_DISABLE,
        .stop_bits = UART_STOP_BITS_1,
        .flow_ctrl = UART_HW_FLOWCTRL_DISABLE
    };
    uart_param_config(MY_UART, &uart_config);
    uart_set_pin(MY_UART, USART_TXD, USART_RXD, USART_RTS, USART_CTS);
    //uart_driver_install(RS485_UART, BUF_SIZE*2, BUF_SIZE, 0, NULL, 0);
    uart_driver_install(MY_UART, BUF_SIZE * 2, BUF_SIZE * 2, 20, &uart_queue, 0);
}

// sending buffers to UART
void UART_send_data(DataBuffer_t buffToSend)
{
	switch(buffToSend)
	{
	case E_DATA_1:
		uart_write_bytes(RS485_UART, (char*)MyData.Buf_1, MyData.Buf_1_len);
		break;

	case E_DATA_2:
		uart_write_bytes(RS485_UART, (char*)MyData.Buf_2, MyData.Buf_2_len);
		break;

    default:
        break;
	}
}

static void tx_task(void *pvParameters)
{
	while(1)
	{
		//send data1 to UART
		if(MyData.got_packet_1 == true)
		{
			UART_send_data(E_DATA_1);
			MyData.got_packet_1 = false;
		}

		//send data2 to UART
		if(MyData.got_packet_2 == true)
		{
			UART_send_data(E_DATA_2);
			MyData.got_packet_2= false;
		}
	}
}

static void RS485_rx_task(void *pvParameters)
{
	uart_event_t event;
    int len = 0;

    while (1)
    {
    	if(xQueueReceive(uart_queue, (void * )&event, (portTickType)portMAX_DELAY))
    	{
    		ESP_LOGI(UART_TAG, "uart[%d] event:", MY_UART);

    		if(event.type == UART_DATA)
    		{
    			ESP_LOGI(UART_TAG, "(UART_DATA), event type: %d", event.type);
				len = uart_read_bytes(MY_UART, MyData.UART_Buf, event.size, portMAX_DELAY);
    		}
    		else
    		{
    			ESP_LOGI(UART_TAG, "uart event type: %d", event.type);
    		}
    	}

		if(len > 0)
		{
			MyData.UART_len = len;
			MyData.UART_got_packet = true;

			else if (MyData.TCP_conn == true) // send UART data to TCP
			{
				printf("FL_UART(len:%d):%s\n",MyData.UART_len, MyData.UART_Buf);
				send(connect_socket, MyData.UART_Buf, MyData.UART_len, 0);
			}
		}
    }
}

void app_main()
{
    uart_init();
    xTaskCreate(rx_task, "rx_task", 8*1024, NULL, 12, NULL);
    xTaskCreate(tx_task, "tx_task", 8*1024, NULL, 5, NULL);
    xTaskCreate(&tcp_server_task,"tcp_server_task", 4*1024, NULL, 5, NULL);
}
and at pastebin: https://pastebin.com/PVSsHV2R

Please give me advice how to receive all data without loses.

newsettler_AI
Posts: 121
Joined: Wed Apr 05, 2017 12:49 pm

Re: Robust method to receive data from UART?

Postby newsettler_AI » Wed May 02, 2018 6:10 pm

Also, I noticed, when sending 150 bytes of data, its devided into two packets 120 + 30 in xQueueReceive.

ESP_Sprite
Posts: 9043
Joined: Thu Nov 26, 2015 4:08 am

Re: Robust method to receive data from UART?

Postby ESP_Sprite » Thu May 03, 2018 3:37 am

Well, as UART data is just a stream of bytes, there's no guarantee whatsoever that it appears as one chunk on the output side. If you effectively want to send packets over the UART, you'll have to packetize/de-packetize them yourself.

newsettler_AI
Posts: 121
Joined: Wed Apr 05, 2017 12:49 pm

Re: Robust method to receive data from UART?

Postby newsettler_AI » Thu May 03, 2018 8:58 am

ESP_Sprite wrote:Well, as UART data is just a stream of bytes, there's no guarantee whatsoever that it appears as one chunk on the output side. If you effectively want to send packets over the UART, you'll have to packetize/de-packetize them yourself.
Thanks for reply.
Can you advice how can I use interrupt to detect incoming byte? I suppose, that first part of my parcel lost because esp32 was processing another task (tcp server or some internal) and just dont have time to process it.

Incoming packets from stm32 already have header-footer structure, so if I can detect at least first incoming byte it will be more space to work around.

ESP_Sprite
Posts: 9043
Joined: Thu Nov 26, 2015 4:08 am

Re: Robust method to receive data from UART?

Postby ESP_Sprite » Thu May 03, 2018 9:25 am

I don't know, I don't see any glaring errors in your code, if any. Do you also only get one event with the 2nd 70'ish bytes of data in it?

newsettler_AI
Posts: 121
Joined: Wed Apr 05, 2017 12:49 pm

Re: Robust method to receive data from UART?

Postby newsettler_AI » Thu May 03, 2018 11:46 am

ESP_Sprite wrote:I don't know, I don't see any glaring errors in your code, if any. Do you also only get one event with the 2nd 70'ish bytes of data in it?
I have tested it more precisely and one thing I noticed.

When I got responce from stm32, there is one more event appears: "frame error" (for first part of parcel ~70 bytes)

So its looks like next:
---> parcel [150bytes] sent
---> event "UART_FRAME_ERR"
---> event "UART_DATA"
---> received packet [77 78 ... 149 150]

I have tested stm's responce with other MCU (atmel) and it received all bytes in packet correctly; stm with 3 different usb-uart dongles (ftdi, cp2102, ch340g) and all of them shows all 150 bytes without issues. So, I suppose, problem is not on stm side.
I read a bit about frame errors, in most cases reason is hardware issues (poor power supply, bad connection, EMI etc).

But I want to note:
- another MCU and usb-uart converters tested under the same conditions as esp (moreover, esp is powering from external power supply 5V@1A);
- small packets (up to 15 bytes) received without issues from stm;
- in case I sending 120+ bytes directly from usb converter to esp, anyway its devided into two events 120 + N (i.e sent 121 -> 2 events 120+1 ; 140 -> 2 envents 120+20)

Is it limitation of internal driver? Its buffer is 128 bytes as I know.

Pibbotley
Posts: 45
Joined: Fri Feb 16, 2018 7:06 pm

Re: Robust method to receive data from UART?

Postby Pibbotley » Thu May 03, 2018 6:52 pm

I too would like a fast and robust UART (or SPI) connection method between a STM32 and a ESP32 as I'd like to use a STM32 as a device driver for an OV2640 camera to stream the JPEGs to the ESP32, but I have had issues with lost bytes at speeds over 1Mbit.

newsettler_AI
Posts: 121
Joined: Wed Apr 05, 2017 12:49 pm

Re: Robust method to receive data from UART?

Postby newsettler_AI » Fri May 04, 2018 10:22 am

I havent sucseed to solve this issue, but I have make work around:

I set delay on stm32 before sending responce over UART, now esp32 receiving all 150 bytes, same as from usb-uart converters.
But still its devided on 2 packets 120+30.

I tried to set

Code: Select all

UART_FIFO_LEN           (256)
at uart.h:
https://github.com/espressif/esp-idf/bl ... uart.h#L37

But nothing changed.

I don't understand, what is 120? Why packets always have been cutted by this length?

P.S. I have cleaned&rebuilt all the project, maybe I need to do more steps to be sure that my changes have taken effect?

ESP_Sprite
Posts: 9043
Joined: Thu Nov 26, 2015 4:08 am

Re: Robust method to receive data from UART?

Postby ESP_Sprite » Sun May 06, 2018 2:47 am

The 120+30 is normal and expected: as I said, you're effectively receiving a serial stream, and the drivers can decide to break up or collate packets at any time it wants. In this case, it's probably a function of the rx threshold the driver sets, plus whatever interrupt latency there is. Also, do not change that define, as far as I know it's a reflection of what the hardware contains, not an user-changeable knob: your UART hardware isn't magically going to contain more transistors because you tweak that define.

I highly suggest you instead write code to collate the responses in your receiving codes and spotting the packet boundaries there instead.

Who is online

Users browsing this forum: Google [Bot] and 131 guests