How to read the size of available data on mbedtls?

[copercini]

Hi, I am porting the WiFiClientSecure arduino library based of WifiClient of ESP32, but the problem is "client.available()" method, how to know if the client has data to read or not?

using ioctl(ssl_client->socket, FIONREAD, &count);
or
mbedtls_ssl_get_bytes_avail(&ssl_client->ssl_ctx));

it always return 0, even when there is.


Here is the full code:
https://github.com/copercini/arduino-es ... ientSecure


The problem is here:
https://github.com/copercini/arduino-es ... e.cpp#L157
https://github.com/copercini/arduino-es ... t.cpp#L286

[ESP_Angus]

This is partially a restriction in the design of mbedTLS, and partially a restriction in the design/necessities of the TLS protocol. Data has to be received via the underlying plain TCP socket, then full TLS messages have to be decrypted, and then that data is available to read back into the user program.

So data can be waiting up to 3 places: unread on the TCP socket, read on the TCP socket but not yet decrypted (because a full TLS message hasn't arrived), or decrypted and available to read back into the user program.

"ioctl(ssl_client->socket, FIONREAD, &count);" should give you the first of these counts (unread on socket). You'll need to enable "Enable SO RCVBUF" option in the esp-idf lwip component configuration (Arduino's precompiled libraries have this enabled as of early Jan.) This ioctl can't account for encrypted data which has been read from the TCP socket and not decrypted yet. It also doesn't correspond to the number of bytes of decrypted data (N bytes of encrypted data waiting on the socket can correspond to any number of bytes after decryption, because of TLS message overhead, housekeeping/management/empty messages, etc.)

"mbedtls_ssl_get_bytes_avail() " gives you the last of these counts (already decrypted data), but can only account for decrypted messages. Calling this doesn't cause any data to be read from the TCP socket or decrypted.

I believe you call call mbedtls_ssl_read(ssl, NULL, 0) and then mbedtls_ssl_get_bytes_avail(ssl). The first call will try to read from the socket and process/decrypt incoming data if possible. Then the second call will return the number of decrypted bytes which would have been available to read after the first call.

Bear in mind that if you follow that approach (making a zero-size mbedtls_ssl_read then calling mbedtls_ssl_get_bytes_avail()) and you get a number of bytes which is non-zero but less than you need, you probably still need to read those bytes out in order to free up mbedTLS' internal buffers for decrypting the next TLS message - the TLS session won't progress until you do this. There's no way to avoid having to do some buffering on the caller's side.

EDIT: I realised this was a bit of a rambling answer. I think that calling mbedtls_ssl_read(ssl, NULL, 0); mbedtls_ssl_get_bytes_avail() in the available() method is probably as good as you can hope for. However you may also need to make the socket non-blocking for the duration of the firstcall.

[copercini]

First thanks for the answer and explanation =)

I think that calling mbedtls_ssl_read(ssl, NULL, 0); mbedtls_ssl_get_bytes_avail() in the available() method is probably as good as you can hope for.

It works, but not how it should...
When 'mbedtls_ssl_read()' is called, it timeout if buffer is empty, 'mbedtls_ssl_get_bytes_avail()' return 0 and MQTT publishes a message. The major problem is the blocking time on arduino IDE.

However you may also need to make the socket non-blocking for the duration of the firstcall.

What is the best implementation for this? just a 'fcntl( ssl_client->fd, F_SETFL, fcntl( ssl_client->fd, F_GETFL, 0 ) | O_NONBLOCK );' after create the sockets connection?

[copercini]

I think it's solved!

After the correct implementation of non-blocking sockets, the things seem to be working fine