Why doesn't NVS Encryption use the standard Flash Encryption mechanism?

EmilenL
Posts: 15
Joined: Sun Oct 17, 2021 5:54 pm

Why doesn't NVS Encryption use the standard Flash Encryption mechanism?

Postby EmilenL » Wed Jul 06, 2022 5:34 pm

The NVS documentation says that "NVS is not directly compatible with the ESP32 flash encryption system". Instead the user is required to have a separate encrypted nvs_key partition to store another set of custom keys used only for nvs.

The page header (32 bytes) and the Entry state bitmap (32 bytes) are stored unencrypted while all the entries (32 bytes each) are stored encrypted using AES-XTS with the custom keys. Looking at nvs_encrypted_partition.cpp in the esp-idf, I see that the encryption/decryption is done manually using mbedtls_aes_crypt_xts.

What I don't understand is why the entries are encrypted in a custom way, instead of just using the standard flash api for this? When flash encryption is enabled, the methods esp_partition_write and esp_partition_read can transparently encrypt/decrypt the data using the hardware cache or encryption block and the methods esp_partition_write_raw and esp_partition_read_raw could be used for the unencrypted header that must be stored in plaintext. That should be much more efficient than manually encrypting/decrypting the data, as well as avoiding to have to create an nvs_key partition, right? Can anyone explain why that more obvious approach is not used?

ESP_Sprite
Posts: 8921
Joined: Thu Nov 26, 2015 4:08 am

Re: Why doesn't NVS Encryption use the standard Flash Encryption mechanism?

Postby ESP_Sprite » Thu Jul 07, 2022 12:58 am

I'm not a NVS expert, but generally an issue with flash encryption is that you always need to write 32 bytes in one go, and that the standard behaviour of flash (you can always make a bit go from 1 to 0, regardless of the data around it) is gone. Could be that NVS relies on that.

WiFive
Posts: 3529
Joined: Tue Dec 01, 2015 7:35 am

Re: Why doesn't NVS Encryption use the standard Flash Encryption mechanism?

Postby WiFive » Thu Jul 07, 2022 2:16 am

Interesting question because flash encryption already uses a tweak key and operates on 32 byte blocks.

Maybe so pre generated partitions are still possible within the flow of secure boot and flash encryption. Otherwise you would have to generate and use flash encryption keys off device or the bootloader would have to know how to encrypt nvs partitions.

Also it is likely that an attacker could reliably guess some plaintext for nvs based on user settings or other available values and could make the main flash encryption key more vulnerable.

EmilenL
Posts: 15
Joined: Sun Oct 17, 2021 5:54 pm

Re: Why doesn't NVS Encryption use the standard Flash Encryption mechanism?

Postby EmilenL » Thu Jul 07, 2022 9:38 am

ESP_Sprite wrote:
Thu Jul 07, 2022 12:58 am
I'm not a NVS expert, but generally an issue with flash encryption is that you always need to write 32 bytes in one go, and that the standard behaviour of flash (you can always make a bit go from 1 to 0, regardless of the data around it) is gone. Could be that NVS relies on that.
This is the exact reason why the page header and entry state table are stored unencrypted; so that individual bits can be written. The entries (32 bytes each) are always written 32 bytes in one go, which is supported by Flash Encryption, as WiFive mentions, so this is cannot be a reason why standard Flash Encryption could not be used.
WiFive wrote:
Thu Jul 07, 2022 2:16 am
Maybe so pre generated partitions are still possible within the flow of secure boot and flash encryption. Otherwise you would have to generate and use flash encryption keys off device or the bootloader would have to know how to encrypt nvs partitions.
This seems like a minor implementation detail. We already have the "idf.py encrypted-app-flash" command which sends an unencrypted app image to the device, which the device then encrypts using the efuse keys and writes to flash. The other flow that is used when enabling encryption on a previously unencrypted device which already has contents on its flash, where the bootloader encrypts the whole flash on-the-fly could easily be patched to handle nvs partitions, by not encrypting the first 64 bytes in each 4K page and unwritten entries (as indicated by the entry state table) containing only 0xff bytes.
WiFive wrote:
Thu Jul 07, 2022 2:16 am
Also it is likely that an attacker could reliably guess some plaintext for nvs based on user settings or other available values and could make the main flash encryption key more vulnerable.
AES is constructed so that even if a plaintext and its corresponding ciphertext is known, it is "impossible" to guess the key. I would say that it is more secure to use Flash Encryption directly, since then the key is stored non-readable by software in efuse. The NVS key, on the other hand, is stored in RAM and could potentially be extracted by an attacker if a buffer overflow bug or similar exists.

EmilenL
Posts: 15
Joined: Sun Oct 17, 2021 5:54 pm

Re: Why doesn't NVS Encryption use the standard Flash Encryption mechanism?

Postby EmilenL » Thu Aug 18, 2022 3:57 pm

Ok so since this commit https://github.com/espressif/esp-idf/co ... f76a996491, it appears the generic flash encryption is now supported by nvs out-of-the-box, since the partition api now supports both raw and non-raw read/write methods, where the non-raw methods use encryption in case encryption is enabled, and the raw methods always bypasses encryption.

I decided to test this out. It was more simple than I expected. The only thing I had to change was remove this line https://github.com/espressif/esp-idf/bl ... up.cpp#L22, mark the nvs partition as "encrypted" in the partition table and make sure the NVS_ENCRYPTION Kconfig option is set to 0.

Nvs now seems to work perfectly fine when flash encryption is enabled in efuse. If I use the esptool to read the raw flash contents, I can see that the page header and entry state bits are non-encrypted (as they should be) and all the entries themselves are encrypted.

ESP_Mahavir
Posts: 188
Joined: Wed Jan 24, 2018 6:51 am

Re: Why doesn't NVS Encryption use the standard Flash Encryption mechanism?

Postby ESP_Mahavir » Fri Aug 19, 2022 3:09 am

Hello,

Is it the case that you had "encrypted" flag set in front of "nvs" partition in your partition table? If yes, then please refer to fix and linked issues in the commit https://github.com/espressif/esp-idf/co ... 66b64adfcd. This may help you.

Thanks.

WiFive
Posts: 3529
Joined: Tue Dec 01, 2015 7:35 am

Re: Why doesn't NVS Encryption use the standard Flash Encryption mechanism?

Postby WiFive » Sun Aug 21, 2022 2:26 am

ESP_Mahavir wrote:
Fri Aug 19, 2022 3:09 am
Hello,

Is it the case that you had "encrypted" flag set in front of "nvs" partition in your partition table? If yes, then please refer to fix and linked issues in the commit https://github.com/espressif/esp-idf/co ... 66b64adfcd. This may help you.

Thanks.
He wants the partition to be treated as encrypted so that fix which forces it to be treated as unencrypted is not helpful in this case.

kokonuts
Posts: 15
Joined: Tue Jul 11, 2017 6:39 pm

Re: Why doesn't NVS Encryption use the standard Flash Encryption mechanism?

Postby kokonuts » Sun Feb 26, 2023 1:17 pm

EmilenL wrote:
Thu Aug 18, 2022 3:57 pm
Ok so since this commit https://github.com/espressif/esp-idf/co ... f76a996491, it appears the generic flash encryption is now supported by nvs out-of-the-box, since the partition api now supports both raw and non-raw read/write methods, where the non-raw methods use encryption in case encryption is enabled, and the raw methods always bypasses encryption.

I decided to test this out. It was more simple than I expected. The only thing I had to change was remove this line https://github.com/espressif/esp-idf/bl ... up.cpp#L22, mark the nvs partition as "encrypted" in the partition table and make sure the NVS_ENCRYPTION Kconfig option is set to 0.

Nvs now seems to work perfectly fine when flash encryption is enabled in efuse. If I use the esptool to read the raw flash contents, I can see that the page header and entry state bits are non-encrypted (as they should be) and all the entries themselves are encrypted.
Hi, I was wondering the same thing and I landed here.So if generic flash encryption is now supported by nvs out-of-the-box, why do the docs still ask for the whole mess of adding own keys for NVS encryption?
What is the downside of following your modification of " remove this line " to get it to work?

EmilenL
Posts: 15
Joined: Sun Oct 17, 2021 5:54 pm

Re: Why doesn't NVS Encryption use the standard Flash Encryption mechanism?

Postby EmilenL » Thu Mar 09, 2023 6:59 am

kokonuts wrote:
Sun Feb 26, 2023 1:17 pm
Hi, I was wondering the same thing and I landed here. So if generic flash encryption is now supported by nvs out-of-the-box, why do the docs still ask for the whole mess of adding own keys for NVS encryption?
What is the downside of following your modification of " remove this line " to get it to work?
Hi. I have still not found any downsides. Would love to hear official statement from Espressif why this method would not be the default one; I think it should be :D.

kokonuts
Posts: 15
Joined: Tue Jul 11, 2017 6:39 pm

Re: Why doesn't NVS Encryption use the standard Flash Encryption mechanism?

Postby kokonuts » Sat Mar 11, 2023 10:36 am

EmilenL wrote:
Thu Mar 09, 2023 6:59 am
kokonuts wrote:
Sun Feb 26, 2023 1:17 pm
Hi, I was wondering the same thing and I landed here. So if generic flash encryption is now supported by nvs out-of-the-box, why do the docs still ask for the whole mess of adding own keys for NVS encryption?
What is the downside of following your modification of " remove this line " to get it to work?
Hi. I have still not found any downsides. Would love to hear official statement from Espressif why this method would not be the default one; I think it should be :D.
Yes, It complicate life with more keys and partitions just for NVS. I am not as brave as you, so I went with FAT :(

Who is online

Users browsing this forum: No registered users and 141 guests