I want to compile 1 project for 2 different partition configurations

simonjo
Posts: 21
Joined: Wed Sep 30, 2020 9:16 am

I want to compile 1 project for 2 different partition configurations

Postby simonjo » Wed May 12, 2021 1:29 pm

Hi,

I have a project running fine on ESP32 WROVER with following partitions:

# ESP-IDF partition table
# Name, Type, Subtype, Offset, Size, Flags
nvs, data, nvs, 0x9000, 16K,
otadata, data, ota, 0xd000, 8K,
phy_init, data, phy, 0xf000, 4K,
factory, app, factory, 0x10000, 1M,
ota_0, app, ota_0, 0x110000, 1M,
ota_1, app, ota_1, 0x210000, 1M,
private, data, fat, , 960K,

The project receives a zip file via HTTP OTA and it extracts 1 XML metadata file from it and stores it in the data partition for later use. But now I have an instance where the metadata file is larger than 960K... so the alternative is below partitioning, so far so good:

# ESP-IDF partition table
# Name, Type, Subtype, Offset, Size, Flags
nvs, data, nvs, 0x9000, 16K,
otadata, data, ota, 0xd000, 8K,
phy_init, data, phy, 0xf000, 4K,
ota_0, app, ota_0, 0x10000, 1M,
ota_1, app, ota_1, 0x110000, 1M,
private, data, fat, , 1984K,

The problem lies in the fact that a new version with the second partition table cannot be installed in the field via OTA upgrade on a module with the first partition table. A second problem is that not all users have the skills or infrastructure to flash the module via wire.

So my alternative is to make 2 projects (project-4p and project-3p) with the different partitioning scheme and deliver 2 binaries for OTA upgrade, as such the user can select the correct one for his/her module.

I can do this by copying the sources of let's say project-4p to project-3p and then compile, but this way I constantly need to monitor files to be in sync as changes/fixes are implemented.

Can I setup my second project (i.e. project-3p) to use it's own partitions.csv but use the source files in the first project (project-4p) ? This way it will always be in sync and the only diff in the bin file is the partitioning.


Thanks for any help on this
Jo

WiFive
Posts: 3529
Joined: Tue Dec 01, 2015 7:35 am

Re: I want to compile 1 project for 2 different partition configurations

Postby WiFive » Thu May 13, 2021 6:33 am

I don't think that is necessary because the partition scheme is not compiled into the app, you just have to have logic in the app to deal with different partition sizes.

simonjo
Posts: 21
Joined: Wed Sep 30, 2020 9:16 am

Re: I want to compile 1 project for 2 different partition configurations

Postby simonjo » Thu May 13, 2021 10:10 am

The application itself is not the issue, it will be 100% equal for both the 4 partition and 3 partition version.

But by removing the factory partition in the '3p version', the OTA partitions will start at another offset and the data partition will be 1Mb larger.

You cannot fix that via OTA.

ESP_Angus
Posts: 2344
Joined: Sun May 08, 2016 4:11 am

Re: I want to compile 1 project for 2 different partition configurations

Postby ESP_Angus » Fri May 14, 2021 12:18 am

Hi Simon,

If I understand correctly, a possible option would be to keep two partition csv files in the same project directory. You can switch between them by changing the file name in the sdkconfig file.

However, because (as WiFive has pointed out) the partition data isn't compiled into the app but only used to make the partition .bin file for flashing, you can build the corresponding "version A" partition .bin file once, copy the bin file somewhere else in your app directory, and flash it manually with esptool.py for testing. This way you can keep everything else the same:

- If testing project B then build and flash it normally
- If testing project A then build and flash it normally, then flash the saved "project A" partition .bin file at the partition table offset.

If you then need to know in the app itself which partition table is in use on the device, but you don't want to compile different binaries, then this should be possible by using the partition APIs to inspect factors such as the size of the data partition (this is read from the data partition in the flash at runtime, so you can look it up on demand).

This way the app itself can decide (for example) which metadata file to download.

Does that solve the problem?

Other alternatives I can see involves either having two configs (to build slightly different versions of the same project), or two separate projects that share some common components in an EXTRA_COMPONENTS_DIR directory. But if the differences are minor then using the same app binary and checking the partition size at runtime should hopefully work.

(The root cause of not being able to update the partition table via OTA is unfortunately pretty intractable, as you've identified.)

simonjo
Posts: 21
Joined: Wed Sep 30, 2020 9:16 am

Re: I want to compile 1 project for 2 different partition configurations

Postby simonjo » Fri May 14, 2021 8:30 pm

Hi Angus,

Thanks for your answer, this made me rethink the problem.

This is such a case where the answer is right in front of you but you overlook it and you need some interaction with others to see it.

Conclusion & fix:
- I keep the first project with all source files and a partitions.csv with 4 partitions
- the second project has a partitions.csv with 3 partitions AND CMakeLists.txt just specifies all files from project 1 (by specifying ../../project1/main/abc.cpp, ...)

And that's it, now I compile each project in turn, the first delivers project-4p.bin and the second delivers project-3p.bin with the same functionality

Cheers, Jo

Who is online

Users browsing this forum: Majestic-12 [Bot] and 105 guests