wearleveling filesystem image uploading

perryc
Posts: 25
Joined: Fri Feb 17, 2017 1:29 am

wearleveling filesystem image uploading

Postby perryc » Wed May 17, 2017 4:14 am

I'm I correct to assume the new wear leveling spiffs would not work with mkspiffs created filesystem images? and if that is correct is anyone working on a new tool or updating mkspiffs?

Perry
Last edited by perryc on Sun May 21, 2017 11:08 am, edited 1 time in total.

ESP_igrr
Posts: 1656
Joined: Tue Dec 01, 2015 8:37 am

Re: wearleveling filesystem image uloading

Postby ESP_igrr » Wed May 17, 2017 4:27 am

It should be possible to prepare the partition image using the standard tools to generate FAT FS. We plan to have a guide for this, but haven't started on it yet.

perryc
Posts: 25
Joined: Fri Feb 17, 2017 1:29 am

Re: wearleveling filesystem image uloading

Postby perryc » Wed May 17, 2017 6:18 am

Thanks very much for the reply.
Build a 1M FAT image and flashed it to 0x100000 but esp_vfs_fat_spiflash_mount() still triggered a reformat so guess I'll need the experts to be my guide once again.

Perry

perryc
Posts: 25
Joined: Fri Feb 17, 2017 1:29 am

Re: wearleveling filesystem image uloading

Postby perryc » Sat May 20, 2017 2:09 am

Success! Using the partitions.csv shown below I found the fat filesytem created by esp_vfs_fat_spiflash_mount() at 0x111000 (Not 0x110000 as expected). Downloaded the fs with:

Code: Select all

$IDF_PATH/components/esptool_py/esptool/esptool.py --baud 115200 --port com3 read_flash 0x111000 0xF0000 spifat.img
could not figure out how to get msys32's mount work so had to use a real linux box to

Code: Select all

mount -o loop spifat.img mntdir
copied files into mounted fs, umounted and used esptool.py to write back to esp32 and we were good to go.

Code: Select all

# Note: if you change the phy_init or app partition offset, make sure to change the offset in Kconfig.projbuild,,,,
nvs,      data, nvs,     0x9000,  0x6000,
phy_init, data, phy,     0xf000,  0x1000,
factory,  app,  factory, 0x10000, 1M,
storage,  data, fat,0x110000,0xf0000,

ESP_Sprite
Posts: 3729
Joined: Thu Nov 26, 2015 4:08 am

Re: wearleveling filesystem image uloading

Postby ESP_Sprite » Sat May 20, 2017 10:18 am

Fyi, do note that this trick works only when the filesystem is relatively new. Once a bit older, the wear level system will have shuffled some sectors around. You would still be able to mount the flash just fine and you can probably, depending on the state of the wear levelling system, still see the files, but their contents may be corrupted.

User avatar
kolban
Posts: 1683
Joined: Mon Nov 16, 2015 4:43 pm
Location: Texas, USA

Re: wearleveling filesystem image uloading

Postby kolban » Sat May 20, 2017 10:35 pm

PerryC, ESP_Sprite, ESP_Igrr,
This is a very interesting thread. In PerryC's story, he seemed to have dumped an image from Flash that was formatted by esp_vfs_fat_spiflash_mount() and successfully mounted it as a Linux mount point, modified it and flashed it back. This is GREAT ... but I take the warnings seriously that because of wear leveling, things will become "corrupted". However, the core thought to me is "**will** become corrupted" ... does that mean that this is a recipe where I can create an initial image for seeding a file system? For example, if I want my ESP32 to be a web server, I could load up the web files into the file system?
Free book on ESP32 available here: https://leanpub.com/kolban-ESP32

perryc
Posts: 25
Joined: Fri Feb 17, 2017 1:29 am

Re: wearleveling filesystem image uloading

Postby perryc » Sat May 20, 2017 11:40 pm

Putting /html content on a fresh system was exactly what we wanted it for but I guess we better build a loader that uses fileio to be safe long term. Can the wear leveling counters be reset so a new fat filesystem can be loaded and can be trusted to work on anything but a fresh module?

Perry

User avatar
kolban
Posts: 1683
Joined: Mon Nov 16, 2015 4:43 pm
Location: Texas, USA

Re: wearleveling filesystem image uloading

Postby kolban » Sun May 21, 2017 1:23 am

Perry,
For testing, I'm tinkering with a TFTP server that can run in the ESP32. This will allow us to "upload" files to be saved in file FATFS. Its a poor mans story, but it will satisfy my immediate needs. If you or anyone else are interested, ping me.
Free book on ESP32 available here: https://leanpub.com/kolban-ESP32

ESP_igrr
Posts: 1656
Joined: Tue Dec 01, 2015 8:37 am

Re: wearleveling filesystem image uloading

Postby ESP_igrr » Sun May 21, 2017 2:40 am

For initial loading, the method will work. In fact, it is similar to the method we have considered while designing the wear levelling library. The idea was to create a file system using existing tools such as mkfs.vfat, mount it, load files into it, unmount, add the correct wear levelling metadata at the end of the partition, and then upload the resulting binary to the ESP32.

User avatar
kolban
Posts: 1683
Joined: Mon Nov 16, 2015 4:43 pm
Location: Texas, USA

Re: wearleveling filesystem image uloading

Postby kolban » Sun May 21, 2017 4:11 am

Unfortunately I couldn't reproduce Perry's tests. I created a partition at 0x30 0000 for 1MB and then used esptool to extract the data. I tried 0x30 0000 for 0x10 0000 and 0x31 0000 for 0x0f 0000 but with neither of the resulting files could I use any mount commands on Linux. Is Perry using the right commands? Are there any suggestions for the commands to use?
Free book on ESP32 available here: https://leanpub.com/kolban-ESP32

Who is online

Users browsing this forum: mitchk and 45 guests