Add Generic ESP32-S3 16MB board: virtual SD card on internal flash - #412
Open
jackjudge01-bit wants to merge 1 commit into
Conversation
GhostESP already solves 'capture/evil-portal/sweep need an SD-shaped mount point' for two specific boards - S3TWatch and AtomS3R - via an IS_S3TWATCH/IS_ATOMS3R Kconfig flag that carves a 'storage' partition out of internal flash and mounts it as FAT at /mnt. That mechanism isn't available for a plain, generic ESP32-S3 board with no SD slot and no display/persona hardware - this adds it as a new board identity, CONFIG_IS_GENERIC_ESP32S3_16MB, rather than editing the existing shared sdkconfig.default.esp32s3 (which is the build target for other 4MB-flash boards too - patching it in place for a 16MB-only partition would have silently broken them). On a 16MB chip, GhostESP's own default partitions.csv (nvs+app0+coredump) totals exactly 4MB, leaving 12MB completely unpartitioned. This claims a slice of it: - A static 4MB 'storage' partition at 0x400000 (partitions_generic_esp32s3_16mb.csv, configs/sdkconfig.generic_esp32s3_16mb: 16MB flash / 8MB octal PSRAM). - Dynamic sizing on top: sd_vstorage_manager.c computes free flash from the live partition table and caps the storage partition at 80% of it, rather than a hardcoded number. New CLI surface matching GhostESP's existing 'sd' command style: sd vstorage info|create|resize|delete. resize and create (when a partition already exists) both require -y/--confirm since either destroys existing partition contents. - The genuinely hard part: writing a new partition table to a running device at runtime, not build time - ESP-IDF reads the partition table once at boot, so any resize needs a reboot. The write path stages the new table to a scratch sector and verifies readback before committing to the live table sector. Known limitation, disclosed rather than hidden: stock ESP-IDF has no backup partition-table slot on this chip target, so a power loss mid-write to the live sector would still need esptool recovery to get the device booting again. Not fixed here. Also fixes an abort() the commit-to-live-table step hits on real hardware: the live partition table sits in esp_flash's protected region (bootloader, partition table, running app), and this build has CONFIG_SPI_FLASH_DANGEROUS_WRITE_ABORTS=y, so esp_flash_erase_region()/ esp_flash_write() abort() on it by default. Bracketed just that one write with esp_flash_set_dangerous_write_protection() - the same sanctioned internal API components/app_update/esp_ota_ops.c already uses for the same class of protected-region write. Verified on real hardware (the actual 16MB-flash/8MB-PSRAM ESP32-S3 board this targets): chipinfo and sd vstorage info confirm the new board identity, the static partition at the right offset/size, and a sane dynamic-sizing cap from live free-flash numbers. sd vstorage resize/create/delete all confirmed working (resize no longer aborts). A manual sd write/sd cat round trip through the mounted virtual storage confirmed working. Also build-verified against this exact target after syncing to the current tip of Development-deki. - @jackjudge01-bit Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
✅ Deploy Preview for ghostespdocs canceled.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
GhostESP already solves "capture/evil-portal/sweep need an SD-shaped mount point" for two specific boards — S3TWatch and AtomS3R — via an
IS_S3TWATCH/IS_ATOMS3RKconfig flag that carves astoragepartition out of internal flash and mounts it as FAT at/mnt. That mechanism isn't available for a plain, generic ESP32-S3 board with no SD slot and no display/persona hardware. This adds it as a new board identity,CONFIG_IS_GENERIC_ESP32S3_16MB, rather than editing the existing sharedsdkconfig.default.esp32s3(the build target for other 4MB-flash boards too — patching it in place for a 16MB-only partition would silently break those).On a 16MB chip, GhostESP's own default
partitions.csv(nvs+app0+coredump) totals exactly 4MB, leaving 12MB completely unpartitioned. This claims a slice of it.How
storagepartition at0x400000(partitions_generic_esp32s3_16mb.csv,configs/sdkconfig.generic_esp32s3_16mb: 16MB flash / 8MB octal PSRAM).main/managers/sd_vstorage_manager.c): computes free flash from the live partition table, caps the storage partition at 80% of it rather than a hardcoded number. New CLI matching GhostESP's existingsdcommand style:sd vstorage info|create|resize|delete.resize/create(when a partition already exists) require-y/--confirmsince either destroys existing partition contents, and both need a reboot to take effect — ESP-IDF reads the partition table once at boot, no hot-reload.Known limitation, disclosed rather than left for someone to discover: stock ESP-IDF has no backup partition-table slot on this chip target, so a power loss mid-write to the live sector during a resize would still need
esptoolrecovery to get the device booting again. Not fixed here — flagging it clearly for review since this touches runtime partition-table writes.Also includes a fix for an
abort()the live-table commit step hits on real hardware: that write lands inesp_flash's protected region (bootloader, partition table, running app), and this build hasCONFIG_SPI_FLASH_DANGEROUS_WRITE_ABORTS=y, soesp_flash_erase_region()/esp_flash_write()abort on it by default. Bracketed just that one write withesp_flash_set_dangerous_write_protection()— the same APIcomponents/app_update/esp_ota_ops.calready uses for the same class of protected-region write.Verification
On the real 16MB-flash/8MB-PSRAM ESP32-S3 board this targets:
chipinfo/sd vstorage infoconfirm the new board identity, the static partition at the right offset/size, and a sane dynamic-sizing cap from live free-flash numbers.sd vstorage resize/create/deleteall confirmed working (resize no longer aborts).sd write/sd catround trip through the mounted virtual storage confirmed working.Development-deki.Related
#411 (this same session's work) fixes a device panic that shows up specifically once a capture command runs while this virtual SD (or a real SD card) is mounted — unrelated to this PR's own changes, but you'll want it alongside this if reviewing/testing capture on this board.
Notes for reviewers
New to this project and to git/PR workflow generally — happy to adjust format, split further, or provide more detail on anything, including the partition-table write path specifically since I know that's the riskiest part of this to review.