Skip to content

Add Generic ESP32-S3 16MB board: virtual SD card on internal flash - #412

Open
jackjudge01-bit wants to merge 1 commit into
GhostESP-Revival:Development-dekifrom
jackjudge01-bit:feature/generic-esp32s3-16mb-virtual-sd
Open

jackjudge01-bit wants to merge 1 commit into
GhostESP-Revival:Development-dekifrom
jackjudge01-bit:feature/generic-esp32s3-16mb-virtual-sd

Conversation

@jackjudge01-bit

Copy link
Copy Markdown

What

GhostESP already solves "capture/evil-portal/sweep need an SD-shaped mount point" for two specific boards — S3TWatch and AtomS3R — via an IS_S3TWATCH/IS_ATOMS3R Kconfig flag that carves a storage partition out of internal flash and mounts it as FAT at /mnt. That mechanism isn't available for a plain, generic ESP32-S3 board with no SD slot and no display/persona hardware. This adds it as a new board identity, CONFIG_IS_GENERIC_ESP32S3_16MB, rather than editing the existing shared sdkconfig.default.esp32s3 (the build target for other 4MB-flash boards too — patching it in place for a 16MB-only partition would silently break those).

On a 16MB chip, GhostESP's own default partitions.csv (nvs+app0+coredump) totals exactly 4MB, leaving 12MB completely unpartitioned. This claims a slice of it.

How

  • A static 4MB storage partition at 0x400000 (partitions_generic_esp32s3_16mb.csv, configs/sdkconfig.generic_esp32s3_16mb: 16MB flash / 8MB octal PSRAM).
  • Dynamic sizing on top (main/managers/sd_vstorage_manager.c): computes free flash from the live partition table, caps the storage partition at 80% of it rather than a hardcoded number. New CLI matching GhostESP's existing sd command style: sd vstorage info|create|resize|delete. resize/create (when a partition already exists) require -y/--confirm since either destroys existing partition contents, and both need a reboot to take effect — ESP-IDF reads the partition table once at boot, no hot-reload.
  • The write path stages the new table to a scratch sector and verifies readback before committing to the live table sector.

Known limitation, disclosed rather than left for someone to discover: stock ESP-IDF has no backup partition-table slot on this chip target, so a power loss mid-write to the live sector during a resize would still need esptool recovery to get the device booting again. Not fixed here — flagging it clearly for review since this touches runtime partition-table writes.

Also includes a fix for an abort() the live-table commit step hits on real hardware: that write lands in esp_flash's protected region (bootloader, partition table, running app), and this build has CONFIG_SPI_FLASH_DANGEROUS_WRITE_ABORTS=y, so esp_flash_erase_region()/esp_flash_write() abort on it by default. Bracketed just that one write with esp_flash_set_dangerous_write_protection() — the same API components/app_update/esp_ota_ops.c already uses for the same class of protected-region write.

Verification

On the real 16MB-flash/8MB-PSRAM ESP32-S3 board this targets:

  • chipinfo / sd vstorage info confirm the new board identity, the static partition at the right offset/size, and a sane dynamic-sizing cap from live free-flash numbers.
  • sd vstorage resize/create/delete all confirmed working (resize no longer aborts).
  • A manual sd write/sd cat round trip through the mounted virtual storage confirmed working.
  • Build-verified against this target after syncing to the current tip of Development-deki.

Related

#411 (this same session's work) fixes a device panic that shows up specifically once a capture command runs while this virtual SD (or a real SD card) is mounted — unrelated to this PR's own changes, but you'll want it alongside this if reviewing/testing capture on this board.

Notes for reviewers

New to this project and to git/PR workflow generally — happy to adjust format, split further, or provide more detail on anything, including the partition-table write path specifically since I know that's the riskiest part of this to review.

GhostESP already solves 'capture/evil-portal/sweep need an SD-shaped
mount point' for two specific boards - S3TWatch and AtomS3R - via an
IS_S3TWATCH/IS_ATOMS3R Kconfig flag that carves a 'storage' partition out
of internal flash and mounts it as FAT at /mnt. That mechanism isn't
available for a plain, generic ESP32-S3 board with no SD slot and no
display/persona hardware - this adds it as a new board identity,
CONFIG_IS_GENERIC_ESP32S3_16MB, rather than editing the existing shared
sdkconfig.default.esp32s3 (which is the build target for other 4MB-flash
boards too - patching it in place for a 16MB-only partition would have
silently broken them).

On a 16MB chip, GhostESP's own default partitions.csv (nvs+app0+coredump)
totals exactly 4MB, leaving 12MB completely unpartitioned. This claims a
slice of it:

- A static 4MB 'storage' partition at 0x400000 (partitions_generic_esp32s3_16mb.csv,
  configs/sdkconfig.generic_esp32s3_16mb: 16MB flash / 8MB octal PSRAM).
- Dynamic sizing on top: sd_vstorage_manager.c computes free flash from
  the live partition table and caps the storage partition at 80% of it,
  rather than a hardcoded number. New CLI surface matching GhostESP's
  existing 'sd' command style: sd vstorage info|create|resize|delete.
  resize and create (when a partition already exists) both require
  -y/--confirm since either destroys existing partition contents.
- The genuinely hard part: writing a new partition table to a running
  device at runtime, not build time - ESP-IDF reads the partition table
  once at boot, so any resize needs a reboot. The write path stages the
  new table to a scratch sector and verifies readback before committing
  to the live table sector.

Known limitation, disclosed rather than hidden: stock ESP-IDF has no
backup partition-table slot on this chip target, so a power loss mid-write
to the live sector would still need esptool recovery to get the device
booting again. Not fixed here.

Also fixes an abort() the commit-to-live-table step hits on real hardware:
the live partition table sits in esp_flash's protected region (bootloader,
partition table, running app), and this build has
CONFIG_SPI_FLASH_DANGEROUS_WRITE_ABORTS=y, so esp_flash_erase_region()/
esp_flash_write() abort() on it by default. Bracketed just that one write
with esp_flash_set_dangerous_write_protection() - the same sanctioned
internal API components/app_update/esp_ota_ops.c already uses for the
same class of protected-region write.

Verified on real hardware (the actual 16MB-flash/8MB-PSRAM ESP32-S3 board
this targets): chipinfo and sd vstorage info confirm the new board
identity, the static partition at the right offset/size, and a sane
dynamic-sizing cap from live free-flash numbers. sd vstorage resize/create/delete
all confirmed working (resize no longer aborts). A manual sd write/sd cat
round trip through the mounted virtual storage confirmed working. Also
build-verified against this exact target after syncing to the current
tip of Development-deki.

- @jackjudge01-bit

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@netlify

netlify Bot commented Sep 22, 2026

Copy link
Copy Markdown

Deploy Preview for ghostespdocs canceled.

Name Link
🔨 Latest commit 7bfb59b
🔍 Latest deploy log https://app.netlify.com/projects/ghostespdocs/deploys/6ab2db91203c2c00081ddf91

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant