English | 简体中文
A local-first, self-hosted temporary inbox for inbound email
C++ SMTP ingress, optional public inboxes, an admin console, and an HTTP API
Mail, attachments, metadata, and audit records are stored on local disk and in SQLite
Quick Start · Demo · Project Scope · Roadmap · Implemented Features · Configuration · Usage · Contributing · Security
Rapid Inbox is built for open-source maintainers, developers, and test teams that need a self-hosted email testing environment for receiving verification codes, CI/E2E email capture, internal-tool integration, and lightweight self-hosting.
Its core goals are reliable ingestion, clear inspection, controlled access, and straightforward recovery. Message bodies, attachments, and indexes are stored on local disk and in SQLite by default, without depending on a third-party email SaaS or cloud database. Receiving mail from the public Internet still requires your own domain, DNS/MX records, and a reachable SMTP ingress.
The project is currently at an early Alpha stage. Interfaces and data structures may continue to change.
In short: Rapid Inbox is an alpha-stage, inbound-only, self-hosted SMTP inbox for email testing, verification-code workflows, and internal integrations. It uses a single-host SQLite/local-filesystem architecture and is not an outbound mail server, hosted mailbox service, or multi-node mail platform.
| Dimension | Current boundary |
|---|---|
| Intended use | Self-hosted inbound test mail, verification-code extraction, CI/E2E email capture, and internal-tool integration |
| Deployment model | Single-host local disk and SQLite. Docker Compose is the primary deployment path; the native systemd installer is secondary. Both run exactly one C++ ingestd and one Python HTTP process for one data directory |
| HTTP paths | The UI and API currently support deployment only at the site root /; reverse-proxy subpaths and ASGI root_path are not supported |
| Explicit non-goals | Outbound SMTP, IMAP/POP3, a full public MTA, hosted mailbox SaaS, advanced anti-spam or malicious-attachment sandboxing, and enterprise multi-node HA |
| Operator responsibilities | The project does not host or automatically configure DNS/MX, SMTP TLS termination, firewalls, backups, or global rate limiting across instances |
| Public-access boundary | Public Web/API access is disabled by default for new domains and the catch-all policy; anonymous Web browsing requires an explicit opt-in, and the public API still requires an API Key with public.read |
| Data retention | "Temporary mailbox" describes the use case, not default automatic destruction; mail is not deleted automatically when retention_days is unset or 0 |
| Authorized use | Operators must process only domains and mail they control or are explicitly authorized to handle; the project must not be used to intercept third-party mail, phish, collect credentials, send spam, or evade third-party rules |
| Maturity | 0.x Alpha; there is no SLA or published throughput guarantee, and the database, configuration, and API may still receive incompatible changes |
See Roadmap Issue #5 for the current plan and capabilities that are explicitly out of scope for now.
The demo uses sanitized test data in an isolated environment and contains no real mailboxes, messages, or credentials. It was recorded with Python's embedded SMTP server. Current deployments persist committed public-mailbox delivery changes in a SQLite live-event outbox and tail them from the Python HTTP process, so the default C++ ingestd and standalone Python SMTP modes can update an open public inbox over WebSocket without a manual refresh. The administration console also uses an authenticated WebSocket: Python SMTP can publish process-local connection/command telemetry, while C++ and separate ingress surface committed deliveries as
delivery_committedevents through the SQLite outbox. Cursor-based reconnects can fall back to committed history.
| Category | Capability |
|---|---|
| Email ingestion | C++ rapid-inbox-ingestd provides durable ACKs, byte-level backpressure, group commit, multi-worker MIME parsing, and poison-task isolation; Python SMTP remains available as a development mode |
| Domain modes | Receive only configured domains, or enable managed_plus_catchall for any-domain SMTP deliveries that reach this service; longest-suffix rules take precedence and domain rules are hot-reloaded |
| Inboxes | Domains are private by default; the per-mailbox public flag defaults to enabled but takes effect only when the domain-level public switch is enabled, and it can still be disabled per mailbox; supports lists, details, raw EML, sandboxed HTML previews, and attachment downloads |
| Live updates | Python and C++ ingress persist committed public-mailbox delivery changes in a SQLite live-event outbox; each HTTP runtime tails the outbox and pushes matching mailbox updates over WebSocket, so open public inboxes do not require manual refreshes. The authenticated administration WebSocket uses cursor resume and gap reporting; committed C++ deliveries appear as delivery_committed, while connection/command telemetry remains process-local |
| Verification-code detection | A scored extraction algorithm with Chinese, English, Japanese, Korean, and Spanish context plus alphanumeric and separated-code patterns |
| Access control | viewer / operator / superadmin RBAC; API Keys are constrained by kind, scope, domain grant mode, mailbox glob, IP, rate limit, and expiry |
| HTTP API | /api/v2 is recommended: Bearer-only, strict models, Problem Details, and stable cursors; public and admin /api/v1 endpoints remain available |
| Observability | JSON/text structured logs, safe Request IDs, Prometheus metrics, live/ready probes, and a cached operations dashboard |
| Persistence and recovery | SQLite WAL stores indexes and the compact live-event outbox that bridges committed delivery notifications from ingress to HTTP; raw / text / html / attachments / manifests live on disk; manifests can rebuild metadata that was not committed |
| Cleanup | Delivery-retention cleanup in batches; file GC is registered inside the transaction and performed outside it with exponential-backoff retries; sessions, empty mailboxes, metrics, and audit records are cleaned independently |
| Maintenance tools | A cross-process .maintenance.lock coordinates clearing mail, pauses new ingestion, removes files, and compacts SQLite |
Docker Compose · C++20 · Python 3.10+ · FastAPI · aiosmtpd · Jinja2 · SQLite · Uvicorn · WebSocket
Docker Compose is the recommended deployment method. From a reviewed checkout, with Docker Engine and Compose v2 installed, run:
./docker-deploy.shThis one command builds the image, creates a mode-0600 private configuration with random bootstrap,
cursor-signing, and metrics secrets, starts the Python control plane, waits for schema migration and
/health/ready, then starts the C++ SMTP ingress. The image runs as a non-root user. Both processes
are supervised in one container and one PID namespace because the cross-process maintenance protocol
records and verifies operating-system PIDs. A SQLite live-event outbox carries committed public-mailbox
delivery notifications across this process boundary. Public and administration WebSocket cursor
generations remain local to one HTTP process, so keep one HTTP worker per instance (as the supplied
runners do); a multi-worker or multi-replica HTTP tier requires sticky routing until cursor generations
are shared.
Default host bindings:
HTTP: 127.0.0.1:8000
SMTP: 0.0.0.0:25
The first successful deployment prints the initial administrator password once. Open:
http://127.0.0.1:8000/admin/login
After the first bootstrap administrator login, the admin console forces a change of the initial password. The generated value remains in
.rapid-inbox-docker/rapid-inbox.env; protect this file as a secret and replace the password in the administration UI immediately.
Mail, SQLite, manifests, and attachments are persisted in the Compose named volume
rapid-inbox_rapid-inbox-data by default. The deployment script never removes this volume.
Use the wrapper for lifecycle operations:
./docker-deploy.sh status
./docker-deploy.sh logs
./docker-deploy.sh credentials
./docker-deploy.sh update # rebuild the current checkout with fresh base images
./docker-deploy.sh down # retain configuration and the named data volumeEdit .rapid-inbox-docker/rapid-inbox.env to change published addresses, ports, or application
settings, then run ./docker-deploy.sh again. For reproducible updates, check out a reviewed tag or
commit first; update does not fetch source code.
Caution
Do not run docker compose down -v unless permanent deletion of the named volume is intended.
Stop the deployment before making a consistent backup, and back up both the named volume and the
private configuration. Do not place the SQLite volume on NFS or another filesystem without reliable
local POSIX locking, and do not scale the Compose service beyond one replica.
Warning
Before the first Internet-facing deployment, confirm that the bootstrap password has been
changed and keep the generated Metrics Token private. Compatibility
ADMIN_TOKEN / PUBLIC_API_KEY credentials are disabled by default; new integrations should use
API Keys issued from the admin console. HTTP is published only on host loopback by default. Put the
admin plane behind a trusted HTTPS reverse proxy before changing that binding. The deployment does
not configure DNS/MX, TLS termination, firewall rules, backups, or upstream anti-abuse controls.
See the Docker deployment guide for backup, restore, rollback, port, and troubleshooting procedures.
For Debian 12+ or Ubuntu 24.04+ hosts that require native services instead of Docker:
sudo bash deploy/system/install.sh installThe installer creates a dedicated rapid-inbox account, installs dependencies, stages a versioned
release under /opt/rapid-inbox, generates /etc/rapid-inbox/rapid-inbox.env, keeps data under
/var/lib/rapid-inbox, initializes and backs up SQLite at the write boundary, installs hardened
HTTP/SMTP units, and verifies HTTP plus SMTP protocol readiness.
sudo bash deploy/system/install.sh status
sudo bash deploy/system/install.sh update
sudo bash deploy/system/install.sh uninstall # preserves configuration and dataSee the native systemd deployment guide for the exact support boundary, managed paths, update rollback, and uninstall behavior.
quickstart.sh is retained for local evaluation and development. It is a foreground launcher, not
a long-running production process manager:
bash quickstart.shIt creates .venv and .env, initializes SQLite, and runs Python HTTP with C++ ingestd. By default,
it downloads the mutable latest ingestd release after SHA-256 verification; pin a reviewed release
with --ingestd-version, or build the current checkout with --build-local. Run
bash quickstart.sh --help for all options, including the Python SMTP compatibility mode.
C++ SMTP ingestd + Python HTTP (manual development topology)
# 1. Build the C++ SMTP ingress
cmake -S cpp/ingestd -B cpp/ingestd/build
cmake --build cpp/ingestd/build
# 2. Start Python HTTP without embedded SMTP
.venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8000
# 3. Start the C++ SMTP ingress
SMTP_HOST=0.0.0.0 SMTP_PORT=25 cpp/ingestd/build/rapid-inbox-ingestd --base-dir .INGEST_DURABLE_ACK=true by default. Before SMTP returns 250 queued, ingestd atomically writes
the raw EML and pending manifest. SQLite metadata can be committed later in a batch; if the process
exits in the meantime, the Python recovery worker rebuilds records from the manifest. Final parsed
manifests and the recovery worker share a 16 MiB per-file budget. If parsed metadata would exceed
that budget, ingestd retains a bounded pending manifest so recovery can parse the raw message again.
INGEST_STORAGE_FSYNC=false guarantees only process-crash recovery. Set it to true to cover host
power loss, at the cost of higher disk-sync latency. Disabling durable ACK returns to acknowledging
as soon as mail enters the in-memory queue; an abnormal process exit can then lose mail that already
received a 250 response.
Each SQLite batch commit rematches all recipients inside the transaction. A concurrent rename/delete
or fallback delivery to another tenant rolls back with policy conflict. If durable ACK was already
returned, the raw file and manifest are retained in an explicit quarantine forensic path, and the
recovery worker uses a persistent tombstone to prevent stale manifests from resurrecting renamed or
deleted domains.
HTTP + embedded Python SMTP in one process (development/compatibility mode)
.venv/bin/rapid-inbox-httpStandalone Python SMTP listener (compatibility mode)
.venv/bin/rapid-inbox-smtpDevelopment mode (module entry point)
.venv/bin/uvicorn app.main:app --reloadRunning uvicorn app.main:app directly does not enable embedded SMTP. To receive SMTP mail,
run rapid-inbox-ingestd in another process. For development, use rapid-inbox-http or
rapid-inbox-smtp.
The repository includes the GitHub Actions workflow .github/workflows/release-ingestd.yml:
- Regular pushes and pull requests run Python tests, validate both deployment paths, execute a real one-command Docker smoke deployment, and build/test C++ ingestd.
- Pushing a
v*tag builds a Linux x86_64 release archive and publishes these files to a GitHub Release:rapid-inbox-ingestd-linux-x86_64.tar.gzrapid-inbox-ingestd-linux-x86_64.tar.gz.sha256
Example release commands:
git tag "$NEW_RELEASE_TAG"
git push origin "$NEW_RELEASE_TAG"The publisher should set NEW_RELEASE_TAG explicitly according to the actual versioning policy.
Published versions are listed under
GitHub Releases.
The Docker path builds a local image from the reviewed checkout; this workflow does not currently
publish a container image. After a Release is published, local-development bash quickstart.sh
downloads prebuilt ingestd from the mutable latest
release by default and prints a drift warning. Reproducible deployments should explicitly pass a
reviewed tag; use --build-local when a local build is required.
The application resolves variables in this order:
Process environment > .env in the current working directory > defaults in app/config.py
Docker passes .rapid-inbox-docker/rapid-inbox.env as process environment; systemd uses
/etc/rapid-inbox/rapid-inbox.env. The repository .env.example is the local-source template.
Complete environment variable table
Python HTTP, compatibility SMTP, and shared configuration:
| Variable | .env.example |
Description |
|---|---|---|
STORAGE_ROOT |
./storage |
Root directory for message files, attachments, manifests, and temporary files |
DATABASE_PATH |
./storage/app.db |
SQLite database path; Python and ingestd must point to the same file |
BOOTSTRAP_ADMIN_USERNAME |
admin |
Administrator username created automatically on the first start |
BOOTSTRAP_ADMIN_PASSWORD |
change-me-now (randomized by deployment scripts and quickstart) |
The code fallback for manual startup is also change-me-now and must not be used with an Internet-facing bind |
SESSION_COOKIE_NAME |
rapid_inbox_session |
Name of the HttpOnly administrator session cookie |
HOST / PORT |
127.0.0.1 / 8000 |
HTTP listen address and port; a non-loopback bind must be placed behind a trusted HTTPS reverse proxy |
HTTP_MAX_REQUEST_BODY_BYTES |
1048576 |
ASGI request-body limit for both Content-Length and streamed/chunked bodies; configurable up to 64 MiB |
HTTP_REQUEST_BODY_TIMEOUT_SECONDS |
15 |
Total time allowed to receive one complete HTTP request body, mitigating slow chunked uploads |
HTTP_BODY_MEMORY_BUDGET_BYTES |
268435456 |
Shared per-process byte budget for all buffered HTTP request bodies; must be at least the per-request limit |
HTTP_CONCURRENCY_LIMIT |
1000 |
Per-process admission limit across HTTP and WebSocket traffic; supported launchers also pass it to Uvicorn as --limit-concurrency, and application middleware enforces it |
HTTP_LIVE_CONNECTION_LIMIT |
256 |
Shared per-HTTP-process limit for administration and public-mailbox WebSockets; excess WebSockets close with 1013. The deprecated v1 SSE compatibility stream shares the same limit and receives 503 when full |
DATABASE_WRITE_QUEUE_CAPACITY / DATABASE_WRITE_MAX_WAITERS |
256 / 1024 |
Dual limits for requests accepted by the single SQLite write actor and requests waiting for it; excess load fails fast with 503 |
DATABASE_READ_POOL_SIZE / DATABASE_READ_QUEUE_CAPACITY / DATABASE_READ_MAX_WAITERS / DATABASE_READ_TIMEOUT_SECONDS |
1 / 256 / 1024 / 5 |
API v2 dedicated read-only actors, admitted-request and waiting-request limits, and end-to-end read timeout. A single actor is the conservative default; benchmark the actual workload before increasing connections. Maintenance first drains requests, then the owner thread closes connections. All values are calculated independently per HTTP process |
SMTP_HOST / SMTP_PORT |
0.0.0.0 / 25 |
SMTP listen address and port |
MAX_MESSAGE_SIZE_BYTES |
52428800 |
Maximum size of one message, shared by Python and C++ |
MAX_RECIPIENTS_PER_MESSAGE |
20 |
Maximum number of canonical recipients per message |
SMTP_IDLE_TIMEOUT_SECONDS |
30 |
Idle timeout for an SMTP session |
SMTP_MAX_CONCURRENT_CONNECTIONS |
1024 |
Python SMTP concurrent-connection limit; 0 is not allowed for non-loopback listeners |
SMTP_CONNECTION_RATE_LIMIT_COUNT |
60000 |
Python/C++ SMTP connections allowed per IP. Python uses bounded expiry/LRU state; C++ uses a fixed-capacity map, periodically removes expired entries, and evicts one entry when full. Both are per-process and provide no global rate limit across instances |
SMTP_CONNECTION_RATE_LIMIT_WINDOW_SECONDS |
60 |
Sliding window for Python/C++ SMTP per-IP connection limits |
SMTP_CLOSE_AFTER_DATA |
true |
Whether Python SMTP closes the connection after completing one DATA command |
PARSE_WORKER_COUNT |
4 |
Number of Python recovery/compatibility parsing workers |
PARSE_QUEUE_MAX_MESSAGES |
10000 |
Budget for queued + active messages in the Python MIME queue |
PARSE_QUEUE_MAX_BYTES |
536870912 |
Raw-message byte budget for queued + active work in the Python MIME queue; must be at least MAX_MESSAGE_SIZE_BYTES |
MESSAGE_PREVIEW_BODY_BYTES |
131072 |
Maximum UTF-8 source bytes read independently for text and HTML in public/admin details; responses include original sizes and truncation flags; configurable up to 16 MiB |
MESSAGE_PREVIEW_HEADERS_BYTES |
65536 |
Maximum mail-header JSON size that details may deserialize; over-budget headers return an empty list with a truncation flag; configurable up to 1 MiB |
MESSAGE_PREVIEW_INLINE_ITEM_BYTES |
65536 |
Source-byte budget for one inline image in an HTML CID preview; an over-budget item retains its original CID and remains available as an attachment download |
MESSAGE_PREVIEW_INLINE_TOTAL_BYTES |
262144 |
Aggregate source-byte budget for all inline images in one HTML preview; the per-item budget must not exceed this total |
FSYNC_STORAGE_WRITES |
false |
Whether Python performs fsync for file and directory writes |
INGRESS_MODE |
managed_only |
managed_only or managed_plus_catchall |
CATCH_ALL_PUBLIC_WEB_ENABLED |
false |
Whether catch-all policy permits public Web access |
CATCH_ALL_PUBLIC_API_ENABLED |
false |
Whether catch-all policy permits public API access |
CATCH_ALL_RETENTION_DAYS |
0 |
Retention period for catch-all deliveries; 0 means no automatic expiry |
RETENTION_CLEANUP_INTERVAL_SECONDS |
30 |
Background cleanup scheduling interval |
SMTP_SESSION_RETENTION_SECONDS |
86400 |
Retention period for completed SMTP sessions |
EMPTY_MAILBOX_RETENTION_SECONDS |
86400 |
Retention period for empty mailboxes with no deliveries |
METRIC_RETENTION_SECONDS |
604800 |
Retention period for mail metric buckets |
AUDIT_RETENTION_DAYS |
90 |
Audit-log retention period |
CLEANUP_BATCH_SIZE / FILE_GC_BATCH_SIZE |
1000 / 500 |
Per-run limits for database cleanup and file GC |
MAINTENANCE_RUN_RETENTION_DAYS |
30 |
Retention period for completed/failed maintenance-run records |
QUARANTINE_RETENTION_DAYS |
30 |
Retention period for quarantine forensic files |
ORPHAN_ARTIFACT_GRACE_SECONDS |
86400 |
Minimum file age before scanning unreferenced raw/text/HTML/attachment artifacts, reducing races with in-flight work |
ARTIFACT_SWEEP_BATCH_SIZE |
500 |
Maximum quarantine and orphan files each scan inspects per cleanup run; a pass resumes across cleanup runs |
DISK_WARNING_THRESHOLD_PERCENT |
85 |
Dashboard disk-usage warning threshold |
LOG_LEVEL / LOG_FORMAT |
INFO / json |
Log level and json / text format |
REQUEST_LOG_ENABLED |
true |
Whether to record structured HTTP access logs without query strings |
METRICS_ENABLED / METRICS_TOKEN |
true / empty |
Prometheus endpoint toggle and token; when metrics are enabled on a non-loopback bind, a token is required or startup is refused |
API_CURSOR_SECRET |
empty (randomized by deployment scripts and quickstart) | HMAC secret for API v2 cursors; manual Internet-facing deployments must configure at least 32 characters |
READINESS_MIN_FREE_DISK_BYTES |
67108864 |
Minimum free disk space required for readiness |
ADMIN_TOKEN / PUBLIC_API_KEY |
disabled | v1 compatibility tokens; enabled only when explicitly configured with non-default random values |
Because both live WebSockets are server-only notification streams, the supported Docker, systemd, quickstart, and Python entrypoints cap client-to-server messages at 16 KiB and queue at most one unhandled message. Reverse proxies should apply compatible or tighter limits.
Preview budgets are per-request limits. CID images grow by roughly 4/3 when converted to data URLs;
if body and inline budgets are increased together, worst-case concurrent memory grows roughly in
proportion to HTTP_CONCURRENCY_LIMIT. Production deployments should set these values together
according to available memory. Full content remains available through streamed raw-message and
attachment downloads.
Hot-path configuration dedicated to C++ rapid-inbox-ingestd:
| Variable | Default | Description |
|---|---|---|
SMTP_MAX_CONNECTIONS |
1024 |
Maximum simultaneous C++ SMTP connections |
SMTP_MAX_LINE_LENGTH |
1000 |
Maximum SMTP command/data line length |
SMTP_LISTEN_BACKLOG |
1024 |
C++ SMTP kernel listen backlog; SMTP_HOST=:: can listen on IPv6 |
SMTP_CONNECTION_RATE_LIMIT_COUNT / SMTP_CONNECTION_RATE_LIMIT_WINDOW_SECONDS |
60000 / 60 |
Per-IP connection sliding window shared by C++ and Python; lower it according to edge abuse risk |
INGEST_QUEUE_MAX_MESSAGES |
10000 |
Total budget for reserved, queued, and active messages |
INGEST_QUEUE_MAX_BYTES |
536870912 |
Byte budget; must be at least MAX_MESSAGE_SIZE_BYTES |
INGEST_RESERVATION_CHUNK_BYTES |
65536 |
Byte reservation growth chunk during DATA, avoiding reservation of the full message limit per connection |
INGEST_BATCH_MAX_MESSAGES |
250 |
Maximum messages in one group commit |
INGEST_FLUSH_INTERVAL_MS |
5 |
Maximum time to wait for messages to join the same batch |
INGEST_SQLITE_BUSY_TIMEOUT_MS |
5000 |
SQLite lock wait time |
INGEST_WORKER_COUNT |
4 |
Number of parallel MIME/file-processing workers |
INGEST_MAX_RETRIES |
3 |
Bounded retries after one message fails, followed by quarantine |
DOMAIN_RELOAD_INTERVAL_MS |
1000 |
Interval for hot-reloading domain rules from SQLite |
INGEST_DURABLE_ACK |
true |
Write raw + pending manifest before SMTP 250 |
INGEST_STORAGE_FSYNC |
false |
fsync files and directories before ACK; protects against power loss when enabled but reduces throughput |
- Start the service and sign in at
/admin/login. - Select an ingestion mode: add one or more managed domains, or enable catch-all mode in system settings.
- Explicitly enable Web or API access for domains/catch-all policy that truly need public inspection; keep the defaults private.
- Deliver a test message to any matching mailbox, for example
code@example.com. - Inspect private mail in the admin console, or use an authorized public page/API.
- Issue automation clients an API Key with the minimum scope and minimum domain grant.
managed_only: accept only domains enabled in the database and matched by exact/subdomain rules.managed_plus_catchall: managed domains still use longest-suffix-first matching; other valid domains fall through to the system's*policy. Domains do not need to be pre-registered individually for deliveries that have already reached this SMTP service.- Catch-all mode does not change public DNS routing and cannot receive third-party mail whose MX does not point to this service. Receiving mail from the public Internet still requires an A/MX record for a domain you control and reachable TCP port 25; for testing, an upstream relay may deliver directly to this host.
- Public Web/API switches are disabled by default for new managed domains and the
*policy. A new mailbox'spublic_enabledflag defaults to enabled, but is a second-level gate under the domain switch; after a domain is explicitly made public, each mailbox can still be disabled. SMTP acceptance does not imply anonymous read access; authorized administrators or service Keys can always read private mail. - When one message is delivered to multiple recipients, recipients are de-duplicated by canonical mailbox. Shared message/raw/attachment files enter final cleanup only after every delivery expires.
- Creating a more specific managed suffix or changing plus/case canonicalization creates a persistent
domain_rehome_jobsrecord for existing catch-all/parent-domain mailboxes in the same write transaction. The worker processes at most 1,000 mailboxes per batch and commits independently, allowing SMTP writers to take priority between batches. Jobs resume from a cursor after cancellation, process exit, or a transient failure. Migration only moves mail one way into a more-specific managed rule, merges duplicate deliveries, and recalculates summaries; a managed mailbox never falls back to*because a rule was disabled.
GET /
GET /mail/{mailbox_address}
GET /mail/{mailbox_address}/{delivery_id}
GET /mail/{mailbox_address}/{delivery_id}/raw
GET /mail/{mailbox_address}/{delivery_id}/attachments/{attachment_id}
curl \
-H "X-API-Key: <your-public-api-key>" \
"http://127.0.0.1:8000/api/v1/public/mailboxes/code@example.com/messages"The endpoint supports limit, the legacy-compatible offset, and returns next_cursor. New
integrations should use next_cursor for pagination:
curl \
-H "X-API-Key: <your-public-api-key>" \
"http://127.0.0.1:8000/api/v1/public/mailboxes/code@example.com/messages?limit=20&cursor=<next_cursor>"New integrations can also use the v2 public-mailbox resource directly. A public Key uses Bearer
authentication and remains constrained by domain grant mode, mailbox glob, and domain/mailbox public
API switches:
curl \
-H "Authorization: Bearer <ri_public_...>" \
"http://127.0.0.1:8000/api/v2/public/mailboxes/code@example.com/messages?limit=20"/api/v2 accepts admin, service, and public Keys, but only standard Bearer headers. Credentials
in query parameters or X-API-Key are rejected. A public Key may call only /me and public mailbox
resources allowed by public.read; service and admin Keys are further narrowed by kind, scope,
domain, and mailbox authorization. JSON responses use strict envelopes, file downloads use their
corresponding media types, and errors use application/problem+json. List cursors are bound to the
resource and filters and signed with API_CURSOR_SECRET; rotating that secret immediately invalidates
old cursors. OpenAPI is available at /docs and /openapi.json.
curl \
-H "Authorization: Bearer <ri_service_...>" \
"http://127.0.0.1:8000/api/v2/messages?limit=50"The current v2 surface covers principal, public mailbox list/verification-code/detail/raw/attachment resources,
domain and DNS checks, mailbox operations, message detail/raw/attachment/reparse/delete, SMTP sessions
and events, audit, dashboard, manual cleanup/clear, system settings, and the full API-Key and admin
lifecycle. The admin live WebSocket (plus its deprecated SSE compatibility route) and cross-message bulk deletion remain under /api/v1/*. New code should
prefer resources covered by v2 and follow the operations actually present in OpenAPI.
Every v2 cursor has a 2,048-byte input limit, an HMAC signature, and resource/filter binding. Retained v1 public-mailbox cursors are also bound to the calling credential and mailbox. v1 domain lists and SMTP events use hard pagination, and bulk deletion accepts at most 1,000 delivery IDs per request. v2 API-Key listing scans at most 5,000 candidates per request; when a low-privilege principal filters out many old Keys, it returns a continuation cursor instead of scanning without bound to fill a page.
Admin sessions have three roles:
viewer: read runtime status, domains, mailboxes, messages, SMTP, audit, settings, Key metadata, and administrator metadata.operator: all viewer permissions plus write access for domain, mailbox, and message handling.superadmin: high-risk system settings, API Keys, administrators, password resets, and session revocation; the system prevents deleting or demoting the last enabled superadmin.
API Keys first constrain selectable scopes by kind: public may use only public.read, service is
for business resources, and only admin may hold Key/administrator/system-write scopes. Domain grants
must be explicit:
none: grants no domains;selected: grants only listed domain IDs; an empty list still denies access;all: grants all current and future domains.
Mailbox globs, allowed IP CIDRs, Header/Query transport modes, per-minute limits, expiry, and revocation further narrow permissions. An empty domain list on a new Key no longer implicitly means all domains. Key creation, update, rotation, revocation, and deletion re-read caller and target permissions in the same SQLite writer transaction, preventing authorization races caused by policy changes while a request waits in the queue.
Administrator delegation is re-authorized in the write transaction as well: creating a login-capable
account requires both admins.write and admins.credentials.write; password reset and session
revocation require admins.credentials.write and admins.sessions.write respectively, and the target
role's effective scope may not exceed the caller's own scope.
Domain, mailbox, and message endpoints filter resources by selected grants. Global resources such as
dashboard/live status, SMTP sessions, audit, system settings, maintenance, and administrators cannot
be safely partitioned by domain, so their v1/v2 endpoints require domain_grant_mode=all; v2 API-Key
lifecycle operations also require all-domain authorization. A restricted v1 Key manager can operate only
on Keys within the caller's scope and domain grants.
A selected-domain principal may update ordinary policy for an authorized domain, but changing the
domain identifier root_domain changes the authorization boundary and is therefore limited to a
principal with domain_grant_mode=all. The service layer confirms this condition again in the same
write transaction.
/health/liveis the process liveness probe;/health/readychecks runtime state, background tasks, SQLite, the storage directory, and minimum free disk;/versionreports the application version, recommended APIv2, and the list of supportedv1/v2APIs./metricsexposes Prometheus text metrics including route-aggregated request counts/latency, in-flight requests, background tasks, readiness, process CPU/memory, and uptime. WithMETRICS_TOKEN, use Bearer orX-Metrics-Token; whenMETRICS_ENABLEDis disabled, the endpoint returns 404. A non-loopback bind with metrics enabled and no token refuses startup; loopback development is the only mode that permits tokenless access.- HTTP responses carry a safe
X-Request-ID. Access logs record route templates but not query strings, preventing compatibility Query Keys from leaking into logs.LOG_FORMAT=jsonsuits collectors andtextsuits local troubleshooting. Formatting and output run on a separate thread with capacity 4,096, so a slow log sink does not block the event loop; queue-full, output-failure, and close-timeout drops are counted byrapid_inbox_log_records_dropped_totalwith fixed reasons. - The
/admindashboard gathers database/disk state asynchronously through a short-TTL shared snapshot and displays RPS, P95, receipt/delivery/rejection/parse-failure counts, SMTP, parse queues, DB/WAL, disk, background jobs, and recent cleanup. Domain/mailbox/message/Key/audit totals are maintained by transaction triggers in one counter row; traffic uses minute buckets, so a 24-hour query reads at most about 1,441 buckets rather than scanning linearly with that day's message count.
Rapid Inbox is designed for a single-host local-disk architecture. It is not an unlimited horizontal scaling promise or an unmeasured throughput guarantee:
- SQLite uses WAL for concurrent reads, but still has one write transaction at a time. Ingestd can parse and write files in parallel; SQLite group commits are serialized briefly by a mutex, and Python changes are serialized by the
DatabaseWriterwrite lock. /api/v2uses Runtime-private persistent read-only actors withmode=ro/query_onlyconnections. Admission, waiting, and deadlines are bounded, and maintenance drains and closes them first. A single actor is a conservative default; benchmark the actual workload before adding connections, because blindly adding threads can amplify Python row-materialization GIL/futex contention. Write connections explicitly usesynchronous=FULL.- The
/api/v2SQLite hot path is offloaded to dedicated actors and raw/attachment responses remain streamed. The dashboard uses a roughly 1.5-second shared cache and a stampede-prevention lock. New high-concurrency integrations should prefer v2; retained v1 routes remain compatibility surface. These optimizations do not remove disk IOPS or the SQLite single-writer ceiling. - API-Key authentication uses a bounded approximately two-second in-process cache. Cache hits do not switch to the default thread pool; misses perform an asynchronous database read. Key changes in this process actively invalidate the cache after commit; selected-domain authorization is not cached to preserve FK-level fail-closed behavior.
last_used_atis written at most about every 30 seconds, so it is an operations signal rather than a per-request audit trail. - API-Key rate limiting uses a fixed-memory token bucket per Key. Bucket capacity equals the per-minute quota and refills evenly over 60 seconds, allowing short bursts up to the bucket capacity. State remains per HTTP process; with N workers, total available quota is roughly N times the per-process quota. Enforce a strict global quota at a reverse proxy/gateway.
- Run exactly one ingestd and one HTTP process for a data directory. Do not scale the Compose
appservice, deploy multiple Swarm/Kubernetes replicas, or point independently managed containers at the same volume. Do not place SQLite WAL on a network filesystem without reliable POSIX locking, and do not let multiple hosts share oneapp.db. - C++ and Python SMTP currently do not implement or advertise STARTTLS. For encrypted public transport, terminate TLS at a verified SMTP proxy or expose the receipt port only to a trusted network; place the HTTP admin plane behind an HTTPS reverse proxy as well.
Before tuning workers, batches, queues, fsync, or HTTP concurrency, use the repository stress scripts on
the actual disk to measure throughput, P95/P99, WAL growth, 451 ratios, recovery time, and power-loss
requirements, and reserve explicit limits for queues and disk.
Rapid Inbox stores structured data in SQLite and splits message content across local directories:
storage/
├── app.db # SQLite indexes and metadata
├── raw/ # Raw EML
├── text/ # Parsed plain text
├── html/ # Parsed HTML
├── attachments/ # Attachments
├── manifests/ # Manifests used for startup recovery
├── quarantine/ # Tasks and manifests that could not be persisted/verified
└── tmp/ # Temporary files
Mail no longer uses a global hard-coded "10-minute" retention period. Managed domains set each
delivery's expires_at through retention_days; when unset, deliveries do not expire automatically.
Catch-all delivery uses CATCH_ALL_RETENTION_DAYS, whose default 0 likewise means no automatic
expiry. Domain-policy changes affect only newly created deliveries and do not rewrite historical
expiry times.
Cleanup removes expired deliveries in CLEANUP_BATCH_SIZE batches. Message metadata is removed only
after every delivery for that message has expired. The same database transaction registers raw,
body, manifest, and attachment paths in file_gc_tasks; actual files are deleted outside the
transaction, with the failure reason recorded and retries using exponential backoff. This avoids
silently leaving half-applied state between a database commit and filesystem failure. SMTP sessions,
empty mailboxes, metric buckets, and audit records have independent retention settings.
Each cleanup run also removes quarantine files older than QUARANTINE_RETENTION_DAYS and incrementally
scans, up to ARTIFACT_SWEEP_BATCH_SIZE, for raw/text/HTML/attachment artifacts with no database,
manifest, quarantine, file-GC, or in-flight ingestion reference. A scan pass resumes across runs from
its previous position. A file must be at least ORPHAN_ARTIFACT_GRACE_SECONDS old to qualify, reducing
races with files being persisted or recovered. Completed maintenance records are removed in batches
according to MAINTENANCE_RUN_RETENTION_DAYS.
The Python parsing queue limits both message count and raw bytes, and the budget covers queued and active workers. Once raw data, manifests, and database state have been persisted, a temporarily full queue still preserves SMTP success. A background pending scan re-enqueues older messages first so already acknowledged mail is not discarded for memory backpressure.
An admin/API DELETE immediately marks the target delivery deleted and sets expires_at to the
current time. The next cleanup run hard-deletes it; if the message has no other deliveries, metadata
and files continue through the file-GC outbox. DELETE therefore takes immediate effect for read
visibility, while disk reclamation is an asynchronous, retryable operation. Call
POST /api/v2/maintenance/cleanup to accelerate processing.
Deleting all mail in a mailbox does not update all historical deliveries in one long transaction.
In the final authorization transaction, the service freezes the mailbox's current deletion generation
and maximum delivery rowid, advances the mailbox to the next generation, and creates a persistent
mailbox_bulk_delete_jobs record. The worker processes at most 1,000 rows per batch and can resume
after cancellation, failure, or restart. Deliveries that arrive or migrate after job creation inherit
the new generation, so an old job cannot delete them even if retention deletes the highest rowid and
SQLite reuses it. Job creation is the authorization linearization point: domain or mailbox grants
revoked before it are rejected atomically; an authorized, persisted job completes within its original
scope.
Only one C++ ingestd may use a given STORAGE_ROOT. On startup it holds a kernel file lock on
.ingestd.instance.lock; normal exit and crashes release the lock automatically. The lock file is
intentionally retained, and only the OS lock indicates ownership. The SQLite writer reuses one
connection and prepared statements across batches. SQLite errors, database-file replacement, or a
maintenance drained ACK invalidate the session so it can be rebuilt safely for the next batch.
Startup recovery checks manifest and raw-file size/SHA-256 and restores uncommitted domain policy,
messages, deliveries, and parsed results. Full history, permanent-failure retries, and same-timestamp
watermark paths are processed in batches through temporary on-disk SQLite, so core replay state does
not grow linearly with total history. Each manifest and each decode/replay batch is limited to 16 MiB.
A manifest with missing domain policy, corruption, or an over-budget payload fails closed into
quarantine; it neither infers public access nor blocks recovery of other mail. "Clear all messages"
in the admin console creates a cross-process .maintenance.lock, makes C++ ingestd temporarily return
421/451, stops parsing, clears mail tables, atomically moves raw/text/html/attachments/manifests/tmp,
and compacts SQLite. Domains, administrators, API Keys, audit/maintenance records, and quarantine are
retained for separate forensic disposal. An expired heartbeat is accepted only after its PID is
confirmed to have exited; a live or unverifiable process must produce a matching drained ACK or time
out.
Do not use mv, os.replace, or backup restoration to replace app.db while the service is running.
A reader in one process can detect an inode change, but a Python writer, C++ ingestd, other HTTP
processes, and -wal/-shm sidecars cannot atomically switch through a single connection pool. To
restore a database, stop every HTTP/SMTP process, handle the main file and sidecars according to the
SQLite backup procedure, run integrity checks, and then start the entire service again.
The current 0.x series evolves toward the new security model and does not promise legacy call
semantics:
- New domains default to
public_web_enabled=falseandpublic_api_enabled=false; public access must be explicitly enabled. - An empty API-Key domain list no longer means all domains. Existing Keys migrate to
selectedwhen grants exist, or fail-closednone; explicitly chooseallwhen access to future domains is needed. - API v2 uses
Authorization: Bearer, strict fields, and cursors; it rejects Query Keys and does not guarantee compatibility with v1 response shapes. Internet-facing deployments must now provide a stableAPI_CURSOR_SECRET. - Mail expiry is delivery-level
expires_at;retention_days=NULL/0means no automatic expiry and no longer inherits the old global 10-minute rule. - C++ ingestd enables durable ACK by default. If an earlier deployment depended on the very low latency of "in-memory enqueue means 250," it can be disabled explicitly, with the risk of losing mail on an abnormal exit.
Before every upgrade, retain a restorable copy of configuration, mail artifacts, and SQLite. For
Docker, stop the deployment, back up .rapid-inbox-docker/rapid-inbox.env and the named volume, check
out the reviewed target commit, then run ./docker-deploy.sh; the wrapper starts SMTP only after HTTP
migration and readiness succeed. For systemd, sudo bash deploy/system/install.sh update stages and
builds first, then stops both writers, creates a consistent SQLite backup, migrates, switches the
release, and performs HTTP/SMTP acceptance. Manual deployments must stop all old HTTP/SMTP processes
and allow exactly one migrator before ingestd starts. Never let old and new binaries write the same
database concurrently. Afterward, recheck domain public switches, Key domain_grant_mode, retention,
Metrics Token, and fsync choices.
# Install
python3 -m venv .venv
.venv/bin/pip install -c constraints-dev.txt -e ".[dev]"
# Test C++ ingestd (build first so the cross-language integration test is not skipped)
cmake -S cpp/ingestd -B cpp/ingestd/build
cmake --build cpp/ingestd/build
ctest --test-dir cpp/ingestd/build --output-on-failure
# Python tests; when ingestd is built, these also run the cross-language integration tests
.venv/bin/pytest
# Selected test files
.venv/bin/pytest tests/test_admin_api.py tests/test_public_routes.pyUse the included scripts to deliver verification-code messages in bulk and sample CPU and memory for C++ ingestd / Python HTTP:
.venv/bin/python tools/smtp_stress_test.py \
--to code@example.com \
--count 5000 \
--concurrency 100 \
--json-output .rapid-inbox-run/smtp-stress.jsonBy default, the tool waits for and verifies SQLite persistence/parsing counts. Add --no-db-check to
measure only SMTP ACKs. The HTTP tool permits only GET/HEAD. Pass its Bearer token through an environment
variable to keep it out of shell history:
RAPID_INBOX_API_TOKEN='<ri_service_...>' \
.venv/bin/python tools/http_stress_test.py \
--url http://127.0.0.1:8000/api/v2/domains \
--count 5000 \
--concurrency 100 \
--json-output .rapid-inbox-run/http-stress.jsonBoth report throughput, P50/P95/P99, and failures. See the actual script help for all parameters:
.venv/bin/python tools/smtp_stress_test.py --help
.venv/bin/python tools/http_stress_test.py --helpRuntime and development dependencies are pinned to exact versions in pyproject.toml, and
constraints-dev.txt records one verified development dependency resolution. After pulling new code into an existing
virtual environment, rerun the installation command so entry points and dependency versions match.
- Never use the default administrator password in a public environment. Prefer disabling compatibility tokens; if they are required, configure separate high-entropy values.
- Catch-all mode accepts deliveries that have reached this SMTP service and whose RCPT domain syntax is valid, but it does not replace DNS/MX configuration. Read access is private by default; do not enable public Web/API globally for convenience.
- When
/metricsis enabled on a non-loopback bind,METRICS_TOKENis mandatory or startup is refused; the metrics endpoint can also be disabled. - API-Key token buckets are local to one HTTP process. Multi-process deployments must also enforce global rate limits at a trusted reverse proxy.
- Docker maps host port
25to non-privileged container port2525; the systemd unit grants onlyCAP_NET_BIND_SERVICEto its dedicated account. Manual deployments still need an equivalent privilege boundary. - Public inboxes are intended for testing and temporary workflows, not sensitive long-term mail.
- Do not commit
.env,storage/, databases, or persisted mail files to Git.
For security issues, read SECURITY.md first.
Issues, fixes, and improvements are welcome. The project is maintained by @wendaochangsheng. Read CONTRIBUTING.md before starting; it covers the development workflow, tests, and pull-request guidance.
Rapid Inbox is released under the MIT License.
Built with ❤ for local-first email workflows