embyx-web

Standalone web management surface for embyx workflows. Phase 2 adds a durable Fill Actor backend, a FastAPI API, and a packaged React management page while keeping filesystem mutation independent from HTTP and UI code.

• Framework-independent FillActorService with explicit scan/plan/apply boundaries.
• SQLite and in-memory repositories for plans, private candidates, results, durable jobs, leases, and local/CloudDrive move journals.
• Explicit SQLite migrations, WAL mode, write-readiness probes, cancellation-safe operations, and restart recovery.
• A bounded persisted scan/apply queue with atomic claim/lease/heartbeat fencing across processes, durable file-level apply progress, and durable, idempotent cancellation of queued or running scans.
• Strict .strm parsing that maps a configured mount namespace to CloudDrive API-native paths, snapshots remote file identity, and moves only the referenced video through CloudDrive with conflict policy Skip.
• Observation-only recovery for ambiguous CloudDrive timeouts and restarts. Legacy local journal records are never replayed in CloudDrive mode; they keep apply unavailable until an operator resolves or migrates them.
• FastAPI endpoints with stable JSON error codes, request/actor/video limits, readiness checks, and Bearer auth for writes.
• React + TypeScript + Vite UI with polling, refresh-safe scan and move progress, in-progress scan cancellation, grouped results, move confirmation, conflicts, partial failures, one-click bulk magnet copying, RSSHub cache readiness, FreshRSS hand-off, and stale-plan recovery.
• Per-actor RSSHub feed prewarming persisted alongside scan jobs, with lease-fenced updates and cache-HIT detection before a FreshRSS subscription action is exposed.
• Static frontend assets included in the Python wheel.
• A lazy, origin-checked adapter boundary for a narrow embyx compatibility API; tests never load legacy secrets.

Public plans contain only opaque IDs and display-safe filenames. Mapping paths, CloudDrive API paths, identities, and operation state remain in private SQLite records. In CloudDrive mode, applying a candidate follows this sequence:

1. Revalidate plan revision, expiry, mapping-root identity, the .strm inode/content digest, its configured source root, and the planned CloudDrive file ID, size, write time, and hashes.
2. Idempotently ensure one target brand directory directly below the configured move-in root, then require its target filename to be absent.
3. Persist prepared and then submitting before invoking one MoveFile call with conflict policy Skip.
4. Force-refresh both remote parents and record success only when the source is absent and the destination has the planned identity.

Timeouts, disconnects, and process exits after submission become unknown. Reconciliation only observes the two remote paths; it never blindly submits another move. An unresolved source is unique across plans and prevents duplicate operations. The derived .strm is never renamed, linked, deleted, or used as the CloudDrive API source. Legacy local move code remains only for compatibility tests and old databases; environment-based production bootstrap refuses to enable apply unless the CloudDrive path mapping is complete.

uv sync --locked

The production bootstrap reads only explicit EMBYX_WEB_* variables:

Create the sentinel deliberately in the actual mounted filesystem of the actor root, every additional root, and the move-in root. A missing marker makes readiness fail and prevents scanning/reconciliation, protecting against an empty mount point being mistaken for the real library.

Apply and all reconciliation are disabled by default. Enabling it requires all three CloudDrive path variables and a runtime adapter configured with EMBYX_RUNTIME_CLOUDDRIVE_ADDRESS, EMBYX_RUNTIME_CLOUDDRIVE_API_TOKEN, and EMBYX_RUNTIME_CLOUDDRIVE_SECURE. Inject the token from a Secret; never commit it or bake it into this public image. The token needs only list, view-properties, create-folder, and move permissions rooted high enough to cover the configured sources and destination. The switch does not affect scans, magnet lookup, RSSHub prewarming, or FreshRSS actions. CloudDrive connectivity remains part of scan readiness when CloudDrive mode is configured; unresolved legacy local journals affect only apply readiness.

Binding to a non-loopback address is rejected unless both EMBYX_WEB_API_TOKEN and EMBYX_WEB_TLS_TERMINATED=true are set. The flag is an operator assertion: this app does not terminate TLS itself, so the listener must be reachable only through the configured TLS reverse proxy.

Run after configuring the environment:

uv run embyx-web

Do not point the loader at the interactive legacy fill_actor entrypoint. The configured module must be a file-backed module located under EMBYX_WEB_RUNTIME_ROOT and expose:

async def list_actor_video_ids(actor_id: str) -> tuple[str, ...]: ...
def resolve_brand(video_id: str) -> str | None: ...
async def find_sukebei_magnet(video_id: str) -> str | None: ...
async def list_cloud_directory(api_dir: str) -> tuple[dict[str, object], ...]: ...
async def ensure_cloud_directory(parent_api_dir: str, folder_name: str) -> dict[str, object]: ...
async def move_cloud_file(source_api_path: str, destination_api_dir: str) -> dict[str, object]: ...
async def aclose() -> None: ...

The loader resolves and validates every package/module origin before execution. This prevents an unrelated same-named module on sys.path from running.

The production container defaults to src.embyx_runtime.fill_actor_api, the required compatibility package layout in the embyx runtime image. The selected base image must contain that package under /app; changing only PYTHONPATH or mounting the media PVC cannot supply code that is absent from the image.

• POST /api/fill-actor/plans validates actor IDs and atomically enqueues a persisted job.
• GET /api/fill-actor/plans/{plan_id} returns {job, plan, feeds}. The plan becomes visible once its scan result is persisted; the job can remain running briefly while per-actor RSSHub cache probes reach a terminal state.
• POST /api/fill-actor/plans/{plan_id}/cancel atomically cancels a queued or running scan. The first request and repeats after a successful cancellation return 200; an unknown job returns 404 unknown_plan, and a job that already reached another terminal state returns 409 plan_not_cancellable. The persisted representation is state=failed with error_code=job_cancelled, which the UI presents as a neutral cancellation rather than a failure. This mutation requires Bearer auth but remains available when media roots are temporarily unavailable.
• POST /api/fill-actor/plans/{plan_id}/apply accepts only a published plan and applies opaque candidate IDs with revision checking. This synchronous endpoint remains available for API compatibility.
• POST /api/fill-actor/plans/{plan_id}/apply-jobs accepts a client-generated request ID and atomically enqueues a durable apply job. Repeating the same ID with the same plan, revision, and ordered candidate set returns the same job; reusing it for a different request returns a conflict.
• GET /api/fill-actor/apply-jobs/{job_id} returns the apply job and its final result once available. Progress counts candidates that have reached a persisted outcome, including conflicts and failures; it does not claim byte-level CloudDrive transfer progress.
• GET /api/health reports database write-readiness and root/sentinel readiness.

Responses never expose private paths or raw exception messages. The queue defaults to 2 workers and at most 32 active queued/running jobs per process configuration.

Cancellation first persists the terminal job and pending feed states. When the request reaches the process that owns the running job, it then stops the actor-catalog, RSSHub, magnet, and filesystem-scan work before returning success. If another process handles the request, the owner observes the lost lease on its next heartbeat and performs the same cleanup asynchronously. Filesystem enumeration uses cooperative cancellation and checks its stop token between roots and directory entries. A single NFS system call already blocked inside the kernel cannot be interrupted safely; a local-owner cancellation response waits for that call to return and for the scan worker to exit instead of claiming the task stopped while background I/O is still active.

Apply jobs are deliberately not cancellable. Once a local or CloudDrive move has started, the current candidate is allowed to reach a durable outcome; shutdown or lost ownership prevents subsequent candidates from starting. The UI stores only the opaque apply job identity and public request metadata in session storage so a refresh or lost POST response can resume polling without submitting a second move.

When RSSHub integration is enabled, each scan schedules the cluster-local /javbus/star/{actor-id} request before actor catalog scanning begins. A background HEAD probe waits for a successful XML response carrying RSSHub-Cache-Status: HIT; cache failures do not fail the library scan. Once ready, the UI builds FreshRSS's standard prefilled add-feed URL from EMBYX_WEB_FRESHRSS_URL, using EMBYX_WEB_FRESHRSS_RSSHUB_URL for the feed URL that FreshRSS itself resolves. It also exposes the configured FreshRSS site as a separate action. Deployment-specific URLs must be injected through environment variables rather than committed as application defaults. The user remains inside FreshRSS's normal authenticated and CSRF-protected confirmation flow, so embyx-web never stores FreshRSS credentials.

cd frontend
npm ci
npm run dev

The development server proxies /api to 127.0.0.1:8000. When auth is enabled, the UI asks for the token after a 401 and retains it only in browser session storage; no token is embedded in build output.

Build the frontend before building the Python artifacts:

cd frontend
npm run build
cd ..
uv build

Vite writes into src/embyx_web/static, which Hatch includes in the wheel and the backend serves at /.

The Dockerfile builds the React bundle and Python wheel in separate stages, installs only locked runtime dependencies and the wheel under /opt/embyx-web, then copies that tree into a digest-pinned compatible ghcr.io/cyrahs/embyx runtime image. The final process uses /app/.venv/bin/python, so the original /app/src package and the legacy virtual-environment dependencies remain visible. PYTHONPATH places /opt/embyx-web first and retains /app for legacy imports.

Build the image with the verified default runtime base, or select another immutable compatible embyx image explicitly:

docker build -t embyx-web:local .
docker build \
  --build-arg EMBYX_RUNTIME_IMAGE=ghcr.io/cyrahs/embyx@sha256:<compatible-digest> \
  -t embyx-web:local .

Container defaults are:

• state database: /var/lib/embyx-web/embyx-web.sqlite3;
• compatibility-layer logs: /var/lib/embyx-web/log;
• runtime root: /app;
• runtime module: src.embyx_runtime.fill_actor_api;
• entrypoint: /app/.venv/bin/python -m embyx_web.

Mount a persistent volume at /var/lib/embyx-web and mount the media-embyx PVC read-only at the paths referenced by EMBYX_WEB_ACTOR_ROOT, EMBYX_WEB_ADDITIONAL_ROOTS, and EMBYX_WEB_MOVE_IN_ROOT. The compatibility API does not need the legacy config.toml; CloudDrive connection values are injected explicitly at runtime, and its optional magnet log directory is controlled only by EMBYX_RUNTIME_LOG_DIR. The image keeps the safe loopback bind default; a Kubernetes Deployment normally sets EMBYX_WEB_HOST=0.0.0.0 together with a Bearer token and EMBYX_WEB_TLS_TERMINATED=true behind a TLS-terminating proxy.

Production mount paths, CloudDrive namespaces, service addresses, and browser-facing URLs belong in deployment env or Secrets, not this public repository. The mapping PVC stays read-only, CloudDrive storage is not mounted into this service, and every configured mapping root must contain its deliberately created sentinel marker.

The container smoke check verifies that the installed web package comes from /opt/embyx-web, legacy src and tap remain discoverable from /app, the compatibility module passes the origin-checked loader, static assets ship in the wheel, and the runtime defaults and entrypoint are intact. Publish a compatible embyx base image before building embyx-web; the smoke check intentionally fails when the selected base predates src.embyx_runtime.fill_actor_api:

bash tests/container_smoke.sh embyx-web:local

uv lock --check
uv sync --locked
uv run pytest
uv run ruff check .
uv run ruff format --check .
uv run python -m compileall -q src tests

cd frontend
npm run lint
npm test
npm run build
cd ..
uv build

docker build -t embyx-web:ci .
bash tests/container_smoke.sh embyx-web:ci

Pushes to the default branch run the same backend, frontend, and container contract checks before publishing ghcr.io/<owner>/embyx-web:latest plus an immutable sha-<commit> tag for both linux/amd64 and linux/arm64. Production GitOps should resolve the published manifest and pin its digest rather than deploy the mutable tag.

Schema 6 adds the apply_jobs table. Before upgrading an existing SQLite database, take and verify an online backup while no jobs are queued or running. Images that only understand schema 5 cannot start against a migrated database; rolling back to such an image therefore also requires restoring the matching schema-5 backup. Prefer a forward fix after any real move has run on schema 6 so its audit records are not discarded. For a staged production rollout, deploy and verify the schema-6 backend with the previous UI first, then use that image as the rollback target while enabling the asynchronous progress UI.

Operators must keep the database, lock file, runtime package, and configured media roots writable only by trusted users.