Review UI for YOLO dataset versions on a mounted disk.
| Page | What it is for |
|---|---|
| Files | Browse the folder tree and download it — a folder as a zip, a single file, or a whole version with the rejected images left out |
| Review | Every image with its boxes and class names: filter, mark OK or not OK, comment |
| Fix | The rejected images, with their boxes editable — redraw, reclassify, send for approval |
| Stats | Image / class / box counts and how the classes are distributed |
| Build | Pick a class-balanced train/val set from the reviewed images and download it |
React + Vite + Tailwind in web/, served by a stdlib-only Python server
(pick.py) that also exposes the API.
cd web && npm install && npm run build && cd .. # once
python3 pick.py /path/to/datasets \
--review /path/to/review --host 127.0.0.1 --port 8800Open http://localhost:8800/.
Working on the UI? Run Vite instead of rebuilding each time — it proxies to the Python server, so keep both up:
python3 pick.py /path/to/datasets --review /path/to/review --port 8800
cd web && npm run dev # http://localhost:5173Every pull request and every push to main runs CI:
lint and tests for the server and the web app, and a Docker build from a clean
checkout that then starts the container and asks it for /api/versions. The
same checks run locally:
pip install Pillow pytest ruff
ruff check . && pytest # server
cd web && npm run lint && npm test # webLint is limited to things that are bugs — undefined names, unused imports,
hook misuse — not style. It matters most on the web side: the build compiles a
component that was never imported, or a variable read outside its scope, and
only the browser finds out. tests/test_review_fixes.py pins the code-review
fixes (the review-log race, input validation, symlinks in downloads).
Issues use templates for bugs and feature requests.
<version>/images/<split>/*.jpg
<version>/labels/<split>/*.txt
<version>/classes.txt # optional
A version is any folder holding an images/ directory, so the root can be
flat or grouped by dataset, and the two can be mixed:
/srv/data/datasets/legacy/… # version at the root
/srv/data/datasets/online/t0.0.0/… # grouped by dataset
/srv/data/datasets/online/v0.0.0/…
/srv/data/datasets/cs/t0.1.0/…
The version id is its path under the root — online/t0.0.0 — and the picker
groups the list by folder. Three levels are searched; the walk stops at each
version, so it never reads inside an images/ folder.
A flat images/*.jpg with no split folders works too — it becomes the split ..
Class names come from classes.txt in the version folder, or in its
labels/ folder, then from each folder above it up to the dataset root — first
found wins. So online/classes.txt covers every version under online/, and a
single version can still override it with its own. data.yaml is read only when
there is no classes.txt anywhere. One class per line, with or without the
index:
car 0 car 0: car
person 1 person 1, person
Blank lines and # comments are skipped. Indices are only believed when every
line carries one, so plain names that start with a digit stay intact. Stats says
which file the names came from.
Version folders are named by hand, and the prefix says how finished the dataset is:
tX.Y.Z— in progress (test)vX.Y.Z— finished (version)
The numbers move the same way under both prefixes:
| Bump | When |
|---|---|
t0.0.0 → t1.0.0 |
a dataset completely different from the ones before |
t0.0.0 → t0.1.0 |
images or classes added |
t0.0.0 → t0.0.1 |
corrections — missing labels, images to remove, resizes |
Versions are listed alphabetically, so every t sorts before every v.
A verdict is OK, not OK, Review — corrected and waiting to be
accepted (yellow) — or Deleted, for an image that should not be in the
dataset at all. Keyboard: 1 OK, 2 not OK, 3 delete, ← → move, esc
close. An OK jumps to the next image; a rejection stays put with the comment
box focused.
Deleting removes nothing from disk. It holds the image out of the export the
way not OK does, keeps it in the grid with a struck-through name, and is
undone by pressing 3 again. Deleted images stay out of the Fix queue — they
are not waiting to be repaired.
Zoom works the same here and in Fix: the wheel zooms toward the cursor, +
and - step, 0 fits the pane. Drag the image to pan — in Fix, where dragging
draws and moves boxes instead, pan by holding space, dragging with the
middle button, or switching on the Pan button in the toolbar.
The readout is the image's natural size, so 1:1 is one image pixel per screen pixel — the scale at which a box a few pixels out is visible.
Boxes opens the class list: tick a class to draw it, untick to hide it, and
click its swatch to recolour it. b clears every box for a clean look at the
picture and puts your selection back — it does not undo your filtering. Both the
colours and the hidden classes are remembered per browser and shared by Review,
Fix and Stats; in Review the hiding covers the card grid as well as the open
image.
A hidden class cannot be selected or grabbed in the editor, and with every class
hidden editing is off altogether, so nothing is redrawn by accident. Colours are
keyed by class name, so car keeps its colour across versions; Reset
colours puts the palette back. A chosen colour is a literal rather than a theme
token, so unlike the defaults it does not change between light and dark — the
label on the box flips between black and white to stay legible on it. Boxes, class tags
and the editor's handles keep their size as you zoom, so a handle stays
grabbable at any magnification.
Comments are a thread. Any image takes any number of them from any number of
reviewers, with or without a verdict, and a later verdict does not disturb them.
⌘↵ / ctrl↵ sends. edit rewords one of your own and × removes it; the
comment keeps its original author and place in the thread and is marked edited.
The card shows a bubble and a count, and Show → Has comments filters to them.
The thread scrolls in its own column, so the verdict buttons and the box table stay put however long it gets, and the box you type in stays at the bottom. The same thread is on the Fix tab beside the editor, with the image's status, so you can read why it was turned down while you repair it.
Verdicts and comments go to an append-only JSONL log per version/split under
--review, never into the dataset. Nothing is ever rewritten, so the history of
who said what is kept.
Everything marked not OK collects on the Fix tab, across every split of the
version. Open one and the boxes are editable: drag empty space to draw, drag
inside a box to move it, corners and edges to resize, 1–9 to set the class,
⌫ to delete, ⌘Z to undo, b to hide the boxes, plus the zoom controls
above.
Delete image is here too: it marks the image Deleted and takes it out of the queue, since it is no longer waiting to be repaired. Undo that from Review, where deleted images are still listed.
Saving does not pass a verdict. It marks the image Review and it waits there until somebody else accepts it on the Review tab — nobody can accept their own correction. Marking it not OK again stays open to whoever fixed it, since that only takes the image back out. Revert to original drops the correction and returns the image to not OK.
Behind --user-header that rule is a real control. Without a proxy, names are
self-declared and it is a courtesy.
The dataset is mounted read-only and is never written to. A corrected label lives in the review log and is substituted into the zip at download time, so the corrected dataset exists only in the file you download.
Download reviewed dataset streams the version as a zip with the corrected
labels in place of the originals, and with every image marked not OK, Review or
Deleted — and its label file — left out. Two records travel inside:
EXCLUDED.csv for what was held back and why, and CORRECTED.csv for which
labels were redrawn and by whom. The dataset itself is untouched.
If any image has never been flagged at all, the download asks first and says how many, rather than quietly shipping images nobody has looked at.
The Build tab picks a class-balanced subset of one or more versions (they
must share a class list) and downloads it ready to train. It runs the same steps
as select_balanced_dataset.py: group consecutive frames into shots, drop
near-duplicates inside each shot (dHash), optionally send whole shots to train or
val, then fill each split with a share of empty images and, after that, a random
image of whichever class is furthest behind.
Images marked OK or never reviewed are eligible; Not OK, Deleted and corrections still waiting for approval never are. Labels are the reviewed ones, corrections included.
Preview colours every image by what happened to it — train, val, test, not
taken (budget reached), other classes only, near-duplicate, excluded — with the
reason on click, above the per-split report: available vs selected per class,
imbalance, empty images, classes that ran out, share in val. Download streams
the same selection as a zip: images/ and labels/ per split (keeping each
image's version and split in its path), train.txt + val.txt + data.yaml, or
test.txt when the val fraction is 0, and manifest.json with every option and
image.
Every preview and download goes to builds.jsonl in the review folder, with
who, when, the seed and every option. It keeps the newest 100 runs; older ones
are dropped as new ones are added. History lists them and Restore puts
a run's options back, so any build can be reproduced.
compose.yaml runs the app with no published ports; Caddy is the sole entrypoint
on the VPN address, doing basic_auth and setting X-Remote-User. With
--user-header X-Remote-User the reviewer name comes from the proxy and any
client-sent name is ignored.
Do not add a
header_up -X-Remote-Userline to thereverse_proxyblock. It looks like hardening and is the opposite: Caddy applies header operations add, then set, then delete, so a deletion written afterheader_up X-Remote-User {http.auth.user.id}strips the value that line just wrote. The app then sees no identity and files everything underunauthenticated. Theheader_upis a set, so it already replaces anything the client sent — there is nothing left to guard against.
If names ever stop arriving, the UI says so in a red bar across the Review page
and the server prints the same warning once to its log — docker compose logs pick. Work is never lost when this happens, only unattributed, and the
orphaned entries can be cleaned up by anyone once identity is flowing again.
Two untracked files hold everything host-specific, so nothing here is ever
committed and git pull can never overwrite a password:
| File | Copy from | Holds |
|---|---|---|
.env |
.env.example |
BIND_ADDR, the VPN address Caddy binds to |
caddy.env |
caddy.env.example |
one REVIEWER_<n>=<username> <hash> per reviewer |
Leave BIND_ADDR out and the app listens on loopback only — a missing .env
makes it unreachable rather than exposed on every interface.
cp .env.example .env && cp caddy.env.example caddy.env # once per host
docker run --rm caddy:2 caddy hash-password --plaintext 'THE_PASSWORD'
${EDITOR:-nano} .env caddy.env
docker compose build --build-arg DVC_GID=$(getent group dvc | cut -d: -f3)
docker compose up -dPaste each hash exactly as printed, $ signs and all. Changing a password or
adding a reviewer means editing caddy.env and docker compose up -d caddy —
the Caddyfile is not touched and no rebuild is needed.
caddy.envis passed withformat: rawfor a reason. A bcrypt hash is full of$, and Compose's normal interpolation eats them:$2a$14$Wh…arrives as$2a$14and every login fails with no obvious cause.
The username is what lands in X-Remote-User, so it becomes the name on every
verdict, comment and correction — and it is what the four-eyes rule compares.
You need at least two reviewers for anyone to accept a fix.
Mounts: /srv/data/datasets → /data/datasets ro, /srv/data/review →
/data/review rw.
Never add
header_up -X-Remote-Userto thereverse_proxyblock. Caddy applies header ops add, then set, then delete, so the deletion runs after the line that sets the identity and strips it again — every verdict and comment then lands underunauthenticated. The line that sets it is already a replace, so there is nothing to guard against.
If names stop arriving, the Review page says so in a red bar and the server
prints the same warning once to docker compose logs pick. Nothing is lost
when that happens, only unattributed, and anyone can clean up the orphans.
Everything the UI does is a plain HTTP call, so scripts can use it too.
| Route | Returns |
|---|---|
GET /api/versions |
{root, versions[], pillow} |
GET /api/version?v= |
counts, class names, per-split class counts |
GET /api/items?v=&split=&mode=&cls=&clsmode=&offset=&limit= |
images with boxes and their review flag |
GET /api/stats?v=&mode=&cls=&clsmode= |
the /api/version counts for the images those filters select |
GET /api/build/counts?v= |
{total, eligible, excluded: {reason: n}} — what a build may draw from |
GET /api/build/history |
every build preview and download, newest first |
GET /api/tree?path= |
one directory listing, with breadcrumbs |
GET /api/review/meta |
statuses, whether the log is writable, proxy user |
GET /api/review/summary?v= |
{ok, no, review, total, unflagged} |
GET /api/review/rejects?v= |
every image held back from the export |
GET /api/queue?v=&status= |
images needing work, across every split |
GET /api/review/export?v= |
review log as CSV |
GET /api/refresh |
drop the scan and flag caches |
POST /api/flag |
{v, split, image, status, reviewer} |
POST /api/comment |
{v, split, image, text, reviewer} |
POST /api/labels |
{v, split, image, boxes} — redraw; sets the status to review |
POST /api/labels/revert |
{v, split, image} — drop the redraw, back to no |
POST /api/build |
{versions[], classes[], num, empty, val, by, repeats, dedup, hamming, window, seed} — every image's state plus the report; logged to history |
POST /api/comment/delete |
{v, split, image, id, reviewer} — author only |
GET /img?v=&split=&n=&t=1 |
full image, or a cached thumbnail |
GET /file?path= |
one file |
GET /download?path=&exclude=1 |
streamed zip; exclude=1 applies the review |
GET /build.zip?p= |
the build for the JSON options in p, as a streamed zip |
mode is all|unreviewed|ok|no|review|deleted|commented|unlabeled|empty.
cls
takes one class index or a comma list (0,2,5); clsmode=any (default) keeps images
holding at least one, clsmode=all only those holding every one. Each POST
answers with the image's whole flag, verdict and full thread.
Zips are streamed as the files are read, so a multi-gigabyte version starts downloading immediately and is never staged in memory or on disk.
Older logs written by the previous UI used ok|fix|drop; fix and drop
read back as no, and the inline rejection note becomes the first comment in
that image's thread. Existing review work carries over untouched.