Repository navigation
feat(plugins): add lobsters column type - #27
Merged
Merged
Conversation
34th column type. Lobsters is a tech-focused community of ~30k engineers
running on lobste.rs — adjacent to Hacker News in audience but with
narrower, higher-signal submissions and a tag taxonomy that supports
filtered feeds (rust, ai, programming, security, ...).
Closes the obvious News & Web cluster gap — HN was the only general-tech
discussion column. Lobsters fills the slot for users who want a
second, smaller signal stream alongside HN.
Why keyless: Lobsters exposes JSON variants of every public page —
/hottest, /newest, /active, and per-tag /t/<tag> all return the same
story array via .json. No auth, no token. The story object shape is
documented at lobsters/lobsters @ app/views/stories/_story.json.jbuilder.
Pagination flows through /page/N/{mode}.json (page 1 is the bare root,
no /page/1/ segment — handled in endpointFor).
Standard 3-file plugin (plugin.ts / client.tsx / server.ts) plus the
3 registry edits (manifest, registry, server-registry — parity check
at server module init throws if drift). Same shape as hacker-news,
since the UX maps 1:1 (mode picker, optional filter input, score +
comment-count badges, comments URL distinct from external URL).
Lobsters-specific implementation details:
- Story object's submitter_user can be either a string username or an
object with .username — unwrapAuthor handles both shapes.
- description field is sanitised HTML (p, br, code, em, a only); a
targeted regex strip is safe without pulling in a parser. Falls back
to description_plain if upstream sends it (rare).
- Tag mode supports comma-separated multi-tag filter (/t/rust,go.json) —
matches Lobsters' upstream URL convention.
- Tag mode with empty tag falls back to hottest in server.ts to avoid a
404 on /t/.json — column always renders something rather than throwing.
- hasMore detection uses upstream page size (>=25) NOT post-filter slice,
so visible PAGE_SIZE clamp doesn't stop pagination early on full pages.
- Schema-drift safe: stories missing short_id or title are dropped rather
than rendering as dead rows.
- User-agent header identifies minitor — Lobsters' admins ask scrapers
to identify themselves; user-driven dashboard polling is well within
their rate-limit guidance but a recognisable UA helps if they ever
need to throttle cooperatively.
Anchor icon (a nod to the lobster anchor in lobste.rs branding without
copying the proprietary logo). Brand red #ac130d (the lobster-claw red
on lobste.rs/about) — distinct from HN's orange #ff6600 so the two
news-source columns stay visually differentiated when stacked together.
Renderer matches hacker-news layout (badge + author + relative time +
serif title + snippet + tag pills + score/comments footer). Tag pills
are unique to Lobsters — first 4 tags rendered as small pills under the
snippet, since Lobsters' tag taxonomy is a key part of the signal.
No new env vars / build deps / schema migration — works the moment
users add it, preserving the "first column up in under a minute, zero
infra" README promise. README updated: column count 33 → 34, News &
Web row 4 → 5, hero paragraph picks up "Lobsters" alongside "Hacker
News".
aaronjmars
added a commit
that referenced
this pull request
Jul 9, 2026
Lockfile-only transitive bumps; no package.json changes needed. - brace-expansion 5.0.5 -> 5.0.7 (GHSA-jxxr-4gwj-5jf2, #20, moderate) ReDoS-style DoS: large numeric range defeated the documented max guard. Affected copies under @ts-morph/common (shadcn) and @typescript-eslint. - @babel/core 7.29.0 -> 7.29.7 (GHSA-4x5r-pxfx-6jf8, #27, low) Arbitrary file read via sourceMappingURL comment. Satisfies shadcn ^7.28.0. - esbuild 0.27.7 -> 0.28.1 (GHSA-g7r4-m6w7-qqqr, #26, low, dev-only) Arbitrary file read via dev server on Windows. Pulled up by bumping the transitive tsx 4.21.0 -> 4.23.0 (drizzle-kit allows tsx ^4.21.0). npm audit: 0 vulnerabilities. next build passes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
34th column type — Lobsters (lobste.rs), a tech-focused community of ~30k engineers. Adjacent to Hacker News in audience but with narrower, higher-signal submissions and a tag taxonomy that supports filtered feeds (rust, ai, programming, security, …). Four modes: Hottest (front page), Newest, Active (sorted by recent comment activity), and Tag (single tag, or comma-separated multi-tag).
Why
Closes the obvious News & Web cluster gap — HN was the only general-tech discussion column. Lobsters fills the slot for users who want a second, smaller-signal stream alongside HN. Anyone running a founder dashboard / open-source maintainer dashboard / ML-research dashboard tends to read both — being able to stack `hacker-news` and `lobsters` next to each other on the same deck is the natural layout.
Why keyless: Lobsters exposes JSON variants of every public page —
/hottest,/newest,/active, and per-tag/t/<tag>all return the same story array via.json. No auth, no token. Pagination flows through/page/N/{mode}.json(page 1 is the bare root, no/page/1/segment — handled inendpointFor).Changes
lib/integrations/lobsters.ts(new) — keyless JSON fetcher for the four modes, schema-drift-safe story mapper (drops rows missingshort_id/titlerather than rendering dead content), HTML strip for thedescriptionfield (Lobsters returns sanitised fragments — p/br/code/em/a only — so a targeted regex strip is safe without pulling in a parser; falls back todescription_plainif upstream sends it). Author unwrap handles bothsubmitter_user: \"name\"andsubmitter_user: { username: \"name\" }shapes seen across recent API output. User-agent header identifies minitor — Lobsters' admins ask scrapers to identify themselves so they can throttle cooperatively if needed.lib/columns/plugins/lobsters/{plugin,client,server}.ts(new) — standard 3-file plugin. Zod{ mode: \"hottest\"|\"newest\"|\"active\"|\"tag\", tag: string }. Anchor icon (a nod to the lobster claw in lobste.rs branding without copying the proprietary logo). Brand red#ac130d(the lobster-claw red on lobste.rs/about) — distinct from HN's orange#ff6600so the two news-source columns stay visually differentiated when stacked together.lib/columns/plugins/manifest.ts,lib/columns/registry.ts,lib/columns/server-registry.ts— three registry edits per the README contract. Parity check at server module init throws if any registry drifts.README.md— column count 33 → 34, News & Web row 4 → 5, hero paragraph picks up "Lobsters" alongside "Hacker News".Lobsters-specific quirks handled
/t/rust,go.json) — matches Lobsters' upstream URL convention.server.tsto avoid a 404 on/t/.json— column always renders something rather than throwing.hasMoredetection uses upstream page size (>=25) NOT post-filter slice, so the visiblePAGE_SIZEclamp doesn't stop pagination early on full pages.What you get
No new env vars / build deps / schema migration — works the moment users add it, preserving the "first column up in under a minute, zero infra" README promise.
Built autonomously by Aeon