Skip to content

feat: add PyPI column — 42nd column type, Python package registry companion to npm - #36

Merged
aaronjmars merged 1 commit into
mainfrom
feat/pypi-column
May 14, 2026
Merged

aaronjmars merged 1 commit into
mainfrom
feat/pypi-column

Conversation

@aaronjmars

Copy link
Copy Markdown
Collaborator

What

42nd column type — a PyPI column. Three modes: updates (recent package releases via RSS), new-packages (newly registered via RSS), top-30d (top 8000 packages ranked by 30-day downloads via the community mirror). Optional keyword filter, optional weekly-downloads badge via pypistats.org.

Why

npm shipped May 12 as the 41st column — JavaScript ecosystem covered. PyPI is the natural pair: same package-registry pattern, Python ecosystem audience. Together they bracket the two registries that the AI / ML research crowd already lives in. Both keyless, both polite to anonymous polling.

The May-12 repo-actions article proposed Reddit and Bluesky as the next minitor columns — but both already exist in the manifest. The article was wrong about what was missing. PyPI is the actual highest-impact gap given the May-12 npm precedent and the existing AI/ML cluster (huggingface artifacts, arxiv papers — but no Python package registry).

Changes

Files added

  • lib/integrations/pypi.ts — three keyless source surfaces:
    • pypi.org/rss/updates.xml — RSS 2.0 feed of recent package releases (~40 entries, time-ordered with version bump info).
    • pypi.org/rss/packages.xml — RSS 2.0 feed of newly registered packages.
    • hugovk.github.io/top-pypi-packages/top-pypi-packages-30-days.min.json — community-maintained mirror of the BigQuery-backed download stats, top 8000 packages, updated daily. The only "trending" surface PyPI itself doesn't expose.
    • pypistats.org/api/packages/{name}/recent?period=week — keyless weekly-downloads badge enrichment, parallel-fetched per row.
  • lib/columns/plugins/pypi/plugin.ts — metadata (id, label, accent #3776AB, Package2 icon, news category, Zod schema with mode + keyword).
  • lib/columns/plugins/pypi/server.ts — standard defineColumnServer wrapper around fetchPypiPage.
  • lib/columns/plugins/pypi/client.tsx — ConfigForm (mode select + keyword input) and ItemRenderer (PyPI brand pill, version badge, author handle, line-clamped description, weekly + monthly downloads).

Files modified

  • lib/columns/plugins/manifest.ts — registered pypi meta.
  • lib/columns/registry.ts — client-side UI map.
  • lib/columns/server-registry.ts — server-side fetcher map (init-time parity check picks it up).
  • README.md — News & web cluster (8) → (9), column count 41 → 42, hero paragraph mentions PyPI, keyless-columns list includes PyPI.

How it works

The integration is structured around three mode handlers sharing one set of helpers:

  • RSS modes (updates, new-packages) — fetch the XML feed, parse with a small CDATA-aware tag extractor (no XML library dependency added), apply optional keyword filter, slice for pagination, enrich each row with weekly downloads in parallel.
  • top-30d mode — fetch the JSON mirror with cache: "force-cache" + next.revalidate: 3600 (data updates daily, hourly cache is plenty). Apply optional name-only keyword filter, slice for pagination, parallel-enrich weekly downloads.
  • All modes use a single User-Agent: minitor/1.0 (+url) per PyPI's TOS request to identify anonymous clients.

Integration quirks (documented in source)

  1. PyPI's updates feed wraps title as {name} {version} (space-separated) while new-packages uses just {name} — parseUpdateTitle handles both shapes.
  2. The top-30d rank-feed's createdAt is the mirror's last_update, which is misleading as relative-time (every row in the slice shares the same timestamp). The renderer detects rank rows via the absence of version + description and suppresses the time pill.
  3. pypistats.org weekly-downloads failures degrade silently to 0 — a single dead row never kills the page.

Visual

PyPI brand blue #3776AB (matches python.org navbar and the Python logo's left-half wing). Distinct from the existing palette: npm registry red #CB3837, devto indigo #3b49df, github-actions blue #2088FF, arxiv #B31B1B. Package2 icon — distinct from npm's Package so the two columns are differentiated at a glance.


Built autonomously by Aeon

…panion to npm

Closes the Python-package-registry gap left after npm shipped May 12.
npm covers the JavaScript ecosystem; PyPI covers the Python ecosystem —
together they're the two registries that the AI / ML research crowd
already lives in. Both keyless, both polite to anonymous polling.

Three modes:
  - `updates` (default) — PyPI's public RSS feed of the most-recent
    package releases (~40 entries, time-ordered with version bump info).
  - `new-packages` — RSS feed of newly registered packages, surfacing
    brand-new projects.
  - `top-30d` — top 8000 packages ranked by 30-day downloads, via the
    community-maintained hugovk/top-pypi-packages mirror (the only
    "trending" surface PyPI itself doesn't expose).

Each row optionally enriched with last-week downloads from
pypistats.org (keyless). Optional keyword filter applied client-side —
substring match on title+description for RSS modes, name-only for the
rank feed (no description to search).

Three integration quirks: (1) PyPI's updates feed wraps title as
`{name} {version}` (space-separated) while new-packages uses just
`{name}` — parser handles both; (2) the rank-feed's createdAt is the
mirror's last_update, which is misleading as relative-time, so the
renderer suppresses the time pill on rank rows; (3) pypistats.org
weekly-downloads failures degrade silently to 0 so a single dead row
doesn't kill the page.

Plugin (3 files) + 3 registry edits + fetchPypiPage in
lib/integrations/pypi.ts. #3776AB Python brand blue, Package2 icon
(distinct from npm's Package). README: News & web cluster 8 → 9,
column count 41 → 42.
@aaronjmars
aaronjmars merged commit 497242c into main May 14, 2026
1 check passed
@aaronjmars
aaronjmars deleted the feat/pypi-column branch May 14, 2026 12:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant