feat(extract): live-URL token extraction (@tokenscout/extract)#10
Closed
Atroci wants to merge 2 commits into
Closed
feat(extract): live-URL token extraction (@tokenscout/extract)#10Atroci wants to merge 2 commits into
Atroci wants to merge 2 commits into
Conversation
Ship the headline capability: point at a URL, get design tokens.
Workspace split (so the core stays zero-dependency):
- Move the published core to packages/core (name stays "tokenscout",
no consumer breakage); root becomes a private npm workspace with a
shared tsconfig.base.json and dev tooling.
New @tokenscout/extract package (Playwright as a peer dependency):
- extractSite(url, { breakpoints, top }) drives headless Chromium, reads
computed styles (color/background/border, font-size, margin/padding/gap)
at each breakpoint with optional same-origin crawling, and returns
PageExtract[].
- extractTokens() chains that into the core's assembleTokens for a one-call
URL -> DTCG flow.
- Aggregation is a pure harvest() function (unit-tested); the browser-side
collector is kept thin. A smoke test runs real Chromium against a local
fixture and asserts the end-to-end path.
CI/release:
- ci.yml split into a fast core job and a Chromium-backed extract job.
- release.yml publishes the core from packages/core with provenance.
Docs updated to match: README (live-URL usage, install, roadmap, limits),
ARCHITECTURE (real two-package layout + CI), ROADMAP (Phase 3 in progress),
CHANGELOG.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bring README.pt-BR up to parity with the rewritten English README (live-URL usage, two-package install, results, use cases, limitations). Fix the English README example links, which broke when examples/ moved to packages/core/examples/. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ships the headline capability: point tokenscout at a URL and get design tokens. Stacked on #9 (base will retarget to
mainonce #9 merges).What
packages/core(name staystokenscout, no consumer breakage); root is now a private npm workspace with a sharedtsconfig.base.json.@tokenscout/extractpackage, Playwright as a peer dependency:extractSite(url, { breakpoints, top })drives headless Chromium, reads computed styles (color / background / border, font-size, margin/padding/gap) at each breakpoint with optional same-origin crawling, returnsPageExtract[].extractTokens(url, opts)chains that into the core'sassembleTokensfor a one-call URL → DTCG flow.harvest(); the browser-side collector stays thin. A real-Chromium smoke test runs against a local fixture and asserts the end-to-end path.corejob and a Chromium-backedextractjob. Release: publishes the core frompackages/corewith provenance.Verification
npm run build,typecheck,lint,format:check: clean.Usage
Not in this PR (still roadmap)
Image/asset harvesting and animation capture. Sitemap-driven discovery and tech-stack profiling.
🤖 Generated with Claude Code