From 1e4925c593515d21f50cf54a5355fe5684e0d45c Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 12 Jul 2026 16:55:43 +0000 Subject: [PATCH 1/3] Work through SPEC.md's full next-steps list: schema, CONTRIBUTING, error-report pipeline, disclaimer, and re-checked city research Design layer (schema/): - common-schema.md: normalized per-feature shape, two required timestamps (data_as_of vs source.last_synced), untyped geometry, six-state per-category status enum, and jurisdiction-wide default rules (the Walnut, CA case) as a schema-level concept distinct from block/zone data. - coverage-registry.md: the new data/coverage_registry.json shape (per-category status + population), replacing the old flat status+array shape that couldn't represent a confirmed gap distinctly from "not built yet." - error-report-pipeline.md: the anti-spam design for GitHub-issue-based reports -- no backend, no anonymous auto-post, a real GitHub account to submit through IS the moderation step. Implementation: - Migrated data/coverage_registry.json and the three LA adapter scripts to the new schema (data_as_of set explicitly, not just source.last_synced). - app.js: isCovered() helper for the new per-category registry shape; shared confidenceBadge() across all three category panels (previously permits-only); reportIssueUrl()/reportLink() wiring a "Report a problem" link into every panel; first-run disclaimer modal (localStorage-gated) alongside the persistent footer strip. - .github/ISSUE_TEMPLATE/data-issue.yml + unsigned-rule.yml for the two report types the pipeline needs. - CONTRIBUTING.md: dashboard-tracing method, the common schema, the worked LA adapter example, and how to register a jurisdiction. Research: - Investigated ParkUsher directly (not open source, no API, meters paywalled, uneven coverage) -- doesn't change Seattle/SF prioritization. - Re-checked Chicago's meters gap (held up, with a real-but-unverified private fallback surfaced) and Seattle's sweeping gap (overturned -- real data on a second ArcGIS portal, with a source-confirmed voluntary-enforcement caveat) via the dashboard-tracing method. - Added San Diego as a researched adapter candidate. - Checked municipal-code hosting for LA/NYC/Chicago/Seattle/DC. - Updated SPEC.md's coverage table, ranked candidate list, and Next Steps checklist throughout to reflect all of the above. Known limitation: this session's network policy blocked direct HTTPS to arcgis.com/data.seattle.gov/data.sandiego.gov, so the Seattle and San Diego dashboard-traces are corroborated by search, not confirmed by live query -- flagged explicitly in both the affected research files and SPEC.md. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01YbWncp7hgMyXKDvX4oUcBM --- .github/ISSUE_TEMPLATE/data-issue.yml | 45 ++++++++++++ .github/ISSUE_TEMPLATE/unsigned-rule.yml | 30 ++++++++ CONTRIBUTING.md | 75 +++++++++++++++++++ README.md | 13 ++-- SPEC.md | 91 ++++++++++++++---------- app.js | 86 ++++++++++++++++++---- data/README.md | 2 +- data/coverage_registry.json | 9 ++- index.html | 18 ++++- research/README.md | 10 ++- research/cities/chicago.md | 50 +++++++++++++ research/cities/san-diego.md | 69 ++++++++++++++++++ research/cities/seattle.md | 60 ++++++++++++++++ research/city-hub-scan.md | 6 +- research/municipal-code-hosting.md | 14 ++++ research/national-boundary-layer.md | 2 +- research/parkusher.md | 54 ++++++++++++++ research/states/california.md | 3 +- research/states/illinois.md | 7 ++ research/states/washington.md | 7 ++ schema/common-schema.md | 86 ++++++++++++++++++++++ schema/coverage-registry.md | 48 +++++++++++++ schema/error-report-pipeline.md | 37 ++++++++++ scripts/fetch_la_meters.py | 3 + scripts/fetch_la_sweeping.py | 5 ++ 25 files changed, 762 insertions(+), 68 deletions(-) create mode 100644 .github/ISSUE_TEMPLATE/data-issue.yml create mode 100644 .github/ISSUE_TEMPLATE/unsigned-rule.yml create mode 100644 CONTRIBUTING.md create mode 100644 research/cities/chicago.md create mode 100644 research/cities/san-diego.md create mode 100644 research/cities/seattle.md create mode 100644 research/parkusher.md create mode 100644 research/states/illinois.md create mode 100644 research/states/washington.md create mode 100644 schema/common-schema.md create mode 100644 schema/coverage-registry.md create mode 100644 schema/error-report-pipeline.md diff --git a/.github/ISSUE_TEMPLATE/data-issue.yml b/.github/ISSUE_TEMPLATE/data-issue.yml new file mode 100644 index 0000000..45b1267 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/data-issue.yml @@ -0,0 +1,45 @@ +name: Data issue report +description: Something Chalked shows is wrong — a status, schedule, or zone that doesn't match reality (or the posted sign). +title: "[data-issue] " +labels: ["data-issue"] +body: + - type: markdown + attributes: + value: | + Thanks for flagging this. Every "wrong here" report makes Chalked's data more trustworthy, and — per SPEC.md's framing — a visible backlog of these across cities is itself evidence for why a real national curb-data standard is overdue. + + This label family (`data-issue`, plus `city:*`/`category:*` added during triage) is kept separate from the `epic`/`ready`/`blocked` planning labels used elsewhere in this repo, so data-quality reports don't get lost in build-planning noise. + + If this is about a rule with **no posted sign at all** (a citywide code-only rule, like the Walnut, CA case in `research/municipal-code-hosting.md`), use the **Unsigned rule report** template instead — this one. + - type: input + id: jurisdiction + attributes: + label: Jurisdiction + description: City/county this report is about. + placeholder: "Los Angeles, CA" + validations: + required: true + - type: dropdown + id: category + attributes: + label: Category + options: + - sweeping + - meters + - permits + - crime + - other/not sure + validations: + required: true + - type: textarea + id: observed + attributes: + label: What did you observe? + description: What Chalked showed, what the posted sign (or reality) actually says, and the location (address, cross streets, or lat/lng). + validations: + required: true + - type: textarea + id: evidence + attributes: + label: Photo or link (optional) + description: A photo of the sign, or a link to an official source, if you have one. You can drag-and-drop an image into this box once the issue is created — GitHub hosts it inline. diff --git a/.github/ISSUE_TEMPLATE/unsigned-rule.yml b/.github/ISSUE_TEMPLATE/unsigned-rule.yml new file mode 100644 index 0000000..d8c9ad1 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/unsigned-rule.yml @@ -0,0 +1,30 @@ +name: Unsigned / code-only rule report +description: A real parking rule that exists only in municipal code, with no posted signage anywhere — see the Walnut, CA case in research/municipal-code-hosting.md. +title: "[unsigned-rule] " +labels: ["data-issue", "unsigned-rule"] +body: + - type: markdown + attributes: + value: | + Use this when you know about a real, binding parking rule that **isn't posted anywhere** — Chalked's whole "always defer to the posted sign" safety net doesn't help when there's no sign to defer to. See `research/municipal-code-hosting.md` for why this is its own report type, distinct from a signed-zone data error, and `schema/common-schema.md`'s "jurisdiction-wide default rules" section for how it eventually becomes real data. + + This is exactly the kind of local knowledge that's genuinely hard to find by scraping — Tucker knew about Walnut's rule because he has in-laws there. If you know a city's rule like this, you're the fastest way Chalked finds out about it. + - type: input + id: jurisdiction + attributes: + label: Jurisdiction + placeholder: "Walnut, CA" + validations: + required: true + - type: textarea + id: rule + attributes: + label: What's the rule? + description: Describe it as specifically as you can — who it applies to, when, where it applies (citywide, or a specific area?), and how you know about it (you live there, you got a ticket, you've read the code yourself). + validations: + required: true + - type: input + id: citation + attributes: + label: Municipal code citation (if known) + description: e.g. "Walnut Municipal Code §10.20.040". See research/municipal-code-hosting.md for how to find a city's code online (usually Municode, American Legal, or General Code) if you want to track it down yourself. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..a11adc6 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,75 @@ +# Contributing to Chalked + +Chalked's core engineering work is one adapter per jurisdiction per category — the national map shell, the coverage registry, and the common schema all already exist precisely so a new adapter is the *only* thing a contribution needs to add. This guide is what SPEC.md's Next Steps has been pointing at since the national-shell reframe: "the per-city adapter work becomes the first contributions, proving out the adapter interface... rather than 'the v1 launch.'" + +If you're looking for what to work on, start with SPEC.md's "First adapters to build" ranking, or sort `data/coverage_registry.json` by population × missing categories yourself — see "What to build next" below. + +## Before you write any code: find the data + +Most cities' open-data portals don't advertise everything they actually have. **Read [research/dashboard-tracing-method.md](research/dashboard-tracing-method.md) first** — it's the technique that overturned two of LA's own "this doesn't exist" calls (sweeping and permits both turned out to be real, queryable ArcGIS Feature Services hiding behind citizen-facing dashboards nobody had traced back to source). The short version: + +1. Find the citizen-facing dashboard/map for the category you're after (search ` street sweeping map`, ` parking permit map`, etc.). +2. If it's ArcGIS-based, get the dashboard's item ID from its URL, query `https://www.arcgis.com/sharing/rest/content/items/?f=json` for its Web Map item id, then that Web Map's `/data?f=json` for `operationalLayers[]` — each has a real Feature Service `url` you can query directly. +3. If it's Socrata-based (common for Chicago/NYC/SF-style portals), the API is usually advertised directly on the dataset's own page — check there first. + +**Don't conclude "gap" from a search-engine-level check alone.** Mark it `unconfirmed` in the registry (see below) instead, and leave the dashboard-tracing pass for whoever picks it up next — see `schema/common-schema.md`'s status enum for why `gap` and `unconfirmed` are deliberately different things. + +If what you're after is a rule with **no dataset and no signage at all** (the Walnut, CA case — see [research/municipal-code-hosting.md](research/municipal-code-hosting.md)), that's a different search: check whether the city's municipal code is hosted on Municode, American Legal Publishing, or General Code before assuming you need a bespoke scrape. + +## The common schema + +**Read [schema/common-schema.md](schema/common-schema.md)** before writing an adapter — it's short and defines exactly what your adapter needs to output: a GeoJSON `FeatureCollection`, common fields (`jurisdiction`, `category`, `data_as_of`, `source.{name,url,last_synced}`) on every feature, plus category-specific fields (documented per category in that file, matching what the existing LA adapters already produce). Geometry type is untyped on purpose — use whatever shape your source data actually is (polygon, line, point); don't force it into a different shape to match another city's adapter. + +The two timestamps matter and aren't interchangeable: `source.last_synced` is when *your adapter* last ran; `data_as_of` is how current the *underlying data* actually is, per the source's own claim. For a live-edited source they're the same. For a source like LA's permit data (frozen since 2015 despite a claimed annual refresh), they diverge — and that divergence is exactly the honest signal the UI's confidence badge (`confidenceBadge()` in `app.js`) surfaces to users. Set both correctly; don't just copy `last_synced` into `data_as_of` without checking whether the source is actually current. + +## Worked example: the LA adapters + +`scripts/fetch_la_sweeping.py`, `fetch_la_meters.py`, and `fetch_la_permits.py` are the three real adapters in the repo — read whichever is closest to your target category before writing your own. Shape they all follow: + +1. **Fetch** — pull raw records from the source (ArcGIS `/query` endpoint or Socrata's REST API), no transformation yet. +2. **Transform** — map the source's field names into the common schema's field names (e.g. LA's `Posted_Day` → Chalked's `day_of_week`). Skip and count (don't silently drop) any record missing a field your transform genuinely needs — see `fetch_la_sweeping.py`'s `skipped` counter for the pattern. Set `data_as_of` deliberately, not by default. +3. **Write** — dump the resulting `FeatureCollection` to `data/-.geojson`. This file is generated, not committed (see `.gitignore` and `data/README.md`) — anyone running the site locally regenerates it by re-running your script. + +A contribution PR includes the script (`scripts/fetch__.py`), not the generated `.geojson` output. + +## Registering your adapter + +Once your adapter works, add or update an entry in `data/coverage_registry.json` — **the one hand-maintained file in `data/`**, and the only registration step needed; the map, its blue/gray styling, and the click-fallback logic all read from this file with no other code changes required. See [schema/coverage-registry.md](schema/coverage-registry.md) for the full shape. Minimally: + +```jsonc +{ + "0603526": { // place_id: state FIPS + place FIPS (same as TIGERweb) + "name": "Berkeley", + "state": "CA", + "population": 124321, + "categories": { + "sweeping": "built", // what you just built + "meters": "unconfirmed", // honest default for anything you didn't check + "permits": "unconfirmed", + "crime": "paused" // see ETHICS.md -- don't build this without reading it first + } + } +} +``` + +If you're actively working on a category but haven't merged yet, set it to `in_progress` in a draft PR — this is what that status exists for, so two people don't duplicate the same adapter without knowing it. + +## If you're touching the crime/break-in category + +Don't, yet — it's deliberately paused pending community input, not an oversight. Read `ETHICS.md` and weigh in at [Discussion #1](https://github.com/inkxel/chalked/discussions/1) before writing any code here. + +## If a user reports something wrong before you fix it + +Reports come in through GitHub Issues via `.github/ISSUE_TEMPLATE/data-issue.yml` (a specific data point is wrong) or `unsigned-rule.yml` (a rule exists with no posted sign — see `schema/error-report-pipeline.md` for the full design). If you're triaging one, add `city:` and `category:` labels during review — the templates deliberately don't auto-apply those, since one template covers every jurisdiction. + +## What to build next + +SPEC.md's "First adapters to build" section ranks candidates by population × data completeness, softly weighted (a bigger city with a messier gap can still be worth more than a smaller city with clean data). It's deliberately not a strict queue — anyone can pick up any jurisdiction. If you want to sort for yourself: every entry in `data/coverage_registry.json` carries `population`, and every category not yet `built` is fair game. + +Two research findings worth knowing before you start: +- LA's own "gap" calls for sweeping and permits were both wrong — always dashboard-trace before writing off a category as unavailable. +- Sweeping isn't a universal primary category — Sunbelt cities show little to no sweeping enforcement (see `research/city-hub-scan.md`), so a Sunbelt adapter's natural lead category may be meters or permits instead. Mark sweeping `not_applicable` there, not `gap` — see `schema/common-schema.md`'s status enum for why that distinction matters. + +## Style + +This repo's markdown files write in direct, confident sentences backed by real sources (URLs, dataset IDs, citations) — not hedged filler. Match that. Code comments explain *why*, not *what* — see any existing file in `scripts/` or `app.js` for the tone. diff --git a/README.md b/README.md index 0e055d8..5e414f4 100644 --- a/README.md +++ b/README.md @@ -32,9 +32,10 @@ This is early — spec-stage, no app yet — which means right now the highest-l - **Weigh in on the crime-risk question** — [Discussion #1](https://github.com/inkxel/chalked/discussions/1). This is a live, genuinely undecided ethical call, not a rhetorical question — opinions actually change what gets built. - **Don't see your city below? Open an issue for it.** One issue per jurisdiction is how this gets tracked — see the five already open (Los Angeles, NYC, Chicago, Seattle, DC) for the format. If you know your city's open-data landscape, or even just that it *has* one, that's worth a comment even before anyone commits to building an adapter. -- **Know a "hidden" parking rule with no signage?** (Like Walnut, CA's citywide overnight-permit requirement — no signs anywhere, you're just expected to know.) Say so on the relevant jurisdiction issue, or open a new one. These are exactly the rules a tool like this can't find on its own — see SPEC.md → "Beyond open data." +- **Know a "hidden" parking rule with no signage?** (Like Walnut, CA's citywide overnight-permit requirement — no signs anywhere, you're just expected to know.) [Report it](https://github.com/inkxel/chalked/issues/new?template=unsigned-rule.yml) — these are exactly the rules a tool like this can't find on its own — see SPEC.md → "Beyond open data." +- **See something wrong on the live site?** Click a zone/meter/permit and use its "Report a problem →" link, or [file a data-issue report](https://github.com/inkxel/chalked/issues/new?template=data-issue.yml) directly — see `schema/error-report-pipeline.md` for how this works and why it doesn't post anonymously. - **Help fill in the Coverage list below** — even "I checked and City X has no open data at all" is useful, it stops the same ground getting re-covered. -- **Build an adapter** — the architecture and the per-city data landscape are in [SPEC.md](SPEC.md); a CONTRIBUTING guide with a full worked example is coming once the first real adapter (likely Los Angeles) proves the pattern. Until then, the jurisdiction issues are the place to coordinate. +- **Build an adapter** — see [CONTRIBUTING.md](CONTRIBUTING.md) for the full guide: the dashboard-tracing method for finding data cities don't advertise, the [common schema](schema/common-schema.md) your adapter's output needs to match, and a worked example (the three real LA adapters in `scripts/`). The jurisdiction issues are still the place to coordinate on who's building what. ## Coverage @@ -60,7 +61,7 @@ A broad discovery scan (2026-07-06, see [research/city-hub-scan.md](research/cit - **Los Angeles** — Sweeping: ✅ [LADOT Posted Street Sweeping Routes](https://services1.arcgis.com/PTh9WC0Sf2WS7AAq/arcgis/rest/services/Posted_Street_Sweeping_Routes_Update/FeatureServer/0), actively edited. Meters: ✅ [LADOT Metered Parking Inventory & Policies](https://data.lacity.org/Transportation/LADOT-Metered-Parking-Inventory-Policies/s49e-q6j2) — best-documented meter dataset of any city checked. Permits: ⚠️ [Preferential Parking Districts](https://data.lacity.org/resource/s3st-6nwi.json), real but not updated since 2015-08-13. Crime: ✅ [Crime Data 2020–Present](https://data.lacity.org/Public-Safety/Crime-Data-from-2020-to-Present/2nrs-mtv8). → [Issue #2](https://github.com/inkxel/chalked/issues/2) - San Francisco — Sweeping: ✅ [DataSF `yhqp-riqs`](https://data.sfgov.org/resource/yhqp-riqs.json). Meters: ✅ [DataSF `8vzz-qzz9`](https://data.sfgov.org/resource/8vzz-qzz9.json). Permits/RPP: ⚠️ [DataSF `hi6h-neyh`](https://data.sfgov.org/resource/hi6h-neyh.json) — SFMTA's own 2017 set, flagged by the city as not comprehensively updated. Crime: ✅ [DataSF `ab4h-6ztd`](https://data.sfgov.org/resource/ab4h-6ztd.json). **Already excellently covered by [CURB](https://curb.guide)** — no Chalked adapter needed. - Glendale, Pasadena, Santa Monica, West Hollywood — Sweeping: 🔍 via [sweep.la](https://sweep.la), not independently sourced. Meters/Permits/Crime: not yet researched. -- **San Diego** *(new candidate, found via broad scan)* — Sweeping: ✅ confirmed real ArcGIS map + open dataset at [data.sandiego.gov](https://data.sandiego.gov/datasets/street-sweeping-schedule/) — best-documented open data found in that scan's West-region batch. Meters/Permits/Crime: not yet researched. +- **San Diego** *(new candidate, ~1.4M population — ranks ~3rd-4th on the adapter priority list)* — Meters: ✅ [Parking Meters Locations](https://data.sandiego.gov/datasets/parking-meters-locations/), daily-updated, ~5,300 spaces, one vendor (IPS Group "Park Smarter"). Sweeping: ⚠️ real dataset + a live ArcGIS/GeoSweep backend confirmed to exist, but the Feature Service trace isn't finished yet — ~90% of swept blocks have no posted sign (voluntary, not ticketed). Permits: likely a gap — no open dataset found, one untraced ArcGIS viewer. Crime: ✅ [Police NIBRS Crime Offenses](https://data.sandiego.gov/datasets/police-nibrs/). No independent aggregator found. See [research/cities/san-diego.md](research/cities/san-diego.md). **Colorado** — Denver — hub found (denvergov.org), vendor **ParkMobile**; sweeping page discoverable, city pushes a native app (Pocketgov) for sweeping alerts. No independent aggregator. Datasets not yet independently confirmed. @@ -76,7 +77,7 @@ A broad discovery scan (2026-07-06, see [research/city-hub-scan.md](research/cit **Idaho** — Boise — not yet researched. -**Illinois** — **Chicago** — Sweeping: ✅ [Street Sweeping Schedule 2025](https://data.cityofchicago.org/Sanitation/Street-Sweeping-Schedule-2025/a2xx-z2ja). Permits: ✅ [Permit Parking Zones](https://data.cityofchicago.org/Transportation/Permit-Parking-Zones/qiag-khha). Meters: ❌ private concessionaire, no open API (unofficial scraper: [stevevance/Chicago-Parking-Meters](https://github.com/stevevance/Chicago-Parking-Meters)). Crime: ✅ [Motor Vehicle Theft](https://data.cityofchicago.org/Public-Safety/motor-vehicle-theft/7ac4-d9tk). Independent aggregator: **[We The Sweeple](https://wethesweeple.com)**, confirmed open source ([github.com/srobbin/sweeparoundus](https://github.com/srobbin/sweeparoundus)) — sweeping alerts/calendar only, not comprehensive. → [Issue #4](https://github.com/inkxel/chalked/issues/4) +**Illinois** — **Chicago** — Sweeping: ✅ [Street Sweeping Schedule – 2026](https://data.cityofchicago.org/Sanitation/Street-Sweeping-Schedule-2026/u5ai-3efk), confirmed re-checked, current-year. Permits: ✅ [Parking Permit Zones](https://data.cityofchicago.org/Transportation/Parking-Permit-Zones/u9xt-hiju), updated as recently as 2026-04-30 — use this ID, not the older `qiag-khha`/`az5k-c8i5` duplicates. Meters: ❌ confirmed no open government dataset exists (checked directly, not assumed) — the concessionaire's own consumer map has a real backing API previously scraped ([stevevance/Chicago-Parking-Meters](https://github.com/stevevance/Chicago-Parking-Meters)), but that project's dead since 2019 and the endpoint's live status is unverified; Chicago Parking Meters LLC is also mid-sale as of 2026. Crime: ✅ [Crimes – 2001 to Present](https://data.cityofchicago.org/Public-Safety/Crimes-2001-to-Present/ijzp-q8t2). Independent aggregator: **[We The Sweeple](https://wethesweeple.com)**, confirmed open source ([github.com/srobbin/sweeparoundus](https://github.com/srobbin/sweeparoundus)) — sweeping alerts/calendar only, not comprehensive. See [research/cities/chicago.md](research/cities/chicago.md). → [Issue #4](https://github.com/inkxel/chalked/issues/4) **Indiana** — Indianapolis — decent hub (parkindy.info) supporting both **ParkMobile and Flowbird**. Sweeping entirely separate, not linked. No independent aggregator. Datasets not yet independently confirmed. @@ -144,7 +145,7 @@ A broad discovery scan (2026-07-06, see [research/city-hub-scan.md](research/cit **Virginia** — Virginia Beach — not yet researched. -**Washington** — **Seattle** — Permits/RPZ: ✅ [Restricted Parking Zones](https://data-seattlecitygis.opendata.arcgis.com/datasets/SeattleCityGIS::restricted-parking-zones). Crime: ✅ [SPD Crime Data 2008–Present](https://data.seattle.gov/Public-Safety/SPD-Crime-Data-2008-Present/tazs-3rd5). Meters: 🔍 described as unusually rich in research, exact link unconfirmed. Sweeping: not yet confirmed open — worth a dashboard-tracing check before assuming absent. → [Issue #5](https://github.com/inkxel/chalked/issues/5) +**Washington** — **Seattle** — Sweeping: ✅ "unconfirmed" overturned — real SPU-owned route data on Seattle's separate ArcGIS portal ([Seattle GeoData](https://data-seattlecitygis.opendata.arcgis.com/datasets/SeattleCityGIS::street-sweeping-routes/about), item `d6cb9de98ee840fb8b6504f0b2e234f5`) — **but Seattle doesn't ticket for missed sweeping (confirmed via SDOT/SPU's own page)**, so this should show as "real schedule, low real-world stakes," not full-confidence restricted parking. Permits/RPZ: ✅ [Restricted Parking Zones](https://data.seattle.gov/dataset/Restricted-Parking-Zones/netm-8y6z), claimed daily refresh. Meters: ✅ [Paid Parking Occupancy (Last 30 Days)](https://data.seattle.gov/Transportation/Paid-Parking-Occupancy-Last-30-Days-/rke9-rsvs) + historical data back to 2012 — richer than originally described, not just "rich." Crime: ✅ [SPD Crime Data 2008–Present](https://data.seattle.gov/Public-Safety/SPD-Crime-Data-2008-Present/tazs-3rd5). See [research/cities/seattle.md](research/cities/seattle.md). → [Issue #5](https://github.com/inkxel/chalked/issues/5) **West Virginia** — Charleston — not yet researched. @@ -162,7 +163,7 @@ The sharper version of this hit closer to home. We live in new construction, and ## Disclaimer -This is informational only. It is not responsible for citations, towing, or any consequence of relying on it. **Always defer to posted physical signage over anything this shows.** The data here is aggregated from public sources and can be incomplete, outdated, or wrong — see SPEC.md for known staleness issues in specific datasets. If you see something wrong, please report it (see SPEC.md for the planned error-reporting flow) — but don't park on the strength of an app over what the sign in front of you says. +This is informational only. It is not responsible for citations, towing, or any consequence of relying on it. **Always defer to posted physical signage over anything this shows.** The data here is aggregated from public sources and can be incomplete, outdated, or wrong — see SPEC.md for known staleness issues in specific datasets, and the app's own per-category "data as of" badge, which surfaces staleness directly rather than showing old data with the same confidence as fresh data. If you see something wrong, [report it](https://github.com/inkxel/chalked/issues/new?template=data-issue.yml) — see `schema/error-report-pipeline.md` for how reports get triaged — but don't park on the strength of an app over what the sign in front of you says. ## License MIT (see [LICENSE](LICENSE)). diff --git a/SPEC.md b/SPEC.md index 90726e0..acf2b6a 100644 --- a/SPEC.md +++ b/SPEC.md @@ -1,18 +1,22 @@ # Chalked — Spec -## Current status (2026-07-06) — read this first +## Current status (2026-07-12) — read this first -**A real, working, browser-verified v0 slice exists.** Not just spec — actual code, at the repo root (`index.html`, `app.js`, `scripts/`, `data/`). +**A real, working, browser-verified v0 slice exists, now with a formal schema layer, a contribution guide, and a working error-report pipeline on top of it.** Not just spec — actual code, at the repo root (`index.html`, `app.js`, `scripts/`, `data/`, `schema/`, `.github/ISSUE_TEMPLATE/`) plus `CONTRIBUTING.md`. **What works right now:** - National coverage map: all 19,731 US incorporated places, LA shown supported, everywhere else honestly gray, click anywhere for a real jurisdiction-specific answer. See [research/national-boundary-layer.md](research/national-boundary-layer.md). -- LA fully built: sweeping (864 zones, live green/amber/red status), meters (34,943 points, clustered), permits (155 districts, staleness-disclosed). Crime is deliberately paused — see [ETHICS.md](ETHICS.md) and [Discussion #1](https://github.com/inkxel/chalked/discussions/1). +- LA fully built: sweeping (864 zones, live green/amber/red status), meters (34,943 points, clustered), permits (155 districts, staleness-disclosed). Every category now shows a shared "data as of" confidence badge (`confidenceBadge()` in `app.js`), not just permits. Crime is deliberately paused — see [ETHICS.md](ETHICS.md) and [Discussion #1](https://github.com/inkxel/chalked/discussions/1). - Address search (Nominatim, US-constrained) and a `?debugTime=` override for testing status coloring without waiting for a real weekday morning. -- Everything above was verified in an actual browser via the Playwright CLI skill, not just logic-tested — see commit history for specifics on what broke and what was fixed (a CORS geocoder failure, a ZIP-ambiguity bug, a coordinate-system testing gotcha documented in `app.js`). +- A first-run disclaimer notice (localStorage-gated, plus the persistent footer strip) and a "Report a problem →" link on every category panel, building a prefilled GitHub issue via `.github/ISSUE_TEMPLATE/data-issue.yml` — no backend, no anonymous auto-post; see [schema/error-report-pipeline.md](schema/error-report-pipeline.md) for why that's the actual anti-spam design, not a placeholder for one. A second template, `unsigned-rule.yml`, covers the Walnut-style "no sign at all" report type. +- The common schema (`schema/common-schema.md`) and coverage registry schema (`schema/coverage-registry.md`) are now written down formally — six-state per-category status (`built`/`in_progress`/`gap`/`not_applicable`/`unconfirmed`/`paused`), two required timestamps per feature (`data_as_of` vs. `source.last_synced`), geometry-shape tolerance, and jurisdiction-wide default rules (the Walnut, CA case) as a schema-level concept distinct from block/zone data. `data/coverage_registry.json` and the three LA adapters were migrated to match. +- `CONTRIBUTING.md` — the dashboard-tracing method, the common schema, the worked LA-adapter example, and how to register a new jurisdiction — is now real, not "coming once the first adapter proves the pattern" (it already has). +- Re-checked Chicago and Seattle's flagged gaps directly (dashboard-tracing method, same rigor LA's own reversals got) — see [research/cities/chicago.md](research/cities/chicago.md) and [research/cities/seattle.md](research/cities/seattle.md); investigated ParkUsher directly ([research/parkusher.md](research/parkusher.md)); added San Diego as a researched candidate ([research/cities/san-diego.md](research/cities/san-diego.md)); checked municipal-code hosting for LA/NYC/Chicago/Seattle/DC ([research/municipal-code-hosting.md](research/municipal-code-hosting.md)). Summarized in the coverage table and "First adapters to build" below. +- Everything above was verified in an actual browser via the Playwright CLI skill where the sandbox's network policy allowed it, not just logic-tested — see commit history for specifics on what broke and what was fixed (a CORS geocoder failure, a ZIP-ambiguity bug, a coordinate-system testing gotcha documented in `app.js`). **Honest limitation on this round specifically:** the environment this round of research and UI verification ran in blocks direct HTTPS to `arcgis.com`, `data.lacity.org`, `data.seattle.gov`, and similar hosts at the network-policy level (not a site-side block — confirmed via the proxy's own status log) — real map-layer rendering against live LA/Seattle/San Diego/Chicago data, and the ArcGIS dashboard-traces for Seattle sweeping and San Diego sweeping/permits, need a re-run from an environment that can actually reach those hosts before being fully load-bearing. The disclaimer modal, confidence-badge date math, and report-link URL construction were verified directly (via synthetic fixtures + Playwright, and isolated logic checks) since those don't depend on the blocked hosts. **To pick this up cold:** run the four `scripts/fetch_*.py` adapters (see README → "Running the v0 slice locally"), serve with `python3 -m http.server`, open it, and look. Then read "Next steps" at the bottom of this file — it's kept current, not aspirational. -**Not built yet, most-likely-next first:** a second city's adapter (to prove the pattern generalizes beyond LA), the error-report → GitHub Issue pipeline, the ParkUsher investigation, the disclaimer UI, the CONTRIBUTING guide. +**Not built yet, most-likely-next first:** a second city's adapter (LA's still the only one built — Chicago/Seattle/San Diego are all now researched candidates, not adapters), live-verifying the Seattle/San Diego dashboard traces this round couldn't finish, and the jurisdiction-wide default rules concept has a schema but zero real data behind it yet (Walnut's rule is still only documented in prose). --- @@ -34,11 +38,12 @@ All confirmed via each city's own open-data portal (Socrata or ArcGIS Open Data | City | Sweeping | Meters | Permit zones | Crime data | Notes | |---|---|---|---|---|---| | **San Francisco** | ✅ open | ✅ open (+ rate schedules) | ✅ open (RPP eligibility parcels) | ✅ open | Only city confirmed 4-for-4 — the obvious first launch city | -| **Chicago** | ✅ open | ❌ gap | ✅ open | ✅ open | Meters run by a private concessionaire (Chicago Parking Meters LLC), no open API — historically scraped | -| **NYC** | Folded into regs | ✅ open (richest reg + meter data of any city) | ❌ no real permit-zone model | ✅ open | Uses alternate-side-parking instead of a sweeping program; no citywide RPP like SF/DC/Chicago | +| **Chicago** | ✅ open | ❌ gap, re-confirmed 2026-07-12 | ✅ open | ✅ open | Meters: checked directly (not assumed) — no open government dataset exists. Not a dead end though: the private concessionaire's own consumer map has a real backing API, previously scraped by a now-dead (2019) community project — unverified-live, and Chicago Parking Meters LLC is mid-sale as of 2026. See [research/cities/chicago.md](research/cities/chicago.md). | +| **NYC** | Folded into regs | ✅ open (richest reg + meter data of any city) | ❌ no real permit-zone model | ✅ open | Uses alternate-side-parking instead of a sweeping program; no citywide RPP like SF/DC/Chicago. Municipal code lives on American Legal Publishing, split across the Admin Code and NYC Rules (Title 34 RCNY) — the operative parking rules are in Rules, not the more obvious-sounding Admin Code, see [research/municipal-code-hosting.md](research/municipal-code-hosting.md). | | **Los Angeles** | ✅ open — corrected 2026-07-05 | ✅ open (best-documented meter dataset anywhere) | ⚠️ open but stale — corrected 2026-07-05 | ✅ open | Sweeping: real, unauthenticated ArcGIS Feature Service, official LA Bureau of Street Services, actively edited — see correction below. Permits: real dataset (`LADOT_PPD`, Socrata `s3st-6nwi`), polygon geometry + district number/name — but **not updated since 2015-08-13** despite a claimed "annual" refresh commitment. Real data, real staleness risk — see correction below. | -| **Seattle** | ❌ unconfirmed | ✅ open (unusually rich — historical paid-occupancy by block-minute since 2012) | ✅ open (RPZ) | ✅ open | | -| **Washington DC** | N/A — different regime | ⚠️ unconfirmed | ✅ open | ✅ open | No sweeping program at all — snow-emergency/leaf-season rules instead | +| **Seattle** | ✅ open, "unconfirmed" overturned 2026-07-12 | ✅ open (unusually rich — historical paid-occupancy by block-minute since 2012, confirmed richer than originally described) | ✅ open (RPZ, claimed daily refresh) | ✅ open | Sweeping data lives on Seattle's *second* open-data portal (Seattle GeoData/ArcGIS, owned by Seattle Public Utilities, not SDOT) — real, but **the city doesn't ticket for missed sweeping** (confirmed via SPU's own page), so it shouldn't be shown with LA/SF's full restricted-parking confidence. Live Feature Service trace still needs finishing from an unblocked environment. See [research/cities/seattle.md](research/cities/seattle.md). | +| **San Diego** | ⚠️ real dataset + live GIS backend confirmed, dashboard trace unfinished | ✅ open (daily-updated, ~5,300 spaces, one clear vendor — IPS Group) | ❌ likely gap, not resolved | ✅ open | Added as a researched candidate 2026-07-12, ~1.4M population. `data.sandiego.gov` runs on JKAN (static file catalog, not a live query API like Socrata) — "queryable" needs that caveat. ~90% of San Diego's swept blocks have no posted sign (voluntary, unticketed), same "real but unenforced" honesty problem as Seattle's sweeping. See [research/cities/san-diego.md](research/cities/san-diego.md). | +| **Washington DC** | N/A — different regime | ⚠️ unconfirmed | ✅ open | ✅ open | No sweeping program at all — snow-emergency/leaf-season rules instead. Parking rules are DC's own regulations (DCMR), self-published rather than farmed out to Municode/American Legal — DCMR §18-2405 is literally titled "No Sign Required," a useful section-name pattern to grep for elsewhere. See [research/municipal-code-hosting.md](research/municipal-code-hosting.md). | | **Boston** | ⚠️ unconfirmed | ✅ open | ⚠️ unconfirmed | — | | **No aggregator exists today that indexes multiple cities' open parking data in one place.** That gap — not any single data source — is the actual new value here. @@ -52,9 +57,11 @@ OSM has real tagging conventions (`parking:lane=*`, `parking:condition=*`), but ## Prior art - **Commercial space is split by function, not overlapping this idea directly:** ParkMobile/PayByPhone = meter payment only. SpotHero/ParkWhiz/BestParking = garage/lot booking, not street parking. - **SpotAngels is the real closest multi-city analog** — sweeping + meters + permit zones + crowdsourced rules across ~200 cities — but proprietary, closed-data, and doesn't do break-in risk at all. +- **[ParkUsher](research/parkusher.md)** — the closest *comprehensive* multi-city analog found (Boston, NYC, Montreal, Toronto, Vancouver, Seattle, SF), investigated directly 2026-07-12: not open source, no public API, data sourcing looks substantially manual/crowdsourced rather than an official open-data pipeline, meters paywalled behind a subscription, coverage confirmed uneven even within its own 7 cities (Vancouver's rollout is explicitly partial). A tiny (3-6 person), effectively unfunded pre-seed startup. Doesn't deprioritize Seattle or SF from Chalked's candidate list the way CURB deprioritizes SF — it confirms the open, comprehensive, multi-city gap is still real. - **The actual gap:** nobody combines regulation data with a crime-risk overlay, and no multi-city aggregator is open source. - **[San Francisco](research/cities/san-francisco.md)** — already excellently solved by CURB (curb.guide), open source. Deliberately not a Chalked launch candidate as a result — see First adapters to build. -- **[Los Angeles](research/cities/los-angeles.md)** — sweeping and permits both real and queryable, found via the **[dashboard-tracing method](research/dashboard-tracing-method.md)** (worth applying to Chicago/Seattle's remaining gaps before trusting them); permits carries a real staleness caveat (not updated since 2015). Includes sweep.la regional context (LA + 4 neighboring cities, aggregated, not confirmed open source). +- **[Los Angeles](research/cities/los-angeles.md)** — sweeping and permits both real and queryable, found via the **[dashboard-tracing method](research/dashboard-tracing-method.md)**. That same method was applied to [Chicago](research/cities/chicago.md) and [Seattle](research/cities/seattle.md)'s flagged gaps (2026-07-12): Seattle's sweeping "unconfirmed" call was overturned (real data, just on a second ArcGIS portal SDOT doesn't own); Chicago's meters "gap" held up on the government-data side but surfaced a real, unverified private-API fallback. LA permits carries a real staleness caveat (not updated since 2015). Includes sweep.la regional context (LA + 4 neighboring cities, aggregated, not confirmed open source). +- **[San Diego](research/cities/san-diego.md)** — added as a researched candidate 2026-07-12 via the broad city-hub scan. No independent aggregator found (one abandoned civic-tech SMS tool, `sd-sweepaware`, doesn't count). Strong on meters, promising-but-unfinished on sweeping, likely gapped on permits. Full findings, sources, and the reusable method live in `research/` — this section is the summary. @@ -80,8 +87,8 @@ Tucker's proposal, and it holds up: instead of picking cities to launch in seque Given the "every city is its own integration" finding plus the national-shell reframe above, the shape is: **a national map + coverage registry that exists independent of data, with pluggable per-jurisdiction adapters behind a common schema** — CurbLR's original intent, DIY'd, wrapped in a Zillow/Redfin-style coverage map instead of waiting on city-side adoption of a standard that stalled. - **National base layer** — US Census TIGER/Line (or Cartographic Boundary) files for every incorporated place + county. Solved, uniform, complete, no per-city work required. This is what makes the map itself national on day one. -- **Coverage registry** — per jurisdiction: `supported` / `unsupported` (/ `in-progress`), plus which categories (sweeping/meters/permits/crime) are actually built for it. Partial coverage is the normal case, not an edge case — the registry needs to represent "Chicago minus meters," not just a binary yes/no. -- **Common schema** — one normalized representation for sweeping schedule, meter rules (including free periods), permit-zone status, and crime-risk overlay, with every field nullable per category, plus a **"data as of" timestamp per category** (not just per adapter) — LA's permit data is real but frozen since 2015, and the schema needs to carry that honestly rather than presenting stale data with the same confidence as fresh data. +- **Coverage registry** — **now formally specced and built**, see [schema/coverage-registry.md](schema/coverage-registry.md): per jurisdiction, per category, one of six statuses (`built`/`in_progress`/`gap`/`not_applicable`/`unconfirmed`/`paused`), plus population for the contribution-priority ranking. Partial coverage is the normal case, not an edge case — the registry represents "Chicago minus meters" directly, not as a binary yes/no. +- **Common schema** — **now formally specced**, see [schema/common-schema.md](schema/common-schema.md): one normalized GeoJSON-based representation for sweeping schedule, meter rules (including free periods), permit-zone status, and (once unpaused) a crime-risk overlay, with every field nullable per category, plus **two required timestamps per feature** — `data_as_of` (the underlying data's real vintage) and `source.last_synced` (when Chalked's own adapter last ran) — because those two diverge exactly when it matters most: LA's permit data syncs successfully every run but its `data_as_of` has stayed frozen at 2015 the whole time, and collapsing to one timestamp would hide that. - **Per-jurisdiction adapter** — one ingestion module per city/county, each mapping that place's actual open-data quirks (Socrata vs. ArcGIS, different field names, different update cadences — including tracing hidden ArcGIS dashboards back to their feature services, per the LA finding) into the common schema. This is most of the real engineering work, ongoing maintenance — portals change schemas without warning — and, going forward, the thing community contributors do. - **Crime-risk layer** — separate from the regulation data proper; filters each jurisdiction's broad crime-code taxonomy down to vehicle-break-in-relevant categories, then aggregates to a density/risk signal (not raw incident pins — avoids the product feeling like a crime map). - **Lookup** — address/pin in, resolve jurisdiction via the national base layer, check the coverage registry, either return the combined read (supported) or show the honest "not yet covered — help us add it" state linking to the contribution guide (unsupported). @@ -105,24 +112,24 @@ Visual inspiration: **onX Offroad** — color-coded trails by difficulty, plus B ## Trust, error reporting & disclaimer -Three related pieces, all Tucker's additions (2026-07-05), all load-bearing for a project that tells people where they can legally park. +Three related pieces, all Tucker's additions (2026-07-05), all load-bearing for a project that tells people where they can legally park. **All three are now built (2026-07-12), not just designed.** ### Per-category confidence, shown to the user — not just in the schema -The "data as of" timestamp (above) isn't only a backend field — it needs to surface as an actual visible confidence signal per category, e.g. "Sweeping: current" vs. "Permits: data from 2015, may be outdated." Silent staleness is worse than no data at all, because it reads as confident when it isn't. LA's permit dataset is the concrete case that forced this: real data, but a decade stale, and a user needs to see that distinction before trusting it over a posted sign. +**Built.** `confidenceBadge()` in `app.js`, shared across all three category panels (sweeping/meters/permits — not just permits, which had a bespoke version before this round): "Data current as of ⟨date⟩" vs. "Data as of ⟨date⟩ — may be outdated" past a 180-day threshold, driven by the `data_as_of` field every adapter now sets (see [schema/common-schema.md](schema/common-schema.md)). LA's permit dataset is still the concrete case that forced this — real data, but a decade stale — but the badge is generic; a future stale sweeping/meters source gets the same honest treatment automatically. ### User-reported errors, Google-Maps-style, auto-routed to GitHub Issues -When Chalked shows something wrong — says a block is clear when the posted sign says otherwise — a user should be able to flag it inline, the way Google Maps lets you report a map error. The report (location, category, what they observed, ideally a photo of the sign) should auto-file as a GitHub issue on this repo, labeled by jurisdiction + category (e.g. `data-issue`, `city:los-angeles`, `category:permits` — a distinct label family from the `epic`/`ready`/`blocked` planning labels, so data-quality reports and build-planning issues don't collide in the tracker) so maintainers can triage per city. This turns user friction directly into a public, trackable data-quality record — and doubles as evidence for the "this needs a standard" argument, since a visible backlog of "wrong here" reports across cities makes the fragmentation case better than any essay would. +**Built**, via a design that resolves the anti-spam requirement below rather than deferring it — see [schema/error-report-pipeline.md](schema/error-report-pipeline.md) for the full reasoning. Two GitHub issue-form templates (`.github/ISSUE_TEMPLATE/data-issue.yml`, `unsigned-rule.yml`) apply the `data-issue` label (a distinct family from `epic`/`ready`/`blocked`); `city:*`/`category:*` labels get added by a maintainer at triage rather than auto-applied, since one template covers every jurisdiction. Every category panel in `app.js` (`renderPanel`/`renderMeterPanel`/`renderPermitPanel`) now has a "Report a problem →" link built by `reportIssueUrl()`, prefilling jurisdiction/category/title via GitHub's own form-prefill query params. -**Open design requirement, not yet solved:** auto-filing anonymous user input as public GitHub issues is a real spam/abuse vector. Needs at least basic rate-limiting or a lightweight moderation/staging step before reports go public — do not ship the naive "instant direct post" version without one. +**The anti-spam requirement is resolved, not sidestepped:** Chalked has no backend, so there's no anonymous-auto-post path to secure in the first place — the link opens GitHub's own issue-creation UI, and submitting requires a real GitHub account. That's the actual moderation step, not a placeholder for a future one; see the design doc for why this beats shipping a token-bearing serverless proxy at this project's current size. ### Disclaimer — heavy, upfront, non-negotiable -The site is informational only. Not responsible for citations, towing, or any consequence of relying on it. Users must always defer to posted physical signage over anything Chalked shows — the same posture CURB takes ("the posted sign is always the source of truth"), and necessary here for the same reason: this is inferred/aggregated public data, not a legal guarantee. Needs to be prominent (not buried in a footer link) — a first-run notice or persistent banner, not just a ToS page nobody reads. +**Built.** A first-run modal (`#disclaimer-modal-overlay` in `index.html`, wired in `app.js`) shows once per browser — gated by `localStorage`, not per-page-load — plus the persistent footer strip stays for anyone who wants to re-check it later. The site is informational only, not responsible for citations/towing/any consequence of relying on it, and users must always defer to posted physical signage — the same posture CURB takes ("the posted sign is always the source of truth"), necessary here for the same reason: this is inferred/aggregated public data, not a legal guarantee. ## Beyond open data: unsigned, code-only rules -The Walnut, CA case — a citywide overnight-permit rule with **no signage anywhere** — showed that "always defer to the posted sign" doesn't cover every real rule. Full write-up, including how to actually find these without scraping every municipal code in the country up front, in **[research/municipal-code-hosting.md](research/municipal-code-hosting.md)**. +The Walnut, CA case — a citywide overnight-permit rule with **no signage anywhere** — showed that "always defer to the posted sign" doesn't cover every real rule. Full write-up, including how to actually find these without scraping every municipal code in the country up front, in **[research/municipal-code-hosting.md](research/municipal-code-hosting.md)** — now including a platform check (2026-07-12) for LA, NYC, Chicago, Seattle, and DC's municipal codes, three of which (LA, NYC, Chicago) turned out to share the same hosting platform (American Legal Publishing), and one finding worth flagging directly: Chicago's own code (§9-64-090) has a real Walnut-shaped rule — a citywide, unsigned ban on overnight parking of trucks/RVs/commercial vehicles on any residential street. -**Schema implication:** the common schema needs a rule type that isn't tied to a block or zone at all — a **jurisdiction-wide default rule** (e.g. "overnight parking anywhere in city limits requires a permit, no exceptions"), layered *underneath* the block-specific sweeping/meter/permit-zone data most of this spec has been about. A city can have both at once — specific signed zones *and* an invisible blanket default. +**Schema implication — now formally specced, not just noted:** [schema/common-schema.md](schema/common-schema.md)'s "Jurisdiction-wide default rules" section defines a rule type that isn't tied to a block or zone at all (e.g. "overnight parking anywhere in city limits requires a permit, no exceptions"), layered *underneath* the block-specific sweeping/meter/permit-zone data most of this spec has been about, with a `signed: false` field as the load-bearing flag telling the UI this rule needs proactive surfacing rather than "check the posted sign." A city can have both at once — specific signed zones *and* an invisible blanket default. No real jurisdiction has actual data in this shape yet (Walnut's own rule is still only documented in prose) — the schema exists, the first real entry doesn't. The [`unsigned-rule.yml`](.github/ISSUE_TEMPLATE/unsigned-rule.yml) issue template is the intended intake path for exactly this kind of report, per the error-report pipeline below. ## First adapters to build (not "launch cities" — the map covers everywhere from day one) These are the first few jurisdictions worth building real adapters for, to prove the schema and seed the coverage map with real data — not a sequential rollout plan, since every place is already visible on the national map regardless. @@ -134,14 +141,15 @@ Data completeness alone isn't the whole story — a jurisdiction with clean data Convenient synergy: the same Census TIGER/Line source providing the national boundary layer also publishes population estimates for those same places — this ranking factor doesn't need a separate data source. -Ranked by combined open-data coverage among jurisdictions with no comparable *open-source* tool, each with its known gap, population noted (rounded, city proper) since it now matters to the call. **Re-check "unconfirmed" gaps via the [dashboard-tracing method](research/dashboard-tracing-method.md) before trusting them** — LA's sweeping and permit gaps both turned out to be wrong when actually checked. -1. **Los Angeles** (~3.9M) — the strongest combined candidate: real population *and* the most complete data. Sweeping (fresh, actively edited), meters (best-documented dataset of any city), crime (open), permits (open, real, but frozen since 2015 — disclose the staleness, don't hide it). sweep.la already covers LA + 4 neighbors but isn't confirmed open source — a real, open Chalked adapter for LA still has a clear reason to exist. +Ranked by combined open-data coverage among jurisdictions with no comparable *open-source* tool, each with its known gap, population noted (rounded, city proper) since it now matters to the call. **Re-check "unconfirmed" gaps via the [dashboard-tracing method](research/dashboard-tracing-method.md) before trusting them** — LA's sweeping and permit gaps both turned out to be wrong when actually checked, and so (partially) did Seattle's sweeping gap on 2026-07-12. +1. **Los Angeles** (~3.9M) — the strongest combined candidate: real population *and* the most complete data, and the only one actually built. Sweeping (fresh, actively edited), meters (best-documented dataset of any city), crime (open), permits (open, real, but frozen since 2015 — disclosed, not hidden). sweep.la already covers LA + 4 neighbors but isn't confirmed open source — a real, open Chalked adapter for LA still has a clear reason to exist. 2. **NYC** (~8.3M) — by far the largest population of any candidate, and the richest regulation + meter data of any city — but no permit-zone model to speak of, and a genuinely different rule shape (alternate-side parking, not sweeping) that's more architectural work than a drop-in adapter. Worth weighing seriously despite the gap, given the population is more than double LA's — NYC's alternate-side enforcement is also famously aggressive (real, high-volume citation activity), which is exactly the enforcement-intensity signal population is a proxy for. -3. **Chicago** (~2.7M) — strong on sweeping/permits/crime; meters is the one real hole (private concessionaire, no open API — would need a scraper or an explicit "meters unknown here" state). No CURB/sweep.la-equivalent found — genuine open territory, but re-check before committing given the LA misses. -4. **Seattle** (~750K) — strong on meters/permits/crime; sweeping schedule not yet confirmed open — but given LA's misses, worth checking for a hidden ArcGIS dashboard before concluding it's actually absent. -5. **Washington DC** (~670K) — solid permits + crime, but an entirely different seasonal-rule regime (snow/leaf season, no sweeping program) and the smallest population of the group. +3. **Chicago** (~2.7M) — strong on sweeping/permits/crime; meters is the one real hole, **re-checked 2026-07-12 and it held up**: no open government dataset, confirmed by direct portal search, not just an unsearched gap like LA's misses turned out to be. A real fallback exists (the concessionaire's own consumer-map API, previously scraped, now unverified-live since 2019) but carries real legal/maintenance risk a government open-data source wouldn't. No CURB/sweep.la-equivalent found — genuine open territory. +4. **San Diego** (~1.4M) — *added 2026-07-12.* Strong on meters (open, daily-updated, one clear vendor, small enough to skip clustering), open on crime, but sweeping's dashboard trace is unfinished (real backend confirmed, FeatureServer not yet queried live) and permits is likely a gap. No independent aggregator found. Ranks here provisionally — resolve the sweeping trace and the permits question before treating this slot as settled. +5. **Seattle** (~750K) — sweeping's "unconfirmed" call **overturned 2026-07-12**: real SPU-owned route data exists, just on Seattle's second (ArcGIS) open-data portal — but the city doesn't ticket for missed sweeping, a real enforcement-intensity discount against its population-based rank here. Meters and permits both confirmed richer/fresher than originally described (meters: block-minute paid-occupancy data since 2012; permits: claimed daily refresh). +6. **Washington DC** (~670K) — solid permits + crime, but an entirely different seasonal-rule regime (snow/leaf season, no sweeping program) and the smallest population of the group. -**Read:** LA is the clean pick — high population, most complete data. NYC is the real judgment call — the highest population and enforcement-intensity signal by far, worth the extra architectural work the missing permit model and different rule shape demand, rather than defaulting to whichever jurisdiction merely has the tidiest data. +**Read:** LA is the clean pick, and the only one built so far. NYC is the real judgment call — the highest population and enforcement-intensity signal by far, worth the extra architectural work the missing permit model and different rule shape demand, rather than defaulting to whichever jurisdiction merely has the tidiest data. San Diego and Seattle both moved this round — San Diego added new, Seattle's sweeping gap partially overturned but discounted by its voluntary-compliance reality — worth another look once the blocked-network follow-ups (live dashboard traces for both) actually happen. ## Data pipeline: from research to a live, self-updating site @@ -171,23 +179,30 @@ Plain-English version of how everything in `research/` actually becomes somethin 3. [x] ~~Build the thin vertical slice~~ — **done (2026-07-06), now complete.** Real LA sweeping adapter, address/pin lookup, green/amber/red status computed live against real posted schedules — plus (added same day, see below) LA meters, LA permits, and the national boundary/coverage-registry layer. Every piece browser-verified via Playwright, not just logic-tested. See README → "Running the v0 slice locally" for the current honest scope. **From the 29-city broad scan (2026-07-06, see [research/city-hub-scan.md](research/city-hub-scan.md)):** -- [ ] Investigate ParkUsher directly (architecture, open-source status, data sourcing) — the closest prior-art analog to CURB/sweep.la found anywhere, covering Boston/NYC/Seattle/SF and others -- [ ] Consider San Diego as an addition to the first-adapter candidate set — confirmed real, open sweeping dataset, not on the original 5-city list -- [ ] When designing the common schema (below), make sure "this category doesn't meaningfully apply here" (e.g. sweeping in most Sunbelt cities) is representable distinctly from "no data yet" +- [x] ~~Investigate ParkUsher directly (architecture, open-source status, data sourcing)~~ — **done (2026-07-12).** Not open source, no public API, data sourcing looks substantially manual/crowdsourced, meters paywalled, coverage confirmed uneven even within its own 7 cities. Doesn't deprioritize Seattle or SF the way CURB deprioritizes SF. See [research/parkusher.md](research/parkusher.md). +- [x] ~~Consider San Diego as an addition to the first-adapter candidate set~~ — **done (2026-07-12).** Added to the ranked list above (~4th, provisionally) — strong meters, promising-but-unfinished sweeping, likely-gapped permits, no independent aggregator. See [research/cities/san-diego.md](research/cities/san-diego.md). +- [x] ~~When designing the common schema, make sure "this category doesn't meaningfully apply here" is representable distinctly from "no data yet"~~ — **done, see [schema/common-schema.md](schema/common-schema.md)'s status enum:** `not_applicable` vs. `unconfirmed` are two of its six explicit states. **After the slice exists, roughly in this order:** - [x] ~~Verify the v0 slice actually renders and works in a real browser~~ — **done (2026-07-06), via Playwright.** Map, zone coloring, click interactions, and address search all confirmed working against the real page. Found and fixed one real bug in the process: the Census Geocoder doesn't support CORS, so browser-based address search failed outright — swapped to Nominatim, confirmed working. - [x] ~~Build the national boundary/coverage-registry layer~~ — **done (2026-07-06).** All 19,731 US incorporated places (Census TIGERweb, `scripts/fetch_national_places.py`), canvas-rendered, LA highlighted against everywhere else honestly gray. `data/coverage_registry.json` (hand-maintained, not generated — adding a city means adding an entry there) drives which places show as supported. Click-anywhere resolution verified for all three real cases: inside LA proper, inside a separate uncovered city (San Fernando — validates the place-level-not-county-level design decision), and genuinely unincorporated land. Full writeup: [research/national-boundary-layer.md](research/national-boundary-layer.md). **This is the site now actually matching its own README pitch**, not just the single-city demo it was before today. -- [ ] Design the coverage registry schema (per-jurisdiction status + per-category granularity + population, so a contributor-facing "most-needed" view can sort by population × missing coverage) -- [ ] Design the common schema with explicit per-category nullability and a "data as of" timestamp per category — and geometry-shape tolerance (LA's routes are polygons, SF's blocks are line segments; the schema needs to handle both, not assume one) -- [ ] Design the per-category confidence/staleness UI (not just the backend timestamp) +- [x] ~~Design the coverage registry schema (per-jurisdiction status + per-category granularity + population...)~~ — **done (2026-07-12), see [schema/coverage-registry.md](schema/coverage-registry.md).** Six-state per-category status, population field, migrated `data/coverage_registry.json` and `app.js`'s `isCovered()` to match. +- [x] ~~Design the common schema with explicit per-category nullability and a "data as of" timestamp per category — and geometry-shape tolerance~~ — **done (2026-07-12), see [schema/common-schema.md](schema/common-schema.md).** Two required timestamps (`data_as_of` vs. `source.last_synced`), untyped geometry, and category-specific field tables for sweeping/meters/permits matching what the real LA adapters already produce. All three LA adapter scripts updated to set `data_as_of` explicitly, not just `source.last_synced`. +- [x] ~~Design the per-category confidence/staleness UI~~ — **done (2026-07-12).** `confidenceBadge()` in `app.js`, shared across all three category panels — see "Trust, error reporting & disclaimer" above. - [x] ~~Extend to LA's meters~~ — **done (2026-07-06).** `scripts/fetch_la_meters.py`, 34,943 points, clustered rendering (Leaflet.markercluster — the right tool any time a per-point dataset gets into the thousands), browser-verified. Found the dataset has no operating-hours field at all — shown honestly as a rate/limit fact, not implied as a schedule. See [research/cities/los-angeles.md](research/cities/los-angeles.md). - [x] ~~Extend to LA's permits~~ — **done (2026-07-06).** `scripts/fetch_la_permits.py`, 155 districts, dashed-blue eligibility-flag treatment (not the color scale), 2015 staleness surfaced directly in the UI. Browser-verified — including catching a real coordinate-system gotcha in the Playwright test itself (see app.js comment near `window.map`), not an app bug. **LA's first three categories are now all built** (sweeping, meters, permits) — crime remains paused per the ethics discussion. Next: weigh NYC seriously despite its permit-model gap, given its outsized population and enforcement intensity. -- [ ] Re-check Chicago and Seattle's data gaps using the dashboard-tracing method before trusting either "unconfirmed"/gap call -- [ ] Write the CONTRIBUTING guide for adding a jurisdiction (dashboard-tracing method, common schema, worked example) — needed before asking anyone to help -- [ ] Design the error-report → GitHub Issue pipeline, including the anti-spam/moderation step before anything posts publicly -- [ ] Extend the error-report pipeline to accept "my city has an unsigned rule like X" as its own report type, feeding targeted per-city code lookups instead of blind nationwide scraping -- [ ] Check whether each first-adapter city's municipal code is hosted on Municode/American Legal/General Code before assuming a bespoke scrape is needed -- [ ] Write the disclaimer and decide its placement (first-run notice vs. persistent banner) before any public build ships -- [ ] Add jurisdiction-wide default rules as a schema-level concept (distinct from block/zone-specific data) — needed for cases like Walnut, CA's unsigned citywide overnight-permit rule +- [x] ~~Re-check Chicago and Seattle's data gaps using the dashboard-tracing method before trusting either "unconfirmed"/gap call~~ — **done (2026-07-12).** Chicago's meters gap held up (confirmed no open government data), with a real-but-unverified private-API fallback surfaced. Seattle's sweeping gap was overturned (real data, second ArcGIS portal), with a new source-confirmed voluntary-enforcement caveat. See [research/cities/chicago.md](research/cities/chicago.md), [research/cities/seattle.md](research/cities/seattle.md). Live Feature Service verification for Seattle (and San Diego) still needs a re-run from an environment that can reach `arcgis.com` — this round's sandbox couldn't. +- [x] ~~Write the CONTRIBUTING guide for adding a jurisdiction~~ — **done (2026-07-12), see [CONTRIBUTING.md](CONTRIBUTING.md).** Dashboard-tracing method, the common schema, the worked LA-adapter example, and how to register a jurisdiction in the coverage registry. +- [x] ~~Design the error-report → GitHub Issue pipeline, including the anti-spam/moderation step~~ — **done (2026-07-12), see [schema/error-report-pipeline.md](schema/error-report-pipeline.md).** No backend, no anonymous auto-post — a real GitHub account to submit through IS the anti-spam step. `.github/ISSUE_TEMPLATE/data-issue.yml` + prefilled "Report a problem →" links wired into every category panel in `app.js`. +- [x] ~~Extend the error-report pipeline to accept "my city has an unsigned rule like X" as its own report type~~ — **done (2026-07-12).** `.github/ISSUE_TEMPLATE/unsigned-rule.yml`, linked from the site footer (there's no zone geometry to attach a "report a problem" link to for a jurisdiction-wide rule). +- [x] ~~Check whether each first-adapter city's municipal code is hosted on Municode/American Legal/General Code~~ — **done (2026-07-12).** LA/NYC/Chicago on American Legal Publishing, Seattle on Municode, DC self-hosts (DCMR, not farmed out). See the new table in [research/municipal-code-hosting.md](research/municipal-code-hosting.md). +- [x] ~~Write the disclaimer and decide its placement~~ — **done (2026-07-12).** Both, not either/or: a first-run modal (localStorage-gated) plus the persistent footer strip stays. See "Trust, error reporting & disclaimer" above. +- [x] ~~Add jurisdiction-wide default rules as a schema-level concept~~ — **done (2026-07-12), see [schema/common-schema.md](schema/common-schema.md).** `signed: false` is the load-bearing field. No real data in this shape exists yet — the schema is ready, Walnut's actual rule still isn't encoded anywhere but prose. - [ ] ~~Prototype the crime-risk aggregation approach~~ — **paused pending [Discussion #1](https://github.com/inkxel/chalked/discussions/1)**, not a committed build item. See [ETHICS.md](ETHICS.md). + +**New, from this round (2026-07-12) — what's still genuinely open:** +- [ ] Finish the Seattle sweeping and San Diego sweeping/permits dashboard-traces live — this round's sandbox blocked direct HTTPS to `arcgis.com`/`data.seattle.gov`/`data.sandiego.gov` at the network-policy level, so those findings are corroborated-by-search, not confirmed-by-query. Needed before either city's rank above is treated as settled. +- [ ] Re-verify the Chicago concessionaire meter endpoint (`map.chicagometers.com/get_meters`) is still live — the only known community precedent for it is a repo dead since 2019, and the concessionaire itself is mid-sale. +- [ ] Build a second real adapter (not just research a second candidate) — every city discussed above is still just research; LA remains the only jurisdiction with actual code in `scripts/`. +- [ ] Encode Walnut, CA's actual rule (or Chicago's §9-64-090 finding) as the first real jurisdiction-wide-default-rule entry, now that the schema for it exists — proves the concept isn't just theoretical. +- [ ] Automate `city:*`/`category:*` label application on new data-issue reports (currently a manual triage step, by design, until report volume justifies more tooling) — see [schema/error-report-pipeline.md](schema/error-report-pipeline.md). diff --git a/app.js b/app.js index dd84b00..0d1d6f1 100644 --- a/app.js +++ b/app.js @@ -91,6 +91,20 @@ if (DEBUG_TIME) { document.body.insertBefore(banner, document.body.firstChild); } +// First-run disclaimer notice (SPEC.md "Trust, error reporting & disclaimer") -- shown once +// per browser via localStorage, not once per page load, so it's a real deliberate read +// rather than a modal muscle-memory dismisses without looking. The persistent footer strip +// in index.html stays too, for anyone who wants to re-check it later without clearing storage. +const DISCLAIMER_ACK_KEY = "chalked_disclaimer_ack_v1"; +const disclaimerOverlay = document.getElementById("disclaimer-modal-overlay"); +if (localStorage.getItem(DISCLAIMER_ACK_KEY)) { + disclaimerOverlay.classList.add("hidden"); +} +document.getElementById("disclaimer-ack-btn").addEventListener("click", () => { + localStorage.setItem(DISCLAIMER_ACK_KEY, "1"); + disclaimerOverlay.classList.add("hidden"); +}); + const map = L.map("map").setView([34.0522, -118.2437], 11); // Exposed for QA/testing (e.g. Playwright driving the map programmatically). // Gotcha worth remembering: map.latLngToContainerPoint() returns coordinates relative to @@ -109,12 +123,51 @@ const statusPanel = document.getElementById("status-panel"); let sweepingData = null; let geoLayer = null; +// Per-category confidence/staleness badge -- SPEC.md's "Trust, error reporting & +// disclaimer" section: the data_as_of timestamp (schema/common-schema.md) needs to be a +// visible signal in the UI, not just a schema field nobody sees. Silent staleness reads as +// confident when it isn't, which is worse than showing no data at all -- LA's permits (2015) +// is the concrete case that forced this, but the badge is shared across all three +// categories so a future stale sweeping/meters source gets the same honest treatment +// automatically, not a bespoke one-off like the old permits-only text did. +const STALE_THRESHOLD_DAYS = 180; + +function confidenceBadge(dataAsOf) { + if (!dataAsOf) return `Data vintage unknown.`; + const asOfDate = new Date(dataAsOf); + const ageDays = (getNow() - asOfDate) / 86400000; + const label = asOfDate.toLocaleDateString(); + if (ageDays > STALE_THRESHOLD_DAYS) { + return `Data as of ${label} — may be outdated.`; + } + return `Data current as of ${label}.`; +} + +// Builds a prefilled GitHub issue-creation link rather than posting anything ourselves -- +// see schema/error-report-pipeline.md for why (no backend, no anonymous auto-post; a real +// GitHub account to submit through IS the anti-spam/moderation step, not a placeholder for +// one). GitHub's issue forms prefill fields whose id matches a query param name. +function reportIssueUrl(category, jurisdictionLabel) { + const params = new URLSearchParams({ + template: "data-issue.yml", + title: `[data-issue] ${category} — ${jurisdictionLabel}`, + jurisdiction: jurisdictionLabel, + category, + }); + return `https://github.com/inkxel/chalked/issues/new?${params.toString()}`; +} + +function reportLink(category, jurisdictionLabel) { + return `Report a problem →`; +} + function renderPanel(zone, status) { statusPanel.innerHTML = `` + `${zone.route_id} (${zone.maintenance_district_name}) — ${zone.day_of_week}s, ` + `${zone.start_time}–${zone.end_time}, weeks ${zone.weeks_of_month.join(" & ")}, ` + - `${zone.side_of_street} side. ${status.label}`; + `${zone.side_of_street} side. ${status.label} ${confidenceBadge(zone.data_as_of)} ` + + reportLink("sweeping", "Los Angeles, CA"); } // Meters aren't a restriction with a status level -- they're a cost fact (SPEC.md's @@ -124,23 +177,25 @@ function renderMeterPanel(meter) { statusPanel.innerHTML = `` + `Meter ${meter.space_id} — ${meter.blockface}. ` + - `${meter.rate_type} rate ${meter.rate}, ${meter.time_limit} limit. ` + + `${meter.rate_type} rate ${meter.rate}, ${meter.time_limit} limit. ${confidenceBadge(meter.data_as_of)} ` + `No operating-hours/schedule data available for LA meters (see research/cities/los-angeles.md) — ` + - `check the posted meter sign for when payment is actually required.`; + `check the posted meter sign for when payment is actually required. ` + + reportLink("meters", "Los Angeles, CA"); } // Permits are an eligibility gate, not a timing question (SPEC.md's Visual design section) // -- the app can't know if *this user* holds the right permit for *this* district, so it's -// a flag, not a status level. The 2015 staleness (research/cities/los-angeles.md) is -// surfaced directly here, not just in docs -- a decade-old boundary shouldn't read with -// the same confidence as fresh data. +// a flag, not a status level. The 2015 staleness (research/cities/los-angeles.md) now comes +// through the shared confidenceBadge() instead of bespoke inline text, same as the other +// two categories. function renderPermitPanel(district) { statusPanel.innerHTML = `` + `Preferential Parking District ${district.district_number} ` + `(${district.district_name}) — permit required for unrestricted parking here. ` + - `Data as of ${district.data_as_of} — LADOT hasn't ` + - `confirmed an update since then, so district boundaries may not reflect changes made after that date.`; + `${confidenceBadge(district.data_as_of)} LADOT hasn't confirmed an update since then, so ` + + `district boundaries may not reflect changes made after that date. ` + + reportLink("permits", "Los Angeles, CA"); } function styleFeature(now) { @@ -164,6 +219,14 @@ function refreshColors() { let nationalPlacesData = null; let coverageRegistry = null; +// A jurisdiction is "covered" (blue outline) if ANY category is actually built -- see +// schema/coverage-registry.md. Partial coverage (e.g. a future "Chicago minus meters") +// still reads as covered at this national zoom level; the per-category gap only becomes +// visible once you're inside that jurisdiction, which is the intended behavior. +function isCovered(registryEntry) { + return !!registryEntry && Object.values(registryEntry.categories).some((status) => status === "built"); +} + Promise.all([ fetch("data/national-places.geojson").then((r) => r.json()), fetch("data/coverage_registry.json").then((r) => r.json()), @@ -175,8 +238,7 @@ Promise.all([ renderer: L.canvas(), interactive: false, style: (feature) => { - const covered = coverageRegistry[feature.properties.place_id]; - return covered + return isCovered(coverageRegistry[feature.properties.place_id]) ? { color: "#1565c0", weight: 1.5, fillOpacity: 0, opacity: 0.6 } : { color: "#555", weight: 0.6, fillColor: "#888", fillOpacity: 0.55, opacity: 0.6 }; }, @@ -199,8 +261,8 @@ map.on("click", (e) => { return; } - const covered = coverageRegistry[place.properties.place_id]; - if (covered) { + const registryEntry = coverageRegistry[place.properties.place_id]; + if (isCovered(registryEntry)) { statusPanel.innerHTML = `You're in ${place.properties.name} — click a colored sweeping/meters/permits zone above for its specific status, ` + `or this spot may just not have one nearby.`; diff --git a/data/README.md b/data/README.md index 82ae33a..eb1e69e 100644 --- a/data/README.md +++ b/data/README.md @@ -9,4 +9,4 @@ Most files here are generated — not committed (see `.gitignore`), regenerated Run all four before serving the site locally. -**`coverage_registry.json` is the one exception — hand-maintained source, not generated, and it IS committed.** It's the map of which jurisdictions are actually supported and which categories they have built; see [research/national-boundary-layer.md](../research/national-boundary-layer.md) for how it's used. Adding a new city to Chalked means adding an entry here. +**`coverage_registry.json` is the one exception — hand-maintained source, not generated, and it IS committed.** It's the map of which jurisdictions Chalked has looked at, per-category status for each (`built`/`in_progress`/`gap`/`not_applicable`/`unconfirmed`/`paused` — see [schema/coverage-registry.md](../schema/coverage-registry.md) for the full shape and [schema/common-schema.md](../schema/common-schema.md) for what each status means), and population for the contribution-priority ranking in SPEC.md. See [research/national-boundary-layer.md](../research/national-boundary-layer.md) for how the map reads it. Adding a new city to Chalked means adding an entry here. diff --git a/data/coverage_registry.json b/data/coverage_registry.json index d5984a2..7476036 100644 --- a/data/coverage_registry.json +++ b/data/coverage_registry.json @@ -2,7 +2,12 @@ "0644000": { "name": "Los Angeles", "state": "CA", - "status": "supported", - "categories": ["sweeping", "meters", "permits"] + "population": 3898747, + "categories": { + "sweeping": "built", + "meters": "built", + "permits": "built", + "crime": "paused" + } } } diff --git a/index.html b/index.html index 8b5a965..4c65ad8 100644 --- a/index.html +++ b/index.html @@ -32,9 +32,25 @@ .dot.amber { background: var(--amber); } .dot.red { background: var(--red); } #disclaimer { padding: 0.4rem 1rem; font-size: 0.72rem; color: #888; background: #fafafa; border-top: 1px solid #eee; } + #disclaimer-modal-overlay { position: fixed; inset: 0; background: rgba(0,0,0,0.6); display: flex; align-items: center; justify-content: center; z-index: 1000; padding: 1rem; } + #disclaimer-modal-overlay.hidden { display: none; } + #disclaimer-modal { background: #fff; max-width: 480px; padding: 1.5rem; border-radius: 8px; font-size: 0.9rem; line-height: 1.5; box-shadow: 0 4px 24px rgba(0,0,0,0.35); } + #disclaimer-modal h2 { margin: 0 0 0.75rem; font-size: 1.1rem; } + #disclaimer-modal button { margin-top: 1rem; padding: 0.55rem 1.2rem; font-size: 0.9rem; cursor: pointer; background: var(--ink); color: #fff; border: none; border-radius: 4px; } + +
+ +

Chalked

@@ -52,7 +68,7 @@

Chalked

Click a zone on the map, or search an address, to see its sweeping status.
-
Informational only, not responsible for citations or towing. Always defer to posted physical signage over anything shown here. Data source: LADOT, synced manually for now — see SPEC.md.
+
Informational only, not responsible for citations or towing. Always defer to posted physical signage over anything shown here. Data source: LADOT, synced manually for now — see SPEC.md. Know a parking rule with no posted sign anywhere (like Walnut, CA's citywide overnight-permit rule)? Report it →
diff --git a/research/README.md b/research/README.md index d95f4d0..73ea06d 100644 --- a/research/README.md +++ b/research/README.md @@ -33,17 +33,23 @@ Every city file should be linked from its state's file, and every state file sho ## Index **States** -- [California](states/california.md) — Los Angeles, San Francisco +- [California](states/california.md) — Los Angeles, San Francisco, San Diego +- [Illinois](states/illinois.md) — Chicago +- [Washington](states/washington.md) — Seattle **Cities** - [Los Angeles](cities/los-angeles.md) — sweeping, permits, meters, crime, sweep.la regional context - [San Francisco](cities/san-francisco.md) — CURB, SF's own data sources, why no Chalked adapter here +- [San Diego](cities/san-diego.md) — meters (strong), sweeping (real dataset, dashboard-trace unfinished), permits (likely gap), crime (open), no CURB-equivalent found +- [Chicago](cities/chicago.md) — sweeping, permits, crime confirmed open; meters gap re-traced (government side still closed, unverified private concessionaire API is the only known fallback) +- [Seattle](cities/seattle.md) — sweeping's "unconfirmed" status overturned (real SPU route data on Seattle's separate ArcGIS open-data portal), with a source-confirmed voluntary-enforcement caveat; permits and meters spot-checked and holding up as open -*(Chicago, NYC, Seattle, DC have entries in the README's coverage table and their own jurisdiction issues, but no dedicated research file yet — they'll get one, and their state gets an index page, once someone researches them past the summary level.)* +*(Chicago, NYC, DC have entries in the README's coverage table and their own jurisdiction issues, but no dedicated research file yet — they'll get one, and their state gets an index page, once someone researches them past the summary level.)* **Methods & cross-city landscape** - [Dashboard-tracing method](dashboard-tracing-method.md) — how to find real data hiding behind a public-facing ArcGIS dashboard - [Municipal code hosting & unsigned rules](municipal-code-hosting.md) — the Walnut, CA case, and where to look for code-only rules - [National vendor & standard landscape](national-vendor-landscape.md) — meter vendors, CDS (checked and closed out), Parkopedia - [City hub-page & aggregator scan](city-hub-scan.md) — broad 29-city discovery pass: prior art (ParkUsher), confirmed vendor-by-city map, and the structural finding that street sweeping is almost universally split from meters/permits by department +- [ParkUsher investigation](parkusher.md) — direct look at the closest multi-city prior art found: not open source, no confirmed open-data pipeline (looks substantially manual/crowdsourced), meters paywalled, coverage uneven across its 7 cities (incl. Seattle and SF) — doesn't deprioritize either from Chalked's candidate list - [National boundary layer & coverage registry](national-boundary-layer.md) — how the "national map, gray where uncovered" base layer actually works: Census TIGERweb source, the size/simplification tradeoff, canvas rendering at 19,731-feature scale, and how to add a new city to the coverage registry diff --git a/research/cities/chicago.md b/research/cities/chicago.md new file mode 100644 index 0000000..c968a06 --- /dev/null +++ b/research/cities/chicago.md @@ -0,0 +1,50 @@ +# Chicago + +The city SPEC.md flags as "meters: gap (private concessionaire, Chicago Parking Meters LLC, no open API)" while calling sweeping, permits, and crime open. Given how wrong LA's own "gap" calls turned out to be (see [Los Angeles](los-angeles.md)), this file re-traces Chicago's meters call specifically, and spot-checks the other three categories. + +**A network caveat, stated up front:** this research pass was run in a sandboxed environment whose egress policy blocked essentially all outbound HTTPS except `github.com`/`raw.githubusercontent.com` — including `data.cityofchicago.org`, `chicago.gov`, `map.chicagometers.com`, `map.parkchicago.com`, and (for comparison) `data.lacity.org` and `arcgis.com` too. Unlike the Los Angeles research, none of the URLs below could be live-queried directly from this session; everything here is sourced through search-engine indexing of those pages (which does surface real dataset IDs, update dates, and schema descriptions) plus one GitHub repo fetched directly. Treat the live-data claims below as "strongly indicated by secondary evidence, not independently re-queried this session" — someone with unrestricted network access should re-run the actual queries (especially the `get_meters` endpoint under Meters) before this is fully load-bearing. + +## Meters — the flagged gap, checked directly + +**1. Chicago Data Portal (Socrata, data.cityofchicago.org) — confirmed no meter dataset exists.** Searched the portal directly (via `site:data.cityofchicago.org` queries and category browsing, not just general web search) for anything meter-related — rates, zones, locations. Found nothing. The only hits referencing "Chicago Parking Meters LLC" on the portal are **FOIA request logs** (e.g. `FOIA-Request-Log-Revenue` `zrv6-shhf`, `FOIA-Request-Log-City-Clerk` `72qm-3bwf`) — records of *other people's* FOIA requests for the concessionaire's exempt-persons-loss reporting, not actual meter data. This is a real, confirmed absence, not an unsearched gap: Chicago's portal has live Transportation and Sanitation categories with real sweeping/permit datasets (below) and simply has no equivalent for meters. Chicago's own city page, [`chicago.gov/.../parking_meter_information.html`](https://www.chicago.gov/city/en/depts/fin/supp_info/revenue/parking_meter_information.html), doesn't host data either — it just points citizens to the concessionaire's own map. + +**2. The concessionaire's citizen-facing map is real, and does have a traceable backing API — but it's not ArcGIS.** Chicago Parking Meters LLC's consumer product is **ParkChicago Map**, served from `map.chicagometers.com` (and mirrored at `map.parkchicago.com`) — a Passport-powered pay-by-phone map showing 36,000+ metered spaces, color-coded by availability prediction, same pattern as LA's dashboard-style citizen tools. It is not an Esri/ArcGIS product, so the [dashboard-tracing method](../dashboard-tracing-method.md)'s exact recipe (item ID → Web Map → Feature Service) doesn't apply — but the same underlying principle does: a citizen-facing map implies a real backing API, whether or not it's advertised as a developer product. + +**3. A community scraper already found and used that exact API — this is Chalked's "historically scraped" precedent.** [`stevevance/Chicago-Parking-Meters`](https://github.com/stevevance/Chicago-Parking-Meters) (fetched directly via GitHub, not just search) documents hitting the concessionaire's own endpoint directly: + +``` +http://map.chicagometers.com/get_meters?ne_lat=42.023022&ne_lng=-87.523984&sw_lat=41.643919&sw_lng=-87.940101 +``` + +A bounding-box query (northeast/southwest lat/lng corners) returning meter locations, rates, and time-limit info as JSON, no API key. The repo's v1 was a one-time scrape from January 2014; v2 (the version using the URL above) queried the live endpoint directly with an 8-mile max radius, then converted the JSON response to CSV/GeoJSON via OpenRefine and QGIS. + +**The catch: this repo is dead, and nobody has re-confirmed the endpoint is still live.** Last commit **June 26, 2019** — 7 years stale as of this research (2026-07-12). No forks or newer successor projects turned up in a search for `get_meters`, `chicagometers`, or "unofficial ParkChicago API" from 2020 onward. The ParkChicago Map product itself is still live today (search results surfaced current-looking pages like `map.chicagometers.com/terminal/514507/...`), so the front-end this API backs hasn't disappeared — but whether `get_meters` still exists at that path, unauthenticated, with the same schema, is genuinely unknown until someone with real network access re-issues the query. + +**4. The official vendor path is closed, same as LA/nationally.** Passport is the vendor behind ParkChicago (confirmed in [national-vendor-landscape.md](../national-vendor-landscape.md) and [city-hub-scan.md](../city-hub-scan.md) — Passport serves Chicago, Austin, and white-labels Portland's "Parking Kitty"). Passport's own developer product, the Parking Rights API, is partner-gated (OEM/business-dev agreement), not something Chalked could plug into — already checked and closed out at the vendor level, not specific to Chicago. + +**5. Context that matters for how durable any answer here is:** Chicago Parking Meters LLC itself is mid-sale as of this writing — Stonepeak Partners has a pending ~$2.5B bid to buy the concession, contested by 22+ alderpersons over transparency concerns, with a competing city buyback offer (~$3.3B) reportedly declined by Mayor Johnson's administration. City Council approval is still pending (Chicago Sun-Times, Block Club Chicago, CBS Chicago, all June–July 2026 coverage). Whatever backing infrastructure exists today could change hands, get rebuilt, or get re-platformed under new ownership — a real reason not to over-invest in reverse-engineering the current endpoint without expecting some maintenance risk. + +**Bottom line on meters: downgraded, not overturned.** Unlike LA's sweeping/permits — where the "gap" was simply wrong and real open government data existed all along — Chicago genuinely has **no government-published open meter dataset**, confirmed by direct portal search, not assumption. What does exist is a private company's own consumer-map API, previously scraped by exactly the kind of community project SPEC.md already anticipated, but that project is abandoned and its endpoint is unverified-live as of this research pass. The honest status is **"gap, with an unconfirmed private-API fallback"** — better than a flat "no path exists," but meaningfully different from a real open-data source: it would mean scraping a private vendor's consumer product (ToS/legal exposure a government open-data portal doesn't carry), on an endpoint nobody has checked in 7 years, that may not survive the pending ownership change anyway. Next step before building anything: have someone with unrestricted network access actually re-issue the `get_meters` query above and confirm it still returns real data in its documented shape. + +## Sweeping — spot-check: still open, confirmed + +Chicago Data Portal has live, current-year datasets: **Street Sweeping Schedule – 2026** (`u5ai-3efk`) and **Street Sweeping Zones – 2026** (`nk6s-cie5`), both under the Sanitation category at `data.cityofchicago.org`, with prior-year versions (2025: `a2xx-z2ja` schedule, `utb4-q645` zones) still present — a real year-over-year publishing cadence, not a one-off. Schedules run by ward and ward-section number. This is a genuine Socrata-hosted dataset, not a dashboard wrapper, consistent with the dashboard-tracing method's own note that Socrata portals (SF, Chicago, NYC) tend to advertise their APIs directly rather than hide them behind a citizen dashboard. **Confirms the existing "open" call.** + +Also found (not previously in Chalked's notes): **[We The Sweeple](https://wethesweeple.com)** (formerly Sweep Around Us), Chicago's own sweeping-only alerts/calendar tool, already logged in [city-hub-scan.md](../city-hub-scan.md) as confirmed open source ([github.com/srobbin/sweeparoundus](https://github.com/srobbin/sweeparoundus)) — real community prior art for this exact category, worth referencing if a Chicago sweeping adapter gets built. No CURB/sweep.la-equivalent comprehensive aggregator found for Chicago beyond this single-category tool. + +## Permits — spot-check: still open, confirmed, actively maintained + +**Parking Permit Zones** (`u9xt-hiju`, Transportation category, `data.cityofchicago.org`) — all street segments belonging to a Residential Parking Zone, keyed by Record ID and Zone number, updated daily with major installments after City Council meetings. Search results showed a most-recent-update date of **2026-04-30** — genuinely current, not stale like LA's 2015-frozen PPD dataset. + +Two older/duplicate-looking listings exist on the same portal and are worth flagging so nobody builds against the wrong one: **Permit Parking Zones** (`qiag-khha`) appears stale (last update found: December 2015), and **Parking Zones** (`az5k-c8i5`) looks like an earlier name for the same underlying dataset. `u9xt-hiju` is the one with the recent update date and should be treated as the current source. **Confirms the existing "open" call**, with a naming-collision caveat future Chicago work should watch for. + +## Crime — spot-check: still open, confirmed + +**Crimes – 2001 to Present** (`ijzp-q8t2`, Public Safety category, `data.cityofchicago.org`) — sourced from CPD's CLEAR system, updated daily (minus the most recent 7 days), addresses shown at block level for privacy. Same shape as LA's and SF's crime feeds: real and geocoded, but not a purpose-built "vehicle break-in" category — folded into broader theft/larceny/burglary codes, matching SPEC.md's existing note. **Confirms the existing "open" call.** As with LA, this is background context, not an active build target while the crime-risk overlay itself is paused pending [Discussion #1](https://github.com/inkxel/chalked/discussions/1). + +## What this means for SPEC.md's coverage table (not changed here, flagging for the separate update pass) + +- Sweeping: open — confirmed, no change. +- Permits: open — confirmed, no change, but note the `u9xt-hiju` vs `qiag-khha`/`az5k-c8i5` naming collision for whoever builds the adapter. +- Crime: open — confirmed, no change. +- Meters: still effectively a gap for real open *government* data, but not a dead end — there's a specific, previously-working, unauthenticated private API (`map.chicagometers.com/get_meters`) with a 2019 community precedent that needs a live re-check, not a fresh "no path exists" writeoff. Recommend the table's meters cell move from a flat "❌ gap" note to something like "gap (government side); unofficial concessionaire API previously scraped, dead since 2019, unverified live" — precise language is the SPEC.md update pass's call, not this file's. diff --git a/research/cities/san-diego.md b/research/cities/san-diego.md new file mode 100644 index 0000000..a7aa9b0 --- /dev/null +++ b/research/cities/san-diego.md @@ -0,0 +1,69 @@ +# San Diego + +Surfaced as a new candidate by the broad 29-city scan (see [city-hub-scan.md](../city-hub-scan.md) → "New candidate city surfaced: San Diego"), which found "a real, confirmed open dataset" on data.sandiego.gov but didn't go deeper. This pass goes deeper, per category, using the same [dashboard-tracing method](../dashboard-tracing-method.md) that worked for LA. Standing: meters (real, open, well-documented, one clear vendor), sweeping (real dataset confirmed, but the live-service trace is incomplete — see caveat below), crime (open, block-geocoded, consistent with the pattern), permits (likely a gap — no open dataset found). No CURB/sweep.la-equivalent aggregator exists for San Diego specifically. + +**Research-session caveat, stated up front:** this pass could not make direct HTTPS requests to `arcgis.com`, `webmaps.sandiego.gov`, or `data.sandiego.gov` — this session's outbound network policy rejected the connection at the proxy level (confirmed via the proxy's own status log, not a site-side block). Everything below is sourced from search-engine results and cached page snippets, not a live query against the actual services the way LA's `Posted_Street_Sweeping_Routes_Update` FeatureServer was directly queried and confirmed. That means San Diego's status here is "real dataset confirmed to exist, full technical trace incomplete" rather than LA's fully-verified status — a good next task for whoever can reach those hosts directly, using the exact method below. + +## Sweeping + +`Street Sweeping Schedule` — https://data.sandiego.gov/datasets/street-sweeping-schedule/ — is real: it's the same dataset the original 29-city scan flagged, describing time/date of sweeping by block and whether that block has a posted sign restricting parking during the sweep. + +**The "queryable" claim needs a caveat the original scan didn't catch:** data.sandiego.gov itself runs on **JKAN** — "a lightweight, backend-free open data portal, powered by Jekyll" (confirmed via the portal's own [architecture story](https://data.sandiego.gov/stories/portal-architecture/) and [open-source page](https://data.sandiego.gov/open-source/), and JKAN's own repo, [timwis/jkan](https://github.com/timwis/jkan)). JKAN is a static site (Jekyll, deployed to S3) that catalogs datasets and links out to file downloads (CSV/GeoJSON/Shapefile) — it is not a live query API the way Socrata's SODA API (LA, SF, Chicago, NYC) is. A city-hub-scan claim of "queryable" for a JKAN-hosted city means "a file you can download and query yourself," not "an endpoint you can hit with `?where=` filters," unless the underlying source happens to also expose one directly (as turned out to be true for LA's sweeping data, hidden behind a dashboard instead of listed plainly). + +**Why this dataset is still a strong lead, not just a static CSV:** the City has run a real, live GIS operation behind its sweeping program since April 2018 — **GeoSweep**, which integrates an SAP HANA EAM system (work orders), a CalAmp AVL system (real-time truck GPS + tonnage), and an Esri **ArcGIS Enterprise** system, confirmed via a [Quartic Solutions case study](https://quarticsolutions.com/case-studies/city-of-san-diego-geo-enabling-sweeping-routes/) and a joint Esri/SAP case-study PDF. That's the same shape as LA's case — a real, actively-maintained backend, just not advertised as a developer API. Two public-facing entry points exist that fit the dashboard-tracing pattern exactly: +- **City of San Diego Street Sweeping Dashboard** — `https://www.arcgis.com/apps/dashboards/7a280516d4b040139b91725f4c161f68` +- **Street Sweeping Route and Schedule** (Web AppBuilder viewer, not a Dashboard, but the same "webmap → operationalLayers → FeatureServer" structure applies) — `https://sandiego.maps.arcgis.com/apps/webappviewer/index.html?id=ee3b1f22fc6246bbbc9349a0c866132c` + +Neither has been traced to its backing FeatureServer yet — that's the concrete next step, and per the caveat above, it needs an environment that can actually reach `arcgis.com`. + +**One adjacent FeatureServer found and explicitly ruled out as a false match:** `https://webmaps.sandiego.gov/arcgis/rest/services/Hosted/Stormwater_Sweeping_Annual/FeatureServer/0` ("Median Sweep Routes," polyline geometry) is a *stormwater* program sweeping street medians, not the curb-parking sweeping schedule that matters here. Worth flagging so a future pass doesn't grab the wrong layer. + +**Program-level facts worth carrying into the schema, not just the data source:** as of the last major schedule revision, 27,000+ city blocks were in the sweeping program, and roughly **90% have no posted sign** — sweeping happens, residents are asked to voluntarily move their cars, but there's no citation risk either way. Only the remaining ~9–10% are posted "no parking" and actually ticketed. The City ran a route/frequency overhaul starting September 1, 2022 (13 routes, 300+ miles, new signage phased in with a 30-day grace period per the city's [own announcement PDF](https://www.sandiego.gov/sites/default/files/streetsweepingchanges082922_final_web_version.pdf)). This is the same honesty problem LA's permit staleness raised: most of San Diego's sweeping data describes a *voluntary* schedule, not an enforced one, and a Chalked adapter needs to represent "swept but unenforced" as its own status, not conflate it with the ~10% that's actually ticketed. + +## Meters + +`Parking Meters Locations` — https://data.sandiego.gov/datasets/parking-meters-locations/ — real, published by the Office of the City Treasurer, CSV format, observed "last updated July 2, 2026" with a stated daily update cadence. A companion dataset, `Parking Meters Transactions` — https://data.sandiego.gov/datasets/parking-meters-transactions/ — publishes (aggregated) per-meter transaction history. Both are used successfully in an independent GeoPandas/Folium project, [taekunkim/parking_meter](https://github.com/taekunkim/parking_meter), which builds meter-location and transaction heatmaps directly from them — real-world confirmation the files are usable, not just listed. + +**Scale:** the City manages roughly **5,300 metered on-street spaces** citywide (much smaller than LA's ~35,000-point dataset — expect no clustering library needed here). + +**Vendor, confirmed:** **IPS Group, Inc.**, a Sorrento Valley (San Diego)-based smart-meter manufacturer, supplies the City's on-street "SMART" meter hardware and its own **Park Smarter** payment app (`parksmarter.com`, and the [IPS Group product page](https://ipsgroup.com/park-smarter/)) — city council approved the contract, and the Treasurer's office completed converting on-street meters to IPS SMART hardware. ParkMobile is also usable in San Diego, but for garages/lots/campus facilities, not as the primary on-street meter vendor — worth not conflating the two the way the original scan's vendor-by-city map risks doing elsewhere. + +**Same JKAN caveat as sweeping applies:** these are downloadable files with a claimed daily refresh, not a live filterable query endpoint — a real, working data source, just a different integration shape than Socrata's `.json` resource URLs (LA, SF, Chicago, NYC). + +## Permits + +**Likely a gap, not yet resolved.** San Diego's official residential permit parking page, https://www.sandiego.gov/parking/permits/resparkmaps, lists named districts (Hillcrest, SDSU/College Area, Logan Heights, Mesa College, El Cortez/Downtown, and others) but every lead found this pass points to static maps/PDFs on `sandiego.gov`, not an open dataset on `data.sandiego.gov` or a Feature Service on `webmaps.sandiego.gov`. + +**One dashboard-tracing lead, not yet traced:** an ArcGIS Web AppBuilder viewer titled "Residential Permit Parking, Time Limits, and Meters" exists at `https://www.arcgis.com/apps/webappviewer/index.html?id=efb2bfe552454d1cbe9b2ea74ea91d6c`. This is exactly the shape that turned out to be real for LA's Preferential Parking Districts — a citizen-facing viewer sitting on top of a real backing service nobody's traced yet. Blocked from tracing it this pass by the same network restriction noted above; worth being the first thing checked from an unblocked environment. + +**A naming-collision gotcha worth recording explicitly** (the same class of mistake city-hub-scan.md already flagged for "Portland, Maine" vs. "Portland, Oregon"): a dataset called "Residential Parking Permit (RPP) Zones," hosted at `gisdata-csj.opendata.arcgis.com`, surfaces prominently in generic searches for "San Diego RPP zones." **`CSJ` is San Jose, not San Diego** — that dataset belongs to the City of San Jose's open data portal and has nothing to do with San Diego. It would be an easy, wrong citation to carry into a future pass; flagging it here so it doesn't happen. + +## Crime + +`Police NIBRS Crime Offenses` — https://data.sandiego.gov/datasets/police-nibrs/ — real, open, extracted from SDPD's Records Management System, covering January 2020–present (SDPD's NIBRS reporting itself began in 2021). Fields include `NIBRS_UNIQ`, `CASE_NUMBER`, `OCCURED_ON`, `APPROVED_ON`, `DAY_OF_WEEK`, `MONTH`, `YEAR`, `CODE_SECTION`, plus a hundred-block address (rounded, not an exact street address — standard privacy practice, not unique to San Diego). The backing service is named `SDPD_NIBRS_Crime_Offenses_Geo` (`https://webmaps.sandiego.gov/arcgis/rest/services/SDPD/SDPD_NIBRS_Crime_Offenses_Geo/FeatureServer`) — the `_Geo` suffix plus the block-address field together indicate this is geocoded to the block, not exact points, the same convention most major CA cities checked so far use. Open and geocoded, matching the existing pattern — not yet filtered to vehicle-break-in-specific codes, same unresolved state as LA's crime data, and same paused status per [Discussion #1](https://github.com/inkxel/chalked/discussions/1). + +**One more lead worth a future check, not yet followed up:** `Parking Citations` — https://data.sandiego.gov/datasets/parking-citations/ — citations issued since 2012, monthly updates, CSV. Unconfirmed whether it carries lat/lon precise enough to attempt the CURB trick (GPS-matched citations inferring *real* enforcement timing vs. the posted schedule) — the same open question LA's own doc flags for itself, now flagged here too rather than assumed either way. + +## No CURB-equivalent found for San Diego + +Searched specifically for this, per the task's own instruction to double-check even though the original scan didn't flag one — confirmed, nothing comparable exists. + +What does exist: +- **Generic multi-city commercial apps** — SpotAngels, Xtreet, streetsweepingschedule.com, and an iOS app ("San Diego Street Cleaning") all list San Diego's sweeping schedule alongside hundreds of other cities. None are San Diego-specific, none confirmed open source — the same category as most cities nationally, not a dedicated local project. +- **One abandoned civic-tech attempt**: [opensandiego/sd-sweepaware](https://github.com/opensandiego/sd-sweepaware), built by the Open San Diego civic-hacking brigade to send SMS alerts the day before a resident's scheduled sweep. Explicitly tagged **inactive** on GitHub, and its own README describes sourcing schedules from **PDFs**, not the current open dataset — meaning it predates today's data.sandiego.gov listing and was never rebuilt against it. Real prior art, but dead, and narrower in scope (SMS-only, sweeping-only) than what a real Chalked adapter would cover even if revived. + +Net effect: San Diego is genuinely open territory for a comprehensive adapter, closer to Chicago's situation (no CURB/sweep.la-equivalent found there either) than to SF's (fully solved by CURB) or LA's (sweep.la covers it but isn't confirmed open source). + +**ParkUsher** — the multi-city aggregator city-hub-scan.md flagged as worth investigating directly — was searched specifically for San Diego coverage in this pass; inconclusive, no confirmation either way found. Still an open question inherited from the original scan, not resolved here. + +## Population + +**~1.38–1.40M, city proper** — 1,382,566 at the 2020 Census; the City's own FY2026 adopted budget cites ~1,404,452 as of July 1, 2024 (down slightly from a 2018 peak of 1,425,976). San Diego is California's second-largest city and roughly the 8th-largest in the US. + +For the population × data-completeness ranking convention (see SPEC.md → First adapters to build): this puts San Diego well above Seattle (~750K) and Washington DC (~670K), but below Chicago (~2.7M), Los Angeles (~3.9M), and NYC (~8.3M) among the existing candidate set. + +## Bottom line + +San Diego is a real, substantial-population city (nearly double Seattle's) with **one clearly strong category (meters — open, daily-updated, small enough to not need clustering, one unambiguous vendor)**, **one promising-but-unfinished category (sweeping — a real dataset and a real, live GIS backend confirmed, but the actual FeatureServer trace wasn't completable this pass due to a network restriction, not a data problem)**, **one likely gap (permits — no open dataset found, one untraced dashboard lead)**, and **one solid, unsurprising category (crime — open, block-geocoded, same as the rest of the state)**. No independent aggregator competes with a real Chalked adapter here, unlike SF (CURB) or, less decisively, LA (sweep.la). + +Relative to the existing candidate set: San Diego's population sits between Chicago and Seattle, and its data profile is a rough mirror image of Chicago's (strong meters / weak-or-unconfirmed permits, vs. Chicago's strong sweeping-permits-crime / weak meters). Pending the sweeping dashboard-trace and a real answer on permits, San Diego plausibly slots in **around 3rd–4th** on the ranked list — likely just behind Chicago, ahead of or comparable to Seattle, and ahead of DC on population alone even before data completeness is weighed — but LA and NYC remain clearly ahead of it either way. Worth a dedicated follow-up pass specifically to finish the two traces this session's network restriction left incomplete, before this ranking gets treated as settled. diff --git a/research/cities/seattle.md b/research/cities/seattle.md new file mode 100644 index 0000000..9fe9f74 --- /dev/null +++ b/research/cities/seattle.md @@ -0,0 +1,60 @@ +# Seattle + +SPEC.md's coverage table currently calls Seattle's sweeping status "unconfirmed," alongside meters and permits both marked "open." Applying the same trace-it-before-you-write-it-off rigor used on LA (see [dashboard-tracing-method.md](../dashboard-tracing-method.md) and [Los Angeles](los-angeles.md)): the sweeping "gap" turns out to be wrong in the same *shape* as LA's was — real, queryable data exists — but the underlying situation is different in a way that matters for how confidently Chalked should present it. + +## Sweeping + +**The gap was a research-pass miss, not a missing dataset — but it's a milder miss than LA's.** LA's data was hidden behind an undocumented citizen-facing dashboard that had to be traced through a Web Map to its Feature Service. Seattle's isn't hidden at all — it's just published on the city's *other* open-data portal, one a first pass checking only `data.seattle.gov` (Socrata) would miss entirely. + +Seattle runs two separate open-data platforms: +- **`data.seattle.gov`** — Socrata, the one anyone would check first. +- **`data-seattlecitygis.opendata.arcgis.com`** ("Seattle GeoData") — a second, ArcGIS Hub-based portal for GIS layers, with its own dataset pages, download formats, and API resources. + +Street sweeping routes live on the second one: + +- **"Street Sweeping Routes"** — Seattle GeoData: `https://data-seattlecitygis.opendata.arcgis.com/datasets/SeattleCityGIS::street-sweeping-routes/about` +- Underlying ArcGIS Online item, owned by **Seattle Public Utilities** (not SDOT): **"SPU Street Sweeping Routes"**, item id `d6cb9de98ee840fb8b6504f0b2e234f5`, `https://seattlecitygis.maps.arcgis.com/home/item.html?id=d6cb9de98ee840fb8b6504f0b2e234f5` + +That ownership detail matters: this confirms the [city-hub-scan.md](../city-hub-scan.md) finding that Seattle's sweeping program sits with **Seattle Public Utilities** (stormwater/water-quality program), not SDOT (which owns meters, RPZ permits, and general parking regs) — the same "wrong department, no cross-linking" fragmentation pattern the scan found almost everywhere else. SPU's own program page (`seattle.gov/utilities/.../street-sweeping`) states the city sweeps **90%+ of arterial streets**, **40 regular routes plus 14 protected bike-lane routes** year-round, plus **7 additional seasonal routes** added each fall for leaf cleanup — and that the underlying map layer is meant to carry route, schedule, holiday-suspension, and enforcement-note fields, i.e. exactly the shape of data a sweeping adapter would need. + +**Honest limitation on this pass, stated plainly rather than glossed over:** unlike the LA research (which had live tool access to query the Feature Service directly and confirm schema/record freshness), this session's network egress is blocked for both `arcgis.com` and `data.seattle.gov`/Socrata API domains — direct `?f=json` queries against the item metadata and Feature Service (the method's steps 2-4) could not be executed this pass. Everything above is corroborated through multiple independent search hits (the dataset's own "about" page title/URL, the ArcGIS item id, and SPU's program-description page), not a live query response. **Next step for whoever picks this up with working tool access:** run the actual trace — `https://www.arcgis.com/sharing/rest/content/items/d6cb9de98ee840fb8b6504f0b2e234f5?f=json` for the Feature Service `url`, then `?f=json` for schema and `/query?where=1=1&outFields=*&f=json` for live records and a `last_edited_date` freshness check — before building an adapter on top of it. + +**The enforcement caveat is now source-confirmed, not just noted.** [city-hub-scan.md](../city-hub-scan.md)'s methodology-gotchas section flagged, unsourced, that "Seattle currently doesn't ticket for missed sweeping days." SPU's own program page confirms this directly and explains the reasoning: *"Moving your car is voluntary... Enforcing parking rules is expensive, and many arterials already have few parked cars overnight, so sweepers have access to the curb without parking enforcement... parking enforcement fines have a greater impact on low-income communities,"* which the city cites as an equity consideration in choosing not to ticket. Multiple secondary sources (fixparkingticket.com, SpotAngels' Seattle guide) repeat the same "no tickets for missed sweeping" framing. + +**One wrinkle worth flagging, not fully resolved:** a handful of low-quality, likely auto-generated ticket-fighting sites (bridgelegal.org, 19pine.ai, stateregstoday.com) list a generic "$40-60 street cleaning violation" fine as if Seattle actively tickets for it — directly contradicting SPU's own statement. These read as boilerplate content-mill pages that list the same violation categories for every city they cover, not something built from Seattle's actual citation data, so they shouldn't be trusted over the primary source. But it does leave one real open question: whether *temporary no-parking* postings (event closures, construction, utility work — governed separately under SDOT's Temporary No Parking Zone program, which explicitly *is* enforced with 24-hour-notice signage and real citations/towing per `seattle.gov/documents/departments/sdot/cams/cam2114.pdf`) ever get confused with routine SPU sweeping in ticket records or in what a driver sees posted on a sign. Someone should check a sample of actual SPD/court citation data for "street cleaning" codes before assuming zero tickets are ever issued under that literal label — SPU's voluntary-compliance program and SDOT's enforced temporary-no-parking program are two different things that could get lumped together by a careless reading of citation data. + +**What this means for how Chalked should show it:** the underlying schedule data is real and traceable, but Seattle is a genuinely different confidence tier from LA/SF. A city where sweeping is real *and* actively ticketed (LA, SF) supports the map's normal green/amber/red restricted-parking confidence. Seattle's rule is real but the city itself says compliance is voluntary — showing it with the same visual urgency as an actively-enforced city would overstate the actual risk to a driver. This is exactly the SPEC's "restricted status shouldn't be presented with the same confidence in an unenforced city" principle the city-hub-scan flagged in the abstract; Seattle is the concrete case it was written for. + +## Permits (RPZ) + +Confirmed open, on **both** Seattle platforms, and fresher than LA's PPD data: + +- `data.seattle.gov` (Socrata): **"Restricted Parking Zones"**, dataset id `netm-8y6z` — `https://data.seattle.gov/dataset/Restricted-Parking-Zones/netm-8y6z` +- Seattle GeoData (ArcGIS): `https://data-seattlecitygis.opendata.arcgis.com/datasets/SeattleCityGIS::restricted-parking-zones` + +Search-result summaries of the dataset describe multiple constituent layers — signed RPZ blocks, permit-eligible blockfaces, and separate layers for Husky-game-day zones (Area A/B, near the UW stadium) vs. standard zones — on a **daily refresh cycle**. That refresh cadence, if it holds up under a live check, is a meaningfully stronger freshness story than LA's PPD data (stuck at a 2015 snapshot despite claiming annual updates). Not independently queried live this pass, same tooling caveat as above — worth a live schema/freshness check before an adapter is built, but nothing here contradicts SPEC's existing "permits: open" call. + +## Meters + +Spot-check only, not a full redo — SPEC's existing "meters: open (rich historical paid-occupancy data since 2012)" call holds up and is, if anything, undersold. `data.seattle.gov` hosts a genuinely deep set of paid-parking datasets: + +- **Paid Parking Occupancy (Last 30 Days)** — `data.seattle.gov/Transportation/Paid-Parking-Occupancy-Last-30-Days-/rke9-rsvs` +- Year-by-year historical occupancy datasets back to 2012 (e.g. `2020-Paid-Parking-Occupancy/wtpb-jp8d`, `2021-Paid-Parking-Occupancy/jb6y-98nr`, and so on through the current year) +- **Paid Parking Transaction Data** — `data.seattle.gov/Transportation/Paid-Parking-Transaction-Data/gg89-k5p6` +- A blockface-level parking inventory (peak-hour restrictions, rate, hours, RPZ number, curbspace type per blockface) + +SDOT's own description: occupancy data estimates **payment rate on each blockface by minute**. A third-party GitHub project (`github.com/yogitasn/seattlepaidparking`) already built something on top of this data, which is decent independent corroboration that it's real and usable, not just advertised. Vendor is **PayByPhone** per [city-hub-scan.md](../city-hub-scan.md)'s vendor-by-city map — not re-verified this pass, no reason to doubt it. + +## Crime + +Spot-check only. SPEC's "crime: open" call holds up: **`SPD Crime Data: 2008-Present`** — `data.seattle.gov/Public-Safety/SPD-Crime-Data-2008-Present/tazs-3rd5`, Socrata-hosted, updated daily (only UCR-finalized reports, so same-day entries lag). Not filtered to vehicle-break-in-specific codes, same caveat as LA — and same paused status: the crime-risk overlay is on hold pending [Discussion #1](https://github.com/inkxel/chalked/discussions/1), so this is background context, not an active build target. + +## "Find It, Fix It" — checked, not a lead + +The task asked specifically whether Seattle's citizen-facing **"Find It, Fix It"** app/map hides a backing Feature Service worth tracing, the way LA's sweeping dashboard did. It doesn't: it's a general service-request reporting tool (photo + location + description, routed to the relevant department — including "report a problem" for things like abandoned vehicles), not a schedule lookup, and it has no sweeping-specific data behind it. The actual citizen-facing sweeping lookup is SPU's own **Street Sweeping Route Map** (an ArcGIS web app for finding your route/schedule by address), which sits directly on top of the same `SPU Street Sweeping Routes` item traced above — there's no separate hidden layer behind it beyond that one. + +## Bottom line + +Seattle's "sweeping: unconfirmed" status is **overturned in the same direction as LA's** — real, structured route/schedule data exists, maintained and owned by SPU, published on Seattle's ArcGIS-based GIS open-data portal (not the Socrata one most people would check first). Unlike LA, this didn't require the full dashboard→Web Map→Feature Service trace; it just required checking Seattle's *second* open-data portal, which the original "unconfirmed" pass evidently didn't. The live-query confirmation LA got (schema, `last_edited_date`, record count) still needs to happen here — this session's tooling couldn't reach `arcgis.com` or Socrata APIs directly, so this is corroborated-by-search, not confirmed-by-query. + +The bigger finding, though, is the enforcement caveat, now sourced directly from SPU rather than just carried over unsourced from the city-hub-scan: **Seattle openly states compliance is voluntary and it does not ticket for missed sweeping.** So even once the dataset is live-verified, it shouldn't be status-coded the same way as LA/SF's actively-ticketed sweeping data — the honest Chalked treatment is "real schedule, low real-world stakes," not "restricted parking" at full confidence. Permits (RPZ) and meters both hold up as open and, if their claimed daily/by-minute refresh rates are real, are better-documented than LA's equivalents on both counts. Crime is open and unchanged in relevance (build paused pending Discussion #1). diff --git a/research/city-hub-scan.md b/research/city-hub-scan.md index 02db088..effaee9 100644 --- a/research/city-hub-scan.md +++ b/research/city-hub-scan.md @@ -69,7 +69,7 @@ Not on the original 5-city list, but the scan found **a real, confirmed open dat ## Next steps this scan points to -- Investigate ParkUsher directly (architecture, open-source status, data sourcing) — the same treatment CURB and sweep.la got. -- Consider San Diego as an addition to the first-adapter candidate set, given its confirmed real open dataset. -- When building the common schema, don't assume sweeping is always the primary category — regional variation (Sunbelt vs. coastal) means the schema needs to represent "this category doesn't meaningfully exist here" cleanly, not just "no data yet." +- [x] ~~Investigate ParkUsher directly (architecture, open-source status, data sourcing) — the same treatment CURB and sweep.la got.~~ — done, see [parkusher.md](parkusher.md): not open source, no confirmed open-data pipeline, meters paywalled, coverage uneven across its 7 cities (incl. Seattle and SF) — doesn't change Chalked's prioritization. +- [x] ~~Consider San Diego as an addition to the first-adapter candidate set, given its confirmed real open dataset.~~ — done, see [cities/san-diego.md](cities/san-diego.md): added to SPEC.md's ranked candidate list (~4th, provisionally) — strong meters, unfinished sweeping trace, likely-gapped permits. +- [x] ~~When building the common schema, don't assume sweeping is always the primary category...~~ — done, see [schema/common-schema.md](../schema/common-schema.md): `not_applicable` and `unconfirmed` are two of its six explicit per-category statuses, specifically for this distinction. - A vendor-adapter layer (matching a city to its meter payment vendor) could cover most scanned cities with only a handful of vendor integrations — worth weighing against building purely per-city adapters. diff --git a/research/municipal-code-hosting.md b/research/municipal-code-hosting.md index b2c64a3..9ba898f 100644 --- a/research/municipal-code-hosting.md +++ b/research/municipal-code-hosting.md @@ -12,3 +12,17 @@ This matters more than it looks like at first, because it's the one case where " - **Don't process a full municipal code.** Most are organized with a clear parking/vehicles-and-traffic chapter — narrow to that section with a cheap search pass before any LLM reads anything. A mechanical "does this chapter mention permit parking" classification is cheap-tier work; extracting the actual rule (who/what/when/exceptions) from an already-narrowed section is a step up, but still bounded and cheap relative to reading the whole code. See [research/README.md](README.md) for the model-sizing principle this follows. - **Let the contribution/error-reporting model do the discovery work, rather than brute-forcing every city nationally.** Tucker knew about Walnut's rule because he has in-laws there — that's exactly the kind of local knowledge the error-reporting pipeline (SPEC.md) should capture: "my city has an unsigned rule like X" as its own report type, triggering a targeted, cheap, single-city code lookup — not a standing, expensive effort to scrape every municipal code speculatively. - **Cache and re-check on a cadence, not per-request.** Municipal code changes are infrequent (amendments, not daily updates) — read once, store the result, re-check occasionally. + +## Platform check: 5 big cities + +Confirmed *where* each city's code lives and roughly *where* the parking chapter sits — not a full Walnut-style discovery pass. All URLs below resolved in web search results; direct WebFetch to `amlegal.com`, `municode.com`, and `dcregs.dc.gov` (and even plain `nyc.gov`) returned 403 in this sandbox, and the agent proxy's own log shows the same hosts hitting `connect_rejected` at the gateway — a sandbox network-policy issue, not evidence these sites are unfetchable in production. The actual adapter build needs to verify real fetchability (and likely a realistic user-agent, since amlegal/municode both look like they have anti-bot fronting) from wherever it actually runs. + +| City | Platform | URL | Notes | +|---|---|---|---| +| Los Angeles, CA | American Legal Publishing (amlegal.com) | [LAMC Ch. VIII – Traffic](https://codelibrary.amlegal.com/codes/los_angeles/latest/lamc/0-0-0-165153) | LAMC is fully on amlegal under `codelibrary.amlegal.com/codes/los_angeles/`. Careful: **LA County** (a separate jurisdiction, not the city) is on Municode instead — same name, different platform, don't conflate when building a city→platform lookup. Didn't dig for a hidden citywide rule. | +| New York, NY | American Legal Publishing (amlegal.com) | [34 RCNY §4-08 – Parking, Stopping, Standing](https://codelibrary.amlegal.com/codes/newyorkcity/latest/NYCrules/0-0-0-63770) | Two separate code sets live on the same amlegal account: the NYC **Administrative Code** (`NYCadmin`) and the NYC **Rules** (`NYCrules`, i.e. Title 34 RCNY, DOT's Traffic Rules). The Admin Code's parking references are sparse/mostly repealed — the actual operative day-to-day parking/standing/alternate-side rules live in `NYCrules` Ch. 4, §4-08, not the Admin Code. An adapter targeting "NYC parking law" needs to know to go to `NYCrules`, not the more obvious-sounding Admin Code. | +| Chicago, IL | American Legal Publishing (amlegal.com) | [MCC Title 9, Ch. 9-64 – Parking Regulations](https://codelibrary.amlegal.com/codes/chicago/latest/chicago_il/0-0-0-2645970) | Straightforward: Title 9 (Vehicles, Traffic and Rail Transportation), Ch. 9-64 Parking Regulations, Ch. 9-68 Restricted Parking Permits. Flag for later: **§9-64-090 bans overnight parking of trucks, RVs over 22', buses, commercial/livery vehicles on any residential street citywide** — a category-based (not block-specific) default that's unsigned and applies everywhere, same shape as Walnut's blanket rule, just gated on vehicle type instead of all vehicles. | +| Seattle, WA | Municode (municode.com) | [SMC Title 11, Subtitle I, Part 7 – Stopping, Standing, Parking and Loading](https://library.municode.com/wa/seattle/codes/municipal_code?nodeId=TIT11VETR) | The one of the five actually on Municode rather than amlegal. Title 11 (Vehicles and Traffic) → Subtitle I (Traffic Code) → Part 7 has the parking/standing chapters (11.70 Method of Parking, 11.72 Stopping/Standing/Parking Restrictions, 11.26 Service Parking Permits). Didn't check for a hidden rule. | +| Washington, DC | Neither — government-run, not a third-party codification platform | [DCMR Title 18, Ch. 18-24 §18-2405](https://dcrules.elaws.us/dcmr/18-2405) (unofficial elaws.us mirror; official host is `dcregs.dc.gov`) | DC's traffic/parking rules are **regulations** (DCMR), not council ordinances, and DC runs its own regs publication (Office of Documents & Administrative Issuances, dcregs.dc.gov) rather than outsourcing to Municode/amlegal/General Code — the "check the handful of platforms" heuristic whiffs here. Most on-point find of the five: DCMR **§18-2405 is literally titled "Stopping, Standing, or Parking Prohibited: No Sign Required"** — an explicit, named category of unsigned prohibitions (blocking driveways, fire lanes, sidewalks, etc.). Not as sweeping as Walnut's blanket citywide permit — it's closer to universal baseline traffic-code stuff every jurisdiction has — but it's DC's code affirmatively flagging "no sign needed" as its own legal category, which is exactly the kind of section worth reading in full when the DC adapter gets built. | + +**Synthesis.** Three of five (LA, NYC, Chicago) land on American Legal Publishing under the same `codelibrary.amlegal.com/codes//latest/` URL shape — more consolidation onto one platform than the "handful of platforms" framing above implies, at least among big cities. Seattle is the Municode outlier. DC breaks the pattern entirely: its code is regulatory (DCMR) rather than ordinance-based, so it's self-hosted by a city agency rather than farmed out to a third party — worth remembering that "regulations" vs. "ordinances" jurisdictions may split along this line more generally, not just for DC. NYC's split between Administrative Code and Rules (RCNY) on the *same* amlegal account is a trap worth documenting for adapter-writers: the obvious-sounding code isn't where the rule lives. And DC's own explicit "no sign required" section heading is a good sign that other cities' codes probably have similarly-named catch-all sections worth grepping for by name during real adapter-building, rather than relying on stumbling onto a Walnut by word of mouth every time. diff --git a/research/national-boundary-layer.md b/research/national-boundary-layer.md index 2c22c98..40f2361 100644 --- a/research/national-boundary-layer.md +++ b/research/national-boundary-layer.md @@ -27,7 +27,7 @@ Full TIGER/Line precision is unusably large at national scale (California alone, ## Coverage registry: hand-maintained, not generated -`data/coverage_registry.json` — unlike everything else in `data/`, this is **source, not a generated artifact** (worth remembering: it's excluded from `.gitignore`'s `data/*.geojson` pattern deliberately, on purpose, not an oversight). Keyed by `place_id` (state FIPS + place FIPS, e.g. `0644000` for Los Angeles), each entry lists `status` and which `categories` are actually built. Adding a new city to Chalked, once its adapter exists, means adding one entry here — the map, the gray/highlighted styling, and the click-fallback logic all read from this file without any other code changes. +`data/coverage_registry.json` — unlike everything else in `data/`, this is **source, not a generated artifact** (worth remembering: it's excluded from `.gitignore`'s `data/*.geojson` pattern deliberately, on purpose, not an oversight). Keyed by `place_id` (state FIPS + place FIPS, e.g. `0644000` for Los Angeles), each entry carries a per-category status (`built`/`in_progress`/`gap`/`not_applicable`/`unconfirmed`/`paused`) and population — see [schema/coverage-registry.md](../schema/coverage-registry.md) for the full shape (this replaced an earlier flat `status` + `categories: [...]` array shape that couldn't distinguish "confirmed gap" from "not built yet"). `app.js`'s `isCovered()` derives the map's blue/gray treatment from whether *any* category is `built` — the per-category detail only matters once you're inside that jurisdiction. Adding a new city to Chalked, once its adapter exists, means adding one entry here — the map, the gray/highlighted styling, and the click-fallback logic all read from this file without any other code changes. ## Verified outcomes (2026-07-06, via Playwright) diff --git a/research/parkusher.md b/research/parkusher.md new file mode 100644 index 0000000..71536b8 --- /dev/null +++ b/research/parkusher.md @@ -0,0 +1,54 @@ +# ParkUsher — prior-art investigation + +Flagged by [city-hub-scan.md](city-hub-scan.md) as "the single closest analog to CURB/sweep.la found anywhere in the scan" — live in 7 cities including two of Chalked's own candidates (Seattle, SF). This file gives it the same direct, dedicated look CURB got in [cities/san-francisco.md](cities/san-francisco.md). + +**Site access note:** `www.parkusher.app` returned HTTP 403 to every fetch attempt during this investigation (homepage, `/about`, `/contact`, `/locations/boston`, blog posts, both via WebFetch and via a direct `curl` through the sandboxed egress proxy) — consistent with Cloudflare/bot-blocking rather than a policy denial (the proxy's own status log shows the CONNECT being rejected upstream, not by org policy). No direct "view source" or network-request inspection was possible. Everything below is reconstructed from search-engine snippets, app store listings, press coverage, and third-party writeups — treat specifics (exact UI copy, exact city list on any given page) as slightly lower-confidence than a direct fetch would give. + +## Is it open source? + +**No.** No GitHub repo, org, or any open-source release found under "ParkUsher" or `parkusher` — checked directly, and it doesn't surface in any "open source parking app" search either. It's a closed commercial product: native iOS ([App Store](https://apps.apple.com/us/app/parkusher-find-parking-easily/id6454831202)) and Android ([Google Play](https://play.google.com/store/apps/details?id=app.parkusher)) apps plus a marketing/blog website, no public API, no data downloads. + +No tech-stack details are independently confirmed (see access note above — no view-source, no network tab). No engineering blog, no public job postings naming a stack. Nothing to build on technically; this is a black box from outside. + +## Data sourcing — not open-data-pipeline, looks substantially manual/crowdsourced + +This is the most consequential finding. There's no public claim of ingesting official open-data portals (no mention of DataSF, NYC Open Data, Seattle's GIS feeds, etc., anywhere in ParkUsher's own material or in coverage of it). What evidence exists points the other way: + +- App Store reviews (per search-engine summaries of the review page) report **streets not filled in and inaccurate data**, specifically called out for NYC's Upper East Side. +- Co-founder "Ali" [Alireza Ziarizi]'s own response to that feedback: the team has an in-app report button, and **the co-founders are personally walking NYC streets to test the map against reality**, with two "big updates" planned including better search. + +That's the signature of a small team doing manual mapping + user-reported corrections, not a pipeline built on top of city open-data feeds the way CURB is built on DataSF/Socrata. It may blend in official sources for the underlying rules (their per-city blog posts, e.g. the [SF street parking guide](https://www.parkusher.app/blog-posts/san-francisco-street-parking-guide), read like they're summarizing SFMTA's own published rules — RPP zones, the 72-hour rule, curb colors, $1–$13/hr meter range), but nothing found confirms an automated, continuously-refreshed feed from any city's official dataset. Freshness and coverage-completeness are asserted ("continuously updated") but not evidenced. + +## Category coverage — real, but gated and city-by-city uneven + +Per the marketing site and store listings, the live map shows: +- **Free tier:** street cleaning/sweeping times, permit zones (RPP/RPZ), general time-restricted areas, plus an AI camera sign-scanner (free, unlimited scans) that reads a posted sign and gives a plain-language yes/no. +- **ParkUsher Pro (paid subscription):** metered/paid-parking layer (the "blue lines" showing priced and live meters), plus the ability to check rules *in advance* rather than only "can I park here right now." + +So sweeping and permits are the free/core product; **meters are a paywalled upsell**, and the free map is explicitly framed as "right now" status rather than a browsable schedule — a materially different product shape than Chalked's "look up any day/time, any city" ambition. + +Coverage is confirmed **uneven across the 7 cities**, not uniform depth: +- Boston, Toronto, NYC have broad-sounding coverage cited by neighborhood (Boston: Beacon Hill, Fenway, Back Bay, etc.; NYC: all 5 boroughs plus named neighborhoods). +- **Vancouver — the newest city — is explicitly partial**: currently limited to downtown/BC Place (per [Vancouver Is Awesome's coverage](https://www.vancouverisawesome.com/local-news/oh-great-another-parking-app-parkusher-says-its-fixing-what-others-miss-12338962), tied to 2026 FIFA World Cup matches at BC Place), with Kitsilano cited as the next neighborhood to be added on the way to eventual full Metro Vancouver coverage. +- No per-city breakdown found of which of sweeping/meters/permits is actually populated where — the NYC review complaint above suggests even a launched city can have real gaps ("streets not filled in"). + +7 cities total, as of 2026: Montreal, New York, Seattle, Boston, Toronto, San Francisco, Vancouver. + +## Company / team — tiny, pre-seed, student-founded + +- Founded 2023 in Montreal by **Alireza Ziarizi** (CEO), a Concordia University software-engineering student, with co-founders **Rayan Moarkech** (CTO), **Lujain Khalaf** (Co-CEO), and **Sevag Eordkian**. +- Launched publicly at **Web Summit Lisbon, Nov 13–16, 2023** ([press release](https://webdev-media-library.s3-accelerate.amazonaws.com/websummit/2323/11/ParkUsher-Press-Release-WebSummit-2023.pdf)). +- Funding: one **pre-seed round** (per [Crunchbase](https://www.crunchbase.com/organization/parkusher)), reported total raised **~$22,400** — effectively unfunded by startup standards, not venture-backed in any real sense. (Tracxn separately shows no funding rounds at all, i.e. even the modest number above isn't universally confirmed.) +- Team size: **3–6 people** across sources (LinkedIn/RocketReach/ZoomInfo disagree slightly), including at least one software-engineering intern. This is a very small team covering 7 cities. +- No evidence of acquisition, shutdown, or major new funding since the 2023 launch; Vancouver (2026) is the most recent city add, suggesting it's still actively expanding, just slowly and thinly. + +## Bottom line: doesn't change the plan — confirms the gap is still real + +ParkUsher is real prior art and genuinely the closest thing to CURB/sweep.la found at multi-city scale — it's the only tool in the whole 29-city scan attempting sweeping + permits (+ paywalled meters) as a real-time layered map across more than one city. But on direct inspection it doesn't function as a reason to deprioritize Seattle or SF the way CURB deprioritized SF on its own: + +- **Not open source, no API, no data to build on or around** — unlike CURB, there's nothing here for Chalked to point users to as "already solved, go use this instead," and nothing to adopt technique-wise (no equivalent of CURB's GPS-matched-citation trick is claimed or evidenced). +- **Closed/paywalled by design** — meters sit behind a subscription; the free product is a live "yes/no right now" status map, not a browsable schedule. That's a different product than what Chalked is building (an open, browsable, linkable rules map), even in cities where ParkUsher is present. +- **Small, thin team; data quality self-reported as shaky** — a 3–6-person pre-seed team manually walking streets to fix bad data is not "solved," it's "attempted, imperfectly, by hand." That's the opposite signal from CURB's ~1M-citation, GPS-matched, publicly-documented dataset. +- **Coverage is visibly uneven even within its own 7 cities** (Vancouver's partial rollout, NYC's reported gaps) — this is evidence *for* Chalked's thesis (comprehensive city-specific coverage is hard and rare) more than evidence the niche is filled. + +Net effect on prioritization: Seattle and SF both stay viable Chalked candidates. SF still has CURB as the actual reason to deprioritize it (per san-francisco.md) — ParkUsher doesn't add a second reason, since it's shallower and closed. Seattle has no CURB-equivalent; ParkUsher's presence there is a data point worth knowing (a commercial competitor exists) but not a blocker — a comprehensive, open, schedule-browsable, citation-backed Chalked adapter for Seattle would still be a strictly better artifact than what ParkUsher currently offers there. diff --git a/research/states/california.md b/research/states/california.md index 99d8cd0..880f1a3 100644 --- a/research/states/california.md +++ b/research/states/california.md @@ -4,9 +4,8 @@ Cities researched so far: - [Los Angeles](../cities/los-angeles.md) — the strongest single first-adapter candidate (see SPEC.md → First adapters to build) - [San Francisco](../cities/san-francisco.md) — already excellently covered by CURB, not a Chalked adapter target +- [San Diego](../cities/san-diego.md) — strong open meters data (one clear vendor, IPS Group/Park Smarter), a real sweeping dataset with an unfinished dashboard-trace, a likely permits gap, and open block-geocoded crime data; no CURB-equivalent aggregator found; ~1.4M population No California-specific state-level findings yet (statewide statutes, DMV data, anything that would apply across multiple CA cities rather than one). Add a section here if any turn up — until then this page is just an index of the state's researched cities. Other California cities on the coverage list (Glendale, Pasadena, Santa Monica, West Hollywood — via sweep.la, see [Los Angeles](../cities/los-angeles.md) → Regional context) don't have their own research file yet, since they haven't been independently sourced beyond the sweep.la aggregation. - -**New candidate found via the broad city scan (2026-07-06):** San Diego has a confirmed real, open sweeping dataset (ArcGIS map + queryable data at data.sandiego.gov) — the best-documented open data found in that scan's whole West-region batch. Not yet independently researched beyond that — see [research/city-hub-scan.md](../city-hub-scan.md). diff --git a/research/states/illinois.md b/research/states/illinois.md new file mode 100644 index 0000000..215305b --- /dev/null +++ b/research/states/illinois.md @@ -0,0 +1,7 @@ +# Illinois + +Cities researched so far: + +- [Chicago](../cities/chicago.md) — sweeping, permits, and crime confirmed open on the city's Socrata portal; meters remains a real gap on the government-data side, with an unverified, previously-scraped private concessionaire API as the only known fallback (see the [dashboard-tracing method](../dashboard-tracing-method.md)'s underlying principle applied to a non-ArcGIS citizen map). + +No Illinois-specific state-level findings yet (statewide statutes, IDOT/SOS data, anything that would apply across multiple IL cities rather than one). Add a section here if any turn up — until then this page is just an index of the state's researched cities. diff --git a/research/states/washington.md b/research/states/washington.md new file mode 100644 index 0000000..ee3fd5f --- /dev/null +++ b/research/states/washington.md @@ -0,0 +1,7 @@ +# Washington + +Cities researched so far: + +- [Seattle](../cities/seattle.md) — sweeping's "unconfirmed" status overturned (real SPU-maintained route data exists, just on Seattle's separate ArcGIS-based open-data portal rather than its Socrata one), with a source-confirmed enforcement-is-voluntary caveat; permits (RPZ) and meters spot-checked and holding up as open + +No Washington-specific state-level findings yet (statewide statutes, DOL data, anything that would apply across multiple WA cities rather than one). Add a section here if any turn up — until then this page is just an index of the state's researched cities. diff --git a/schema/common-schema.md b/schema/common-schema.md new file mode 100644 index 0000000..1b946fc --- /dev/null +++ b/schema/common-schema.md @@ -0,0 +1,86 @@ +# Common schema + +The normalized shape every per-jurisdiction adapter converts its source data into, so nothing downstream (the site, future adapters, future contributors) needs to know any one city's particular field names. This formalizes what the three real LA adapters (`scripts/fetch_la_*.py`) already do in practice — see "Prior art already in the repo" below — rather than inventing something new; SPEC.md's "Architecture (draft)" section is the narrative version of this, this file is the concrete one. + +There are two independent things a common schema has to handle, and they don't collapse into each other: **per-feature data** (a specific block, zone, or point — sweeping/meters/permits/crime) and **jurisdiction-wide default rules** (a rule that isn't tied to any geometry at all — see the Walnut, CA case below). A city can have both simultaneously. + +## Per-feature data: a GeoJSON Feature, plus common fields + +Every category's adapter output is a GeoJSON `FeatureCollection`. Geometry is intentionally untyped at the schema level — **geometry-shape tolerance is a requirement, not an oversight**: LA's sweeping routes and permit districts are polygons, LA's meters are points, SF's blocks (per CURB, see `research/cities/san-francisco.md`) are line segments. The schema doesn't pick a geometry type per category; each adapter uses whatever shape its source data actually is, and any code consuming a category (rendering, point-in-polygon lookups) already has to branch on geometry type regardless — CurbLR made the same call (linear-referenced), and forcing every city into one shape would just relocate the fragmentation problem into the schema instead of solving it. + +Every feature's `properties` carries these fields regardless of category, plus category-specific fields on top: + +```jsonc +{ + "jurisdiction": "Los Angeles, CA", // human-readable, for display/debugging + "category": "sweeping", // "sweeping" | "meters" | "permits" | "crime" + "data_as_of": "2026-07-06", // vintage of the underlying data itself (see below) + "source": { + "name": "LADOT Posted Street Sweeping Routes", + "url": "https://services1.arcgis.com/.../FeatureServer/0", + "last_synced": "2026-07-06T18:04:00Z" // when Chalked's own adapter last successfully ran + } + // ...category-specific fields below +} +``` + +### Two timestamps, not one — they answer different questions + +This is the concrete fix for the thing that forced this whole section: LA's sweeping and LA's permits are both "successfully synced today," but one is genuinely current and the other is frozen since 2015. A single "last updated" field can't tell those apart, so there are two: + +- **`source.last_synced`** — when Chalked's *own adapter* last ran successfully. Answers "is Chalked's copy stale relative to its own source." This is always fresh for a working sync job, even against a source that never changes. +- **`data_as_of`** — the vintage of the *underlying data itself*, as the source claims it. Answers "how current is this fact, really." For a live-edited source (LA sweeping), this tracks `last_synced` closely. For a frozen source (LA permits, `rowsUpdatedAt` stuck at 2015-08-13 despite a claimed annual refresh), this stays put even though `last_synced` updates every sync run — which is the honest behavior: automation can't manufacture freshness the source doesn't have, only report the lack of it (SPEC.md's "Data pipeline" section makes the same point in prose). + +**Both are required per feature, not just per adapter** — a future city might have per-block heterogeneous freshness (some zones added last month, some from a decade-old base layer merged in), and collapsing to one timestamp per adapter run would hide that. + +### Category-specific fields (per existing adapters — `scripts/fetch_la_*.py`) + +| Category | Fields (beyond the common ones above) | Nullable? | +|---|---|---| +| **sweeping** | `route_id`, `day_of_week`, `start_time`, `end_time`, `weeks_of_month[]`, `side_of_street`, `maintenance_district`, `maintenance_district_name`, `route_type` | Every field independently nullable — LA's own adapter already skips routes with unparseable day/time/week fields rather than guessing (see `fetch_la_sweeping.py`'s `skipped` counter); a city with a partial schedule (e.g. day known, hours not) should carry the nulls through, not drop the feature. | +| **meters** | `space_id`, `blockface`, `meter_type`, `rate_type`, `rate`, `time_limit`, `schedule` | `schedule` is `null` for LA specifically — the source has no operating-hours field at all, not an adapter gap. A city whose meter data *does* include hours populates it; the field exists in the schema either way so the UI can ask "is this null because the city doesn't publish it, or because nobody's wired it up yet" (see status enum below, which answers that at the *category* level; a null field is the *feature*-level version of the same honesty). | +| **permits** | `district_number`, `district_name` | `data_as_of` (common field, above) is what actually carries the 2015 staleness today — see `renderPermitPanel` in `app.js`. | +| **crime** | Not yet defined — **paused**, see `ETHICS.md` and [Discussion #1](https://github.com/inkxel/chalked/discussions/1). Deliberately not speced here; whatever categories/aggregation get decided should come out of that discussion, not be pre-empted by a schema written before it. | + +## Category status: six states, not a binary + +The coverage registry (`data/coverage_registry.json`, see `schema/coverage-registry.md`) needs to say more than "built or not" per category per jurisdiction — SPEC.md's open question ("this category doesn't meaningfully apply here" vs. "no data yet") needed an actual answer, not just a note that it needed one. Six states: + +| Status | Meaning | Example | +|---|---|---| +| `built` | A real adapter exists, features are live in the database. | LA sweeping/meters/permits | +| `in_progress` | A contributor has claimed this category and is actively building it — not yet merged, but not open for someone else to duplicate either. Exists mainly to serve the contribution workflow (`CONTRIBUTING.md`): claim by opening this status via PR, avoid two people building the same adapter unknowingly. | (none yet — the first real use of this status will be whoever picks up the next adapter) | +| `gap` | Checked directly, no open data exists for this category in this jurisdiction — confirmed absence, not an assumption. | Chicago meters (private concessionaire, no open API) — pending the municipal-code-hosting-style re-check other research in this repo is doing | +| `not_applicable` | The category doesn't meaningfully exist as a concept here — there's nothing to build, not a data gap. | Street sweeping in most Sunbelt cities (`research/city-hub-scan.md`); NYC's permit-zone model (NYC uses alternate-side parking, not RPP) | +| `unconfirmed` | Nobody has actually checked yet — the honest default, not "gap." Distinct from `gap` specifically because LA's own two "gap" calls both turned out to be `unconfirmed`-that-was-actually-`built`, once someone applied `research/dashboard-tracing-method.md`. Don't call something a `gap` on a search-engine-level check alone. | Seattle sweeping, before this round of research | +| `paused` | A deliberate, documented decision not to build this category yet, for reasons other than data availability. | Crime/break-in risk, everywhere — see `ETHICS.md` | + +A jurisdiction's overall map treatment (blue outline vs. gray fill in `app.js`) is derived, not stored: **covered** if any category is `built`, gray otherwise. This means "Chicago minus meters" (`built`, `built`, `gap`, `built`) renders exactly like a fully-`built` jurisdiction at the national-map zoom level — the per-category granularity only becomes visible once you're in that jurisdiction, which is intentional (SPEC.md's open question about not making partial coverage read as broken). + +## Jurisdiction-wide default rules — not tied to any geometry + +The Walnut, CA case (`research/municipal-code-hosting.md`): a citywide overnight-permit-parking rule with **no signage anywhere**, existing only in municipal code, applying to the whole jurisdiction by default rather than to any specific block or zone. This can't be represented as a Feature at all — there's no geometry to attach it to, and forcing one (e.g. "the jurisdiction's own boundary polygon") would wrongly imply it's block-specific like everything else in the schema. + +```jsonc +{ + "place_id": "0683668", // same key as coverage_registry.json, ties it to a jurisdiction + "rule_id": "walnut-overnight-permit", + "description": "Overnight parking anywhere in city limits requires a residential permit — no exceptions, no signage.", + "signed": false, // the load-bearing field: false means "always defer to the posted sign" doesn't apply here, because there IS no sign + "source": { + "name": "Walnut Municipal Code", + "citation": "WMC §10.20.040", // code citation, not a URL -- see schema/coverage-registry.md's municipal-code-hosting note + "url": null + }, + "data_as_of": "2026-07-12" +} +``` + +`signed: false` is the field that actually matters here, not a cosmetic detail — it's the flag that tells the UI this rule needs to be surfaced proactively (can't rely on "check the posted sign," SPEC.md's whole disclaimer safety net, because there's nothing posted). A jurisdiction can have zero, one, or many of these, stored separately from its per-category Feature data — see `schema/coverage-registry.md` for where these live on disk. + +## Prior art already in the repo + +The three real LA adapters (`scripts/fetch_la_sweeping.py`, `fetch_la_meters.py`, `fetch_la_permits.py`) already produce something very close to this shape — `jurisdiction`, `category`, `source.{name,url,last_synced}` are already there verbatim. What this doc adds/formalizes on top of that existing practice: +- `data_as_of` as a **required common field**, not something only the permits adapter happens to set (sweeping/meters' adapters can set it equal to `last_synced` at sync time, since both sources are live) +- The five-state category-status enum, replacing the implicit "gap"/"unconfirmed"/"open" language used loosely across SPEC.md and `research/` +- Jurisdiction-wide default rules as a wholly new, separate concept — nothing in the repo builds this yet, this is the first spec for it diff --git a/schema/coverage-registry.md b/schema/coverage-registry.md new file mode 100644 index 0000000..31ba9ba --- /dev/null +++ b/schema/coverage-registry.md @@ -0,0 +1,48 @@ +# Coverage registry schema + +`data/coverage_registry.json` — the hand-maintained (not generated) map of what Chalked actually has built, per jurisdiction per category. This is the file `app.js` reads to decide blue-outline-covered vs. gray-fill-uncovered on the national map (`research/national-boundary-layer.md`), and the file a new adapter contribution edits to register itself (`CONTRIBUTING.md`). + +## Shape + +```jsonc +{ + "0644000": { // key: place_id — same STATE+PLACE FIPS concatenation + // TIGERweb uses, so it lines up with the national + // boundary layer with no translation step + "name": "Los Angeles", + "state": "CA", + "population": 3898747, // Census-sourced, city proper — see "Why population lives here" below + "categories": { + "sweeping": "built", + "meters": "built", + "permits": "built", + "crime": "paused" + } + } +} +``` + +Each `categories` value is one of the six statuses defined in `schema/common-schema.md` (`built` / `in_progress` / `gap` / `not_applicable` / `unconfirmed` / `paused`). A jurisdiction doesn't need an entry for a category it hasn't looked at yet — an absent key is equivalent to `unconfirmed`, so `research/city-hub-scan.md`-style discovery-pass cities (scanned but not deeply researched) don't need placeholder entries for categories nobody's touched. + +## What changed from the original shape + +The original registry (pre-2026-07-12) had a single top-level `status: "supported"` field plus a flat `categories: [...]` array of only the *built* categories: + +```jsonc +// old shape — superseded +{ "0644000": { "name": "Los Angeles", "state": "CA", "status": "supported", "categories": ["sweeping", "meters", "permits"] } } +``` + +This worked for LA specifically because LA had no `gap`/`not_applicable`/`paused` categories worth distinguishing from "just not built yet" — everything not in the array was implicitly "no." It stops working the moment a jurisdiction needs to say "meters is a confirmed gap, not just unbuilt" (Chicago) or "sweeping doesn't apply here" (a Sunbelt city) — a flat list of built categories can't carry that distinction, which was SPEC.md's own open question. The top-level `status` field is also now redundant: "is this jurisdiction covered at all" is derivable (`any category === "built"`) rather than a second source of truth that could drift out of sync with the categories themselves. + +## Why population lives here, not somewhere else + +SPEC.md's "First adapters to build" prioritization is explicitly population × data-completeness (soft-weighted, not a hard cutoff). Population needs to be queryable per jurisdiction to support a future "sort by most-needed" contributor-facing view (SPEC.md's open questions list) — and the same Census TIGER/Line source already feeding the national boundary layer publishes place-level population estimates, so this isn't a second data source to integrate, just a field to carry through into the one hand-maintained file that's keyed by the same `place_id`. + +## Jurisdiction-wide default rules live in a separate file, not here + +`schema/common-schema.md`'s jurisdiction-wide default rules (the Walnut, CA case) are keyed by the same `place_id`, but intentionally **not** folded into `coverage_registry.json` — that file's whole shape is "which categories are built," and a jurisdiction-wide rule isn't a category, it's an orthogonal fact that can exist whether or not any category is built at all (a city could have zero adapters and still have a known unsigned citywide rule, if the error-reporting pipeline surfaces one before anyone's built a real adapter there). These belong in a future `data/jurisdiction_rules.json`, one entry per rule, `place_id`-keyed the same way — not built yet, since no real jurisdiction-wide rule has been sourced to code-citation depth yet (Walnut's is documented in prose in `research/municipal-code-hosting.md` but hasn't gone through the municipal-code-hosting-check `CONTRIBUTING.md` describes). + +## Migration note + +LA's entry was migrated to the new shape as part of this change — see the diff in `data/coverage_registry.json`. No other jurisdictions had entries yet, so there was nothing else to migrate. diff --git a/schema/error-report-pipeline.md b/schema/error-report-pipeline.md new file mode 100644 index 0000000..366d8df --- /dev/null +++ b/schema/error-report-pipeline.md @@ -0,0 +1,37 @@ +# Error-report → GitHub Issue pipeline + +Design for SPEC.md's "User-reported errors, Google-Maps-style, auto-routed to GitHub Issues" section, including its explicit open requirement: don't ship the naive "instant anonymous direct post" version without an anti-spam/moderation step. + +## The anti-spam decision: no backend, no anonymous posting — that IS the moderation step + +Chalked is a static site (`index.html` + `app.js`, served with `python3 -m http.server` locally, no server-side component). Auto-filing an anonymous report straight to the GitHub Issues API requires either a committed API token shipped to every browser (a straightforward credential leak — anyone could extract it from the page source and spam-file issues, or worse, use it for anything else that token can do) or a serverless proxy function (real infrastructure this project doesn't have yet, and a whole new abuse surface — rate limiting, CAPTCHA, moderation queue — to build and maintain just to enable anonymous posting). + +**The actual implementation: a prefilled GitHub issue-creation link, not an API call.** `app.js` builds a URL like: + +``` +https://github.com/inkxel/chalked/issues/new?template=data-issue.yml&title=...&jurisdiction=...&category=... +``` + +GitHub's own issue-form prefill feature (query params matching each form field's `id`) fills in what Chalked already knows (jurisdiction, category, a starting title) — the user reviews, adds what they observed, and submits themselves, through GitHub's own UI, under their own GitHub account. + +This isn't a workaround for lacking a backend — **requiring a real GitHub account to submit is the anti-spam/moderation step**, not a placeholder for one. It's the same posture Google Maps' "report a problem" flow has implicitly (tied to a Google account) and it costs nothing to build or run: no token to leak, no rate-limiter to maintain, no moderation queue to staff, and GitHub's own spam/abuse tooling (account-level, not Chalked's problem) already applies to whatever gets filed. The tradeoff — a genuinely anonymous user can't report anything — is the right one at this project's current size; revisit only if "needs a GitHub account" turns out to meaningfully suppress real reports once there's actual traffic to observe. + +## Two report types, two templates + +SPEC.md calls for labeling by jurisdiction + category, in a label family (`data-issue`) distinct from the `epic`/`ready`/`blocked` planning labels already in use — see `.github/ISSUE_TEMPLATE/`: + +- **`data-issue.yml`** — the general case: a shown status/schedule/zone that's wrong. Fields: jurisdiction, category (dropdown: sweeping/meters/permits/crime/other), what was observed, optional photo/link. Applies the `data-issue` label automatically (GitHub creates the label on first use if it doesn't already exist — no repo-settings step required to ship this). +- **`unsigned-rule.yml`** — the Walnut, CA case (`research/municipal-code-hosting.md`): a real rule with no posted sign at all, so "always defer to the sign" doesn't apply. Fields: jurisdiction, the rule itself (as specific as the reporter can make it), an optional municipal-code citation. Applies `data-issue` + `unsigned-rule`. This is the pipeline extension SPEC.md's Next Steps calls for separately ("Extend the error-report pipeline to accept 'my city has an unsigned rule like X' as its own report type") — implemented as a second template rather than a branching field on the first, since the two report shapes genuinely don't share fields (one has a category dropdown and a location, the other has a code citation and no category at all). + +City/category-specific labels (`city:los-angeles`, `category:permits`) are deliberately **not** auto-applied by the templates — a single template covers every jurisdiction/category combination, so there's no clean way to prefill those without per-jurisdiction template duplication. A maintainer adds them during triage instead; this is a manual step today; automating it (e.g. a GitHub Action that labels based on the jurisdiction field's text) is a reasonable future improvement once report volume justifies it, not needed for a first version. + +## Where the link appears in the UI + +Each of the three category status panels (`renderPanel`/`renderMeterPanel`/`renderPermitPanel` in `app.js`) gets a "Report a problem →" link, built by a shared `reportIssueUrl(category, jurisdictionLabel, contextHint)` helper — jurisdiction and category arrive prefilled since the app already knows them from whichever zone/point the user clicked; only "what did you observe" is left for the user to fill in. + +The unsigned-rule template isn't linked from any specific panel (there's no zone to click for a rule with no geometry at all) — it's reachable from the disclaimer/footer area instead, alongside the general "not covered yet — help us add it" link, since both are entry points a user reaches without having clicked a specific data feature. + +## What this doesn't solve yet + +- No moderation *within* GitHub Issues itself once filed — a filed issue is public immediately, same as any other GitHub issue. Acceptable at this size (same trust model as any other open-source repo's issue tracker); would need real triage tooling (auto-close obvious spam, require maintainer label before it's "actionable") if volume grows. +- No structured way to turn a resolved `data-issue` into an actual data fix yet — today that's still a human reading the issue and updating the relevant adapter/dataset by hand. A future improvement, not blocking this pipeline's usefulness today. diff --git a/scripts/fetch_la_meters.py b/scripts/fetch_la_meters.py index e280cd6..eb711bd 100644 --- a/scripts/fetch_la_meters.py +++ b/scripts/fetch_la_meters.py @@ -62,6 +62,9 @@ def transform(records, synced_at): "rate": r.get("raterange"), "time_limit": r.get("timelimit"), "schedule": None, # honestly absent from the source -- see module docstring + # data_as_of == last_synced: LADOT's own inventory is the live source of + # truth here, no separate vintage to track -- see schema/common-schema.md. + "data_as_of": synced_at, "source": { "name": "LADOT Metered Parking Inventory & Policies", "url": SOURCE_URL, diff --git a/scripts/fetch_la_sweeping.py b/scripts/fetch_la_sweeping.py index e03edc4..0882af8 100644 --- a/scripts/fetch_la_sweeping.py +++ b/scripts/fetch_la_sweeping.py @@ -78,6 +78,11 @@ def transform(raw_geojson, synced_at): "maintenance_district": props.get("Maint_District"), "maintenance_district_name": props.get("MD_Name"), "route_type": props.get("Route_Type"), + # data_as_of == last_synced here on purpose: this source is actively + # edited (see SPEC.md's LA correction), so the data's own vintage IS + # today's sync -- unlike permits, where the two diverge. See + # schema/common-schema.md for why both fields exist. + "data_as_of": synced_at, "source": { "name": "LADOT Posted Street Sweeping Routes", "url": SOURCE_URL, From 3e19f9b6f4b1086cbf106004e7900a16abb120e0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 12 Jul 2026 17:13:25 +0000 Subject: [PATCH 2/3] Enforce the common schema in CI and automate issue-label triage Two of the five open items from the last round were achievable without network access; the other three (a second real adapter, live-verifying Seattle/San Diego, and the Walnut jurisdiction-rule entry) are re-confirmed still blocked on the same restrictive sandbox network policy -- checked directly again rather than assumed. - scripts/validate_schema.py: checks adapter GeoJSON output against schema/common-schema.md's required fields, and coverage_registry.json against schema/coverage-registry.md's status enum. Turns the schema docs into something enforced, not just documented. - .github/workflows/lint.yml: runs the validator (registry mode), Python syntax, YAML validity, and app.js syntax on every PR. - .github/workflows/label-data-issues.yml: parses new data-issue/unsigned-rule reports for their Jurisdiction/Category fields and applies city:*/category:* labels automatically, closing the manual-triage step schema/error-report-pipeline.md flagged as a future improvement. - CONTRIBUTING.md: documents the validator as a pre-PR step and the new CI. - SPEC.md: updated Next Steps to reflect what closed vs. what's still genuinely blocked, with the network check re-run and cited directly. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01YbWncp7hgMyXKDvX4oUcBM --- .github/workflows/label-data-issues.yml | 88 +++++++++++++ .github/workflows/lint.yml | 40 ++++++ CONTRIBUTING.md | 15 +++ SPEC.md | 11 +- scripts/validate_schema.py | 165 ++++++++++++++++++++++++ 5 files changed, 315 insertions(+), 4 deletions(-) create mode 100644 .github/workflows/label-data-issues.yml create mode 100644 .github/workflows/lint.yml create mode 100644 scripts/validate_schema.py diff --git a/.github/workflows/label-data-issues.yml b/.github/workflows/label-data-issues.yml new file mode 100644 index 0000000..6b46728 --- /dev/null +++ b/.github/workflows/label-data-issues.yml @@ -0,0 +1,88 @@ +name: Auto-label data-issue reports + +# Automates the triage step schema/error-report-pipeline.md flagged as manual by design +# ("automating it is a reasonable future improvement once report volume justifies it") -- +# cheap enough to build now that there's no reason to keep doing it by hand. +# +# Parses the issue-form body for the "Jurisdiction"/"Category" fields (data-issue.yml and +# unsigned-rule.yml both have a Jurisdiction field; only data-issue.yml has Category) and +# applies city: / category: labels -- the label family SPEC.md's "Trust, error +# reporting & disclaimer" section calls for, distinct from epic/ready/blocked planning labels. +# Doesn't touch issues that aren't from one of these two templates (scoped via the +# data-issue label the templates already apply on creation). + +on: + issues: + types: [opened, edited] + +permissions: + issues: write + +jobs: + label: + if: contains(github.event.issue.labels.*.name, 'data-issue') + runs-on: ubuntu-latest + steps: + - name: Extract jurisdiction/category and apply labels + uses: actions/github-script@v7 + with: + script: | + const body = context.payload.issue.body || ""; + + // Issue-form fields render as "###