Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions .github/ISSUE_TEMPLATE/data-issue.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Data issue report
description: Something Chalked shows is wrong — a status, schedule, or zone that doesn't match reality (or the posted sign).
title: "[data-issue] "
labels: ["data-issue"]
body:
- type: markdown
attributes:
value: |
Thanks for flagging this. Every "wrong here" report makes Chalked's data more trustworthy, and — per SPEC.md's framing — a visible backlog of these across cities is itself evidence for why a real national curb-data standard is overdue.

This label family (`data-issue`, plus `city:*`/`category:*` added during triage) is kept separate from the `epic`/`ready`/`blocked` planning labels used elsewhere in this repo, so data-quality reports don't get lost in build-planning noise.

If this is about a rule with **no posted sign at all** (a citywide code-only rule, like the Walnut, CA case in `research/municipal-code-hosting.md`), use the **Unsigned rule report** template instead — this one.
- type: input
id: jurisdiction
attributes:
label: Jurisdiction
description: City/county this report is about.
placeholder: "Los Angeles, CA"
validations:
required: true
- type: dropdown
id: category
attributes:
label: Category
options:
- sweeping
- meters
- permits
- crime
- other/not sure
validations:
required: true
- type: textarea
id: observed
attributes:
label: What did you observe?
description: What Chalked showed, what the posted sign (or reality) actually says, and the location (address, cross streets, or lat/lng).
validations:
required: true
- type: textarea
id: evidence
attributes:
label: Photo or link (optional)
description: A photo of the sign, or a link to an official source, if you have one. You can drag-and-drop an image into this box once the issue is created — GitHub hosts it inline.
30 changes: 30 additions & 0 deletions .github/ISSUE_TEMPLATE/unsigned-rule.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Unsigned / code-only rule report
description: A real parking rule that exists only in municipal code, with no posted signage anywhere — see the Walnut, CA case in research/municipal-code-hosting.md.
title: "[unsigned-rule] "
labels: ["data-issue", "unsigned-rule"]
body:
- type: markdown
attributes:
value: |
Use this when you know about a real, binding parking rule that **isn't posted anywhere** — Chalked's whole "always defer to the posted sign" safety net doesn't help when there's no sign to defer to. See `research/municipal-code-hosting.md` for why this is its own report type, distinct from a signed-zone data error, and `schema/common-schema.md`'s "jurisdiction-wide default rules" section for how it eventually becomes real data.

This is exactly the kind of local knowledge that's genuinely hard to find by scraping — Tucker knew about Walnut's rule because he has in-laws there. If you know a city's rule like this, you're the fastest way Chalked finds out about it.
- type: input
id: jurisdiction
attributes:
label: Jurisdiction
placeholder: "Walnut, CA"
validations:
required: true
- type: textarea
id: rule
attributes:
label: What's the rule?
description: Describe it as specifically as you can — who it applies to, when, where it applies (citywide, or a specific area?), and how you know about it (you live there, you got a ticket, you've read the code yourself).
validations:
required: true
- type: input
id: citation
attributes:
label: Municipal code citation (if known)
description: e.g. "Walnut Municipal Code §10.20.040". See research/municipal-code-hosting.md for how to find a city's code online (usually Municode, American Legal, or General Code) if you want to track it down yourself.
88 changes: 88 additions & 0 deletions .github/workflows/label-data-issues.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
name: Auto-label data-issue reports

# Automates the triage step schema/error-report-pipeline.md flagged as manual by design
# ("automating it is a reasonable future improvement once report volume justifies it") --
# cheap enough to build now that there's no reason to keep doing it by hand.
#
# Parses the issue-form body for the "Jurisdiction"/"Category" fields (data-issue.yml and
# unsigned-rule.yml both have a Jurisdiction field; only data-issue.yml has Category) and
# applies city:<slug> / category:<value> labels -- the label family SPEC.md's "Trust, error
# reporting & disclaimer" section calls for, distinct from epic/ready/blocked planning labels.
# Doesn't touch issues that aren't from one of these two templates (scoped via the
# data-issue label the templates already apply on creation).

on:
issues:
types: [opened, edited]

permissions:
issues: write

jobs:
label:
if: contains(github.event.issue.labels.*.name, 'data-issue')
runs-on: ubuntu-latest
steps:
- name: Extract jurisdiction/category and apply labels
uses: actions/github-script@v7
with:
script: |
const body = context.payload.issue.body || "";

// Issue-form fields render as "### <Label>\n\n<answer>\n\n### <Next label>...".
// Grabs the text between a given heading and the next heading (or end of body).
function extractField(label) {
const re = new RegExp(`### ${label}\\s*\\n\\n([^\\n]+)`, "i");
const match = body.match(re);
return match ? match[1].trim() : null;
}

function slugify(text) {
return text
.split(",")[0] // "Los Angeles, CA" -> "Los Angeles" -- state abbreviation isn't part of the slug
.toLowerCase()
.trim()
.replace(/[^a-z0-9]+/g, "-")
.replace(/^-+|-+$/g, "");
}

const KNOWN_CATEGORIES = new Set(["sweeping", "meters", "permits", "crime"]);

const jurisdiction = extractField("Jurisdiction");
const category = extractField("Category");

const labelsToApply = [];
if (jurisdiction) {
const slug = slugify(jurisdiction);
if (slug) labelsToApply.push(`city:${slug}`);
}
if (category && KNOWN_CATEGORIES.has(category.toLowerCase())) {
labelsToApply.push(`category:${category.toLowerCase()}`);
}

if (labelsToApply.length === 0) {
console.log("No jurisdiction/category field found in the issue body -- nothing to label.");
return;
}

for (const name of labelsToApply) {
try {
await github.rest.issues.createLabel({
owner: context.repo.owner,
repo: context.repo.repo,
name,
color: name.startsWith("city:") ? "c2e0c6" : "bfd4f2",
});
console.log(`Created new label: ${name}`);
} catch (err) {
if (err.status !== 422) throw err; // 422 = label already exists, fine
}
}

await github.rest.issues.addLabels({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
labels: labelsToApply,
});
console.log(`Applied labels: ${labelsToApply.join(", ")}`);
40 changes: 40 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Lint & validate

# Automates the checks this repo has so far been running by hand before every commit
# (see commit history) -- syntax on the Python adapters and app.js, YAML validity on the
# issue-form templates and workflows, and the coverage registry against
# schema/coverage-registry.md via scripts/validate_schema.py. Doesn't touch any external
# network -- data/*.geojson is gitignored (see data/README.md), so there's nothing
# adapter-output-shaped to validate in CI itself; scripts/validate_schema.py's geojson mode
# is for contributors to run locally against their own freshly-fetched output before
# opening a PR (see CONTRIBUTING.md).

on:
pull_request:
push:
branches: [master]

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Python syntax check (scripts/*.py)
run: python3 -m py_compile scripts/*.py

- name: Validate coverage_registry.json against schema/coverage-registry.md
run: python3 scripts/validate_schema.py registry data/coverage_registry.json

- name: YAML validity (issue templates + workflows)
run: |
pip install --quiet pyyaml
python3 -c "
import glob, yaml
for f in glob.glob('.github/ISSUE_TEMPLATE/*.yml') + glob.glob('.github/workflows/*.yml'):
yaml.safe_load(open(f))
print(f'{f}: OK')
"

- name: JS syntax check (app.js)
run: node --check app.js
90 changes: 90 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Contributing to Chalked

Chalked's core engineering work is one adapter per jurisdiction per category — the national map shell, the coverage registry, and the common schema all already exist precisely so a new adapter is the *only* thing a contribution needs to add. This guide is what SPEC.md's Next Steps has been pointing at since the national-shell reframe: "the per-city adapter work becomes the first contributions, proving out the adapter interface... rather than 'the v1 launch.'"

If you're looking for what to work on, start with SPEC.md's "First adapters to build" ranking, or sort `data/coverage_registry.json` by population × missing categories yourself — see "What to build next" below.

## Before you write any code: find the data

Most cities' open-data portals don't advertise everything they actually have. **Read [research/dashboard-tracing-method.md](research/dashboard-tracing-method.md) first** — it's the technique that overturned two of LA's own "this doesn't exist" calls (sweeping and permits both turned out to be real, queryable ArcGIS Feature Services hiding behind citizen-facing dashboards nobody had traced back to source). The short version:

1. Find the citizen-facing dashboard/map for the category you're after (search `<city> street sweeping map`, `<city> parking permit map`, etc.).
2. If it's ArcGIS-based, get the dashboard's item ID from its URL, query `https://www.arcgis.com/sharing/rest/content/items/<id>?f=json` for its Web Map item id, then that Web Map's `/data?f=json` for `operationalLayers[]` — each has a real Feature Service `url` you can query directly.
3. If it's Socrata-based (common for Chicago/NYC/SF-style portals), the API is usually advertised directly on the dataset's own page — check there first.

**Don't conclude "gap" from a search-engine-level check alone.** Mark it `unconfirmed` in the registry (see below) instead, and leave the dashboard-tracing pass for whoever picks it up next — see `schema/common-schema.md`'s status enum for why `gap` and `unconfirmed` are deliberately different things.

If what you're after is a rule with **no dataset and no signage at all** (the Walnut, CA case — see [research/municipal-code-hosting.md](research/municipal-code-hosting.md)), that's a different search: check whether the city's municipal code is hosted on Municode, American Legal Publishing, or General Code before assuming you need a bespoke scrape.

## The common schema

**Read [schema/common-schema.md](schema/common-schema.md)** before writing an adapter — it's short and defines exactly what your adapter needs to output: a GeoJSON `FeatureCollection`, common fields (`jurisdiction`, `category`, `data_as_of`, `source.{name,url,last_synced}`) on every feature, plus category-specific fields (documented per category in that file, matching what the existing LA adapters already produce). Geometry type is untyped on purpose — use whatever shape your source data actually is (polygon, line, point); don't force it into a different shape to match another city's adapter.

The two timestamps matter and aren't interchangeable: `source.last_synced` is when *your adapter* last ran; `data_as_of` is how current the *underlying data* actually is, per the source's own claim. For a live-edited source they're the same. For a source like LA's permit data (frozen since 2015 despite a claimed annual refresh), they diverge — and that divergence is exactly the honest signal the UI's confidence badge (`confidenceBadge()` in `app.js`) surfaces to users. Set both correctly; don't just copy `last_synced` into `data_as_of` without checking whether the source is actually current.

## Worked example: the LA adapters

`scripts/fetch_la_sweeping.py`, `fetch_la_meters.py`, and `fetch_la_permits.py` are the three real adapters in the repo — read whichever is closest to your target category before writing your own. Shape they all follow:

1. **Fetch** — pull raw records from the source (ArcGIS `/query` endpoint or Socrata's REST API), no transformation yet.
2. **Transform** — map the source's field names into the common schema's field names (e.g. LA's `Posted_Day` → Chalked's `day_of_week`). Skip and count (don't silently drop) any record missing a field your transform genuinely needs — see `fetch_la_sweeping.py`'s `skipped` counter for the pattern. Set `data_as_of` deliberately, not by default.
3. **Write** — dump the resulting `FeatureCollection` to `data/<city>-<category>.geojson`. This file is generated, not committed (see `.gitignore` and `data/README.md`) — anyone running the site locally regenerates it by re-running your script.

A contribution PR includes the script (`scripts/fetch_<city>_<category>.py`), not the generated `.geojson` output.

**Before opening the PR, validate your adapter's output against the schema:**

```
python3 scripts/fetch_<city>_<category>.py
python3 scripts/validate_schema.py geojson data/<city>-<category>.geojson
```

This checks structurally what a reviewer would otherwise have to check by eye — every feature has the common fields (`jurisdiction`, `category`, `data_as_of`, `source.{name,url,last_synced}`), a valid `category` value, parseable timestamps, and the category-specific keys documented in `schema/common-schema.md` (present, even if the value is legitimately `null`). It doesn't and can't check that your field *mappings* are correct — that still needs a human read of your transform function against the source's real schema.

## Registering your adapter

Once your adapter works, add or update an entry in `data/coverage_registry.json` — **the one hand-maintained file in `data/`**, and the only registration step needed; the map, its blue/gray styling, and the click-fallback logic all read from this file with no other code changes required. See [schema/coverage-registry.md](schema/coverage-registry.md) for the full shape. Minimally:

```jsonc
{
"0603526": { // place_id: state FIPS + place FIPS (same as TIGERweb)
"name": "Berkeley",
"state": "CA",
"population": 124321,
"categories": {
"sweeping": "built", // what you just built
"meters": "unconfirmed", // honest default for anything you didn't check
"permits": "unconfirmed",
"crime": "paused" // see ETHICS.md -- don't build this without reading it first
}
}
}
```

If you're actively working on a category but haven't merged yet, set it to `in_progress` in a draft PR — this is what that status exists for, so two people don't duplicate the same adapter without knowing it.

## If you're touching the crime/break-in category

Don't, yet — it's deliberately paused pending community input, not an oversight. Read `ETHICS.md` and weigh in at [Discussion #1](https://github.com/inkxel/chalked/discussions/1) before writing any code here.

## If a user reports something wrong before you fix it

Reports come in through GitHub Issues via `.github/ISSUE_TEMPLATE/data-issue.yml` (a specific data point is wrong) or `unsigned-rule.yml` (a rule exists with no posted sign — see `schema/error-report-pipeline.md` for the full design). If you're triaging one, add `city:<jurisdiction>` and `category:<category>` labels during review — the templates deliberately don't auto-apply those, since one template covers every jurisdiction.

## What to build next

SPEC.md's "First adapters to build" section ranks candidates by population × data completeness, softly weighted (a bigger city with a messier gap can still be worth more than a smaller city with clean data). It's deliberately not a strict queue — anyone can pick up any jurisdiction. If you want to sort for yourself: every entry in `data/coverage_registry.json` carries `population`, and every category not yet `built` is fair game.

Two research findings worth knowing before you start:
- LA's own "gap" calls for sweeping and permits were both wrong — always dashboard-trace before writing off a category as unavailable.
- Sweeping isn't a universal primary category — Sunbelt cities show little to no sweeping enforcement (see `research/city-hub-scan.md`), so a Sunbelt adapter's natural lead category may be meters or permits instead. Mark sweeping `not_applicable` there, not `gap` — see `schema/common-schema.md`'s status enum for why that distinction matters.

## CI

`.github/workflows/lint.yml` runs on every PR: Python syntax on `scripts/*.py`, the registry validator above against `data/coverage_registry.json`, YAML validity on the issue templates/workflows, and a syntax check on `app.js`. It can't validate your adapter's actual output (that's gitignored, see `data/README.md`) — run `validate_schema.py geojson` yourself before pushing, per above.

If you're filing a data-issue or unsigned-rule report through the GitHub issue templates, `.github/workflows/label-data-issues.yml` automatically applies `city:*`/`category:*` labels by parsing the Jurisdiction/Category fields you filled in — no manual triage step needed for that part anymore.

## Style

This repo's markdown files write in direct, confident sentences backed by real sources (URLs, dataset IDs, citations) — not hedged filler. Match that. Code comments explain *why*, not *what* — see any existing file in `scripts/` or `app.js` for the tone.
Loading
Loading