All notable changes to this project are documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- Align devDeps pins to the published dsh 0.1.2-alpha.2 line (0.1.1-rc.2 -> 0.1.2-alpha.2); no behavior change to envelope/gating semantics.
- Tests and the loader runner no longer import the
CallIdbrand from@deepseek-ai/dsh-llm(renamed toToolCallIdon host master): the call-id brand is now derived from thedsh-toolsexecution contract, staying green on both the published rc line and the 0.1.2-alpha.1 checkout. - The adaptive-append comments now state the 0.1.2-alpha.1 reality (the
ignorableenvelope was removed, so the probe fails safe and the append is skipped). Behavior unchanged.
- Self-contained offline HTML report.
data_reportgainsformat: html(withkey): a profile or clean report renders as a single.htmlfile with inlined CSS/JS, no external requests (no CDN,<link>, or<script src>), the DAMA six-dimension scorecard, the per-column profile summary table, and the per-rule cleaning summary table. Dataset values are HTML-escaped before reaching the markup. - Versioned report schema. The profile report now carries
schemaVersion(a newREPORT_SCHEMA_VERSION = 1), declared in the profile report's JSON schema so future consumers can detect an incompatible persisted record instead of misreading it.
- DAMA six-dimension quality scorecard in
data_profile: completeness, uniqueness, validity, consistency, timeliness, and accuracy (limited declared-schema definition; accuracy is reported undetermined without a declared schema rather than fabricated). The scorecard flows through the profile report, the persisted storage-domain record, and the tool result. - Full-row sha256 content-hash duplicate detection in
data_profile, reporting the duplicate rate plus a bounded sample of 0-based duplicate row indexes (capped byevidenceRowLimit). - Pre-delivery contract validation summary on
data_clean(dedupe before/after row comparison, uniqueness regression, non-null and type regressions) plus a persisted clean before/after profile diff report under the newclean-diffstorage record kind. data_reporttool plusctx.dataQuality.getReport/listReportsquery methods to read persisted reports back byreportKey(path-safe validation, missing records fail loud) or bykind, ordered chronologically.data_cleandryRunparameter: no file written and no report persisted, returning the per-column cleaning plan (incontract.columnDecisions) plus the expectedcontract/diffPreview.- Numeric distribution
count/distinctfields on numeric column profiles. - Config
scorecardWeights(six per-dimension weights, non-negative, equal by default) driving aweightedOverallscorecard total. - Per-column decision trace in the clean contract (
columnDecisions: strategy + affected rows), persisted with the clean report. - File-encoding reporting in
data_profile(encoding: UTF-8 BOM presence and validity; invalid UTF-8 is reported, not fatal). - Built-in industry profile presets (
retail,saas,fund,real-estate,e-commerce,healthcare,logistics,manufacturing,energy) via thedata_profileindustryPresetparameter, feeding the scorecardaccuracydimension; unknown ids fail loud. - Metric reconciliation on
data_verify: optionalexpectations(rowCount/columnSum/columnMean/uniqueCount/nullCount, each withcolumn,expected, optional relativetolerance) reconcile deterministic computed values; a mismatch is a normalpassed: falseverdict with actual/expected/tolerance detail, while invalid metrics, columns, and out-of-range tolerances fail loud. Results flow into the verify report and its persisted record.
Documented, deliberate non-goals for this repository (recorded rather than silently omitted):
- Native xlsx — a real Excel parser needs a new dependency (or a sidecar); this repo keeps a zero-dependency parsing contract, so CSV/TSV/JSON/JSONL are the supported dataset formats, now with UTF-8/BOM encoding detection.
- SQL / DuckDB access — database drivers are out of scope for the zero-dependency, in-process TypeScript seam; tabular datasets are files.
qsvas an alternative parser — an external binary violates the no-external-processes contract.- Slot client report panel — a browser report panel needs a client half and bundle changes; the consumption path is covered by the
data_reporttool plus storage-domain persistence instead. - Computer-use fallback collection — macOS-only and brittle; the regular filesystem capability already covers the supported collection path.
- Persist the bounded clean
previewin the durablecleanreport so the model-visible canonical result is fully reconstructable from itsreportKey(model-visible ⟺ logged).
- Bumped the
@deepseek-ai/dsh-*dev dependencies to the 0.1.1-rc.2 release line (peers stay>=0.1.0-rc.8 <0.2.0; no rc.2-only API is used) and re-verified the full gate against the published 0.1.1-rc.2 types. Compatibility metadata (dshWorkshop.compatibility.dshVersions) now advertises0.1.1-rc.2, and the CI compat workflow pins thedsh/dsh-base/dsh-headlessinstalls to 0.1.1-rc.2. Session-event behavior is unchanged: 0.1.1-rc.2 still exposes no plugin event-registration surface and noignorableappend option, so the adaptive gate keeps skipping the append and the storage-domain report stays the durable record.
- Bumped the
@deepseek-ai/dsh-*peer and dev dependencies to the 0.1.0-rc.8 release line (peers now>=0.1.0-rc.8 <0.2.0) and re-verified the full gate against the published rc.8 types. Session-event behavior is unchanged: rc.8 still exposes no plugin event-registration surface and noignorableappend option, so the adaptive gate keeps skipping the append and the storage-domain report stays the durable record.
- The
ctx.dataQualitycapability seam (Service Definition / local deterministic Provider / tool Consumers) with the frozen cross-pluginverifyCitations(request)contract for document-number ↔ dataset checks (verified / mismatch / not-found / unverifiable, relative-tolerance numeric comparison). data_profiletool: deterministic dataset profiling (row/column counts, inferred types, missing rates, unique counts, numeric distributions, IQR outliers, mixed-type notes, duplicate-row counts) over workspace CSV/TSV/JSON/JSONL, with optional systematic sampling.data_cleantool: ordered declarative cleaning rules (dedupe,fill-missing,coerce-type,normalize-unit,trim,map-values) with a per-rule audit log; the source file is never overwritten and cleaned output writes only inside the workspace.data_verifytool: declarative verification rules (not-null,unique,range,regex,enum,cross-column,freshness) with capped failing-row evidence; overall failure is a normalpassed: falseresult, never a tool error.- Durable report persistence to the
data_qualitystorage domain (keyed by run timestamp + dataset fingerprint), adaptivedata-quality/profile|clean|verifysession events on hosts that can carry them safely, fail-loud Schemastery config, and workspace-confined path resolution (escape + extension + size + row guards).