Repository navigation
Conversation
42e007b to
71db97f
Compare
60f62c4 to
1b36dc0
Compare
🔍 Adversarial Review — PR #4SummarySolid feature with thorough test coverage, but contains a confirmed race condition between Phase 1 and Phase 2 of 🐛 Bugs & Issues1. Phase 2 overwrites Phase 1 normalization for HTTP/HTTPS variants (Confirmed) When a schema declares
Reproduced locally with rdflib 7.x: from rdflib import Graph, Namespace, URIRef
g = Graph()
g.bind('schema', Namespace('http://schema.org/')) # metamodel
g.namespace_manager.bind('sdo', URIRef('https://schema.org/')) # user
# Phase 1
g.bind('schema', Namespace('https://schema.org/'), override=True, replace=True)
# → schema: https://schema.org/ ✅, schema1: http://schema.org/ (orphan)
# Phase 2
g.bind('schema', Namespace('http://schema.org/'), override=True, replace=True)
# → schema: http://schema.org/ ❌ (HTTPS binding silently lost!)Fix: Phase 2 should check whether # In Phase 2, before rebinding:
current_ns_for_std = dict(graph.namespaces()).get(std_pfx)
if current_ns_for_std and str(current_ns_for_std) != ns_str:
continue # std_pfx already claimed by Phase 1 for a different URI2. The test asserts
|
| PR #1 | PR #4 | |
|---|---|---|
| Approach | Dynamic: {str(ns): str(pfx) for pfx, ns in RdfGraph().namespaces()} |
Static: MappingProxyType({...}) with ~30 entries |
| rdflib dependency | Changes with rdflib version | Frozen, version-independent |
| HTTP/HTTPS variants | Only what rdflib provides | Explicitly includes both |
PR #4's static approach is explicitly designed to solve a problem with PR #1's dynamic approach (rdflib defaults changed between 6.x and 7.x). These will conflict on merge. Recommendation: merge PR #4 first, or rebase PR #1 to use PR #4's static map.
4. Cross-PR conflict — flag forwarding in jsonldgen.py
PR #4 forwards both normalize_prefixes AND deterministic to the embedded ContextGenerator:
for flag in ("normalize_prefixes", "deterministic"):
if hasattr(self, flag):
context_kwargs.setdefault(flag, getattr(self, flag))PR #1 adds deterministic handling in a different location (post-serialization in end_schema). PR #4's forwarding of deterministic creates a dependency on PR #1's field existing. The hasattr guard prevents a crash, but semantically PR #4 shouldn't forward a flag it doesn't define. Either:
- Remove
"deterministic"from the forwarding loop (PR feat(generators): add --normalize-prefixes flag for well-known prefix names #4 doesn't own this flag), or - Document the intentional coupling with PR feat(generators): add --deterministic flag with hybrid RDFC-1.0 + rdflib serialization #1 and establish merge order
5. PR title says fix: but commit says feat: — which is it?
The commit message uses feat(generators): while the PR title uses fix(generators):. This is a new opt-in flag, which is a feature — feat: is correct per conventional commits.
6. Static map maintenance burden
The _WELL_KNOWN_PREFIX_MAP needs manual updates when rdflib adds new curated defaults. The test_matches_rdflib_defaults test catches additions but not removals. Consider adding a reverse check: entries in the static map that are NOT in rdflib's defaults could indicate stale entries.
🧪 Test Coverage Assessment
Strong coverage (18 tests across 7 classes):
- ✅ OWL, SHACL, JSON-LD context normalization (sdo→schema, dce→dc)
- ✅ HTTP/HTTPS schema.org variants
- ✅ Collision detection with warnings (caplog)
- ✅ Cross-generator consistency
- ✅ Static map integrity and copy safety
- ✅ JSONLDGenerator flag forwarding
- ✅ Flag-off backward compatibility
Gaps:
- ❌ No test for
normalize_prefixes+exclude_importsinteraction - ❌ No test for empty schema (zero prefixes) — is
_prefix_remap.clear()safe? - ❌ No test that explicitly sets up both HTTP and HTTPS runtime bindings to exercise Phase 2 overwrite (the scenario from Bug feat(generators): add --deterministic flag with hybrid RDFC-1.0 + rdflib serialization #1)
- ❌ No test for
add_prefixbeing called before__post_init__completes (defensive, but_prefix_remapaccess would crash) - ❌ CURIE remapping test (
test_normalize_prefixes_curie_remapping) only checks thatPerson @idstarts withschema:— doesn't verifyfull_nameslot's@idusesschema:nameinstead ofsdo:name
📋 Fix Plan
- Fix Phase 2 overwrite bug — add a graph-binding check before rebinding in Phase 2 to prevent overwriting Phase 1's normalization
- Add regression test — explicitly set up a graph with both
schema: http://schema.org/andsdo: https://schema.org/, run normalize, and assert the HTTPS binding survives - Remove
"deterministic"from forwarding loop — or gate it behindhasattronly (already done) but add a comment explaining the cross-PR dependency - Fix PR title — change
fix:tofeat:for consistency with the commit - Add empty-schema test — verify no crash when schema has zero prefixes
- Add
exclude_importsinteraction test — verify normalization works when imports are excluded - Strengthen CURIE remapping test — assert that slot URIs (not just class URIs) are remapped in the context
✅ What's Good
- Excellent documentation: the 3-case handling in
visit_schemais well-commented with case numbers - Defensive
_prefix_remapinitialization: correctly placed beforesuper().__post_init__()with clear comment explaining why - Collision detection: warning on skip is the right behavior — no silent data loss
MappingProxyTypefor the static map: prevents accidental mutationwell_known_prefix_map()returns a copy: callers can't corrupt the canonical map- Phase 2 concept is sound: cleaning up runtime-injected orphan bindings is the right idea, it just needs the overwrite guard
- Cross-generator consistency test: ensures OWL, SHACL, and JSON-LD agree — excellent
- Backward compatibility: flag defaults to off, existing behavior preserved
d9fb738 to
dd0ce0c
Compare
8d098e9 to
ef47b47
Compare
8dc3ece to
260a97e
Compare
260a97e to
e56ca65
Compare
e56ca65 to
5f17e19
Compare
ce55ccb to
909eb7d
Compare
a4b329c to
32821d0
Compare
32821d0 to
0342283
Compare
Review update from #4: test_csv_limit left the process-wide CSV field size limit at 20, so later tests on the same xdist worker that read prefixmaps' CSV data failed intermittently. Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
0342283 to
316b430
Compare
Review update from #4: test_csv_limit left the process-wide CSV field size limit at 20, so later tests on the same xdist worker that read prefixmaps' CSV data failed intermittently. Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
ff48839 to
9ede52d
Compare
… names Add an opt-in --normalize-prefixes flag to the OWL, SHACL, JSON-LD context and JSON-LD generators that renames non-standard prefix aliases to their well-known names. The names come from the prefixmaps linked_data context (curated) layered over its merged context, and where rdflib binds a namespace by default, from rdflib, which serializes the output (geo, schema, wgs where prefixmaps says geosparql, sc, WGS84). - Both http://schema.org/ and https://schema.org/ map to 'schema'. - OWL and SHACL share normalize_graph_prefixes(): phase 1 renames schema-declared prefixes, phase 2 cleans up runtime-injected bindings without overwriting HTTPS bindings with HTTP variants. - A standard name already declared for another namespace is skipped with a warning. The flag defaults to off, preserving existing behaviour, and only the generators that honour it offer it: gen-rdf writes the schema's class_uri and prefix declarations as literal data and gen-shex's RDF uses rdflib's names already, so neither emits an alias to rename. The documentation for the flag is in docs/generators/owl.rst. test_csv_limit set csv.field_size_limit(20) without restoring it, so any later test on the same worker that read prefixmaps' CSV data failed; it now restores the limit. Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
9ede52d to
29098c8
Compare
Mirror of linkml#3446, same head. Review there; this PR tracks the fork stack.
Warning
Blocked: depends on an unreleased
prefixmaps, and this breaks the Docker build.packages/linkml/pyproject.tomlcurrently carries a git pin:plus
[tool.hatch.metadata] allow-direct-references = trueto permit it.The fix it depends on (linkml/prefixmaps#82 — the
gxprefix mapping to adead Graphite/Synaptica namespace instead of Gaia-X) is merged in the
prefixmaps
mainbranch but has not been released: the latest version onPyPI is still
0.2.6(2024-10-17).This is not just a policy problem. The git pin is baked into the built wheel's
metadata, so the Docker image build fails:
pip installhas to fetch thedependency from GitHub, and the base image has no
git.That is the one failing check on this PR; every other job is green.
Do not merge until
prefixmapscuts a release. Then the pin becomesprefixmaps >= 0.2.8,allow-direct-referencesis removed, anduv.lockisregenerated — both spots are marked
TODO(prefixmaps-0.2.8). The Dockerfailure resolves itself at that point, since the dependency comes from PyPI.
Summary
Adds an opt-in
--normalize-prefixesflag togen-owl,gen-shacl,gen-jsonld-contextandgen-jsonld, which forwards it to its inline context. The flag renames non-standard prefix aliases for well-known namespaces to their conventional names, for examplesdo:→schema:anddce:→dc:. Namespace IRIs never change; only prefix labels do.Where the names come from
well_known_prefix_map()derives the names from prefixmaps: its curatedlinked_datacontext takes precedence over its broadermergedcontext. Where rdflib binds a namespace by default, its name wins, because rdflib serializes the output:geo,schemaandwgswhere prefixmaps saysgeosparql,scandWGS84. There is no hand-written table. Bothhttp://schema.org/(from prefixmaps) andhttps://schema.org/(from rdflib) map toschema. The pin explained above is needed for thegxcorrection in prefixmaps#82.Behaviour
normalize_graph_prefixes(). It renames schema-declared aliases first, then runtime-injected bindings, and never overwrites an HTTPS binding with its HTTP variant.gen-rdfwrites the schema'sclass_uriand prefix declarations as literal data, andgen-shex's RDF already uses rdflib's names, so neither emits an alias to rename.How was this tested?
test_normalize_prefixes.py(27 tests) and three tests intest_jsonldcontextgen.pycover each generator, the HTTP/HTTPS variants, collisions, and agreement between the generators. The full compiler suite passes after rebasing ontomain(12,421 passed).test_csv_limitnow restorescsv.field_size_limit. It left the limit at 20, soprefixmaps' CSV loading failed in any later test on the same pytest-xdist worker. That produced intermittent failures, reproducible by runningtest_csv_limit[20-1]before these tests.Checklist