Skip to content

chore: feature integration branch for the ENVITED-X pipeline - #3

Draft
jdsika wants to merge 1 commit into
mainfrom
feat/envited-x-pipeline
Draft

jdsika wants to merge 1 commit into
mainfrom
feat/envited-x-pipeline

Conversation

@jdsika

@jdsika jdsika commented Sep 14, 2026 •

Copy link
Copy Markdown

Integration branch. Tracking only, do not merge.

What this branch is

feat/envited-x-pipeline is upstream RDFLib/rdflib@main with every contribution this
organisation is still carrying cherry-picked on top, and nothing else. It is the branch to
check out when you need an rdflib that behaves the way our pipeline expects before
upstream has merged the relevant PRs.

It is rebased on upstream main, never merged into. An empty cherry-pick list is the goal
state: it would mean upstream has merged everything, and the branch could be deleted.

Same convention as ASCS-eV/linkml#14.

What it currently carries

Commit Upstream PR Fork mirror Change
be5ea8fb RDFLib/rdflib#3543 #2 JSON-LD collection syntax can silently duplicate shared RDF lists

Three of the four changes this branch used to carry are now merged upstream and have
been dropped from it:
#3546 (deterministic N-Triples via an opt-in
canon parameter), #3564 (SPARQL IN /
NOT IN value equality) and #3574 (relative
IRI resolution against a base without its fragment).

The branch is rebased onto upstream main at 3c4501c2, so it is now a single commit,
patch-identical to its upstream PR head, applying with no conflicts.

Last verified on this branch at be5ea8fb, Python 3.12: black, ruff and mypy are clean for
the whole project, and the full test suite gives 8096 passed, 0 failed.

What consumes it

Nothing yet, deliberately. asam-openx-standards pins released rdflib==7.6.0
through scripts/requirements.txt, and diffable-rdf depends on released rdflib too.

That pin is the point: rdflib performs the final serialization of every generated OWL and
SHACL artifact, so a change here moves artifact bytes. Consuming this branch would mean
building from a fork, which the pipeline's reproducibility rules do not allow: the lock
records released versions precisely so a third party can obtain them.

The relationship is therefore one-directional. What lands upstream here eventually arrives
as a released rdflib that the pipeline can bump to, with regenerated artifacts in the same
change. The merged #3504 (the Turtle variant
of the RDFLib#3543 defect) is the precedent: diffable-rdf had to work around it until it was
released.

When a release contains RDFLib#3543, diffable-rdf's sentinel test
test_rdflib_jsonld_serializer_does_not_preserve_shared_list_tail will fail by design;
that is the signal to retire the workaround.

Converter.to_collection walked an rdf:List chain checking only that each cell
carries one rdf:first/rdf:rest pair and that the chain is acyclic. It did not
check how many statements point into the chain.

The @list form inlines the whole chain at a single reference and writes its
cells nowhere else, so when a cell is referenced from more than one place each
referring list rendered the shared cells again. The round trip then gained
triples:

    _:tail rdf:first "b" ; rdf:rest rdf:nil .
    _:head rdf:first "a" ; rdf:rest _:tail .
    ex:s1  ex:p _:head .
    ex:s2  ex:p _:tail .            # second reference into the chain

serialized as two independent @list values, so the tail was emitted twice:
6 triples in, 8 out, not isomorphic. A head referenced twice was duplicated
the same way: 4 in, 6 out.

The Serialize RDF as JSON-LD Algorithm (JSON-LD 1.1 Processing Algorithms and
API, section 8.4) only turns a chain into @list when every cell, the head
included, is a blank node that is "referenced once", counted over the whole
dataset. The converter now applies that condition. It counts blank-node
references once per conversion, over every graph it writes, and a chain with
a cell referenced more than once is written with explicit rdf:first/rdf:rest,
which can express any arrangement of sharing. Counting across graphs also
covers a cell that another named graph points at, whose reference was
otherwise left dangling.

add_to_node offered a term declared with "@container": "@list" for any object
with an rdf:first, then iterated to_collection without checking for None. That
already raised TypeError for a chain the converter refuses, such as one whose
cell carries an extra property, and would have for every shared chain. The
term is now chosen from to_collection's result instead.

This is the defect RDFLib#3504 fixed in the Turtle and longturtle serializers; the
JSON-LD converter has its own chain walk.

Fixes RDFLib#3542.
@jdsika
jdsika force-pushed the feat/envited-x-pipeline branch from 46f0890 to be5ea8f Compare October 23, 2026 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant