chore: refresh ENVITED-X integration with reviewed generator features - #14

…SHACL and JSON Schema

gen-shacl dropped class-level any_of, all_of, exactly_one_of and none_of,
so a class stating "a code or a name is required" generated shapes that
accepted everything.  gen-json-schema translated them, but only for the
declaring class, and read an absent slot differently depending on where
an expression was nested.

Both generators now give class expressions one semantics, shared in
linkml.generators.common.class_expression:

- A class expression constrains every instance of its class, so a class
  carries the expressions of its ancestors and mixins, translated in its
  own context.
- The specification does not say what a slot condition means for an
  absent slot.  As an SQL CHECK constraint (ISO/IEC 9075) reads a
  condition on null, a condition that doesn't decide whether its slot
  may be absent is unknown for an absent slot.  An instance is invalid
  only when an expression is definitely false, and exactly_one_of holds
  when exactly one member is definitely true.  The operators compose,
  and gen-json-schema no longer rejects an instance meeting only the
  first member of an exactly_one_of while accepting one meeting none.

gen-shacl emits sh:or, sh:and, sh:xone and sh:not (the operators'
exact_mappings, SHACL 4.6) over anonymous member shapes, each in its
"not false" form or, under sh:not, its "definitely true" form, where
such a condition also has sh:minCount 1.  Conditions translate required,
value_presence, the cardinalities, minimum/maximum_value, pattern,
equals_string(_in), equals_number and range; a parameter SHACL allows
once per shape that one condition needs twice moves into an sh:and
member.  An operator whose members use anything else is skipped whole,
with one warning naming the shapes it is missing from.

gen-json-schema builds the same forms with anyOf, allOf, oneOf and not,
resolves conditions on the induced slot, and also enforces cardinalities
(minItems / maxItems, or minProperties / maxProperties for a slot inlined
as a dict, whose values are the values of the dict) and range in
class-level conditions.
value_presence: ABSENT on several slots of one expression now requires
each to be absent rather than not all of them present, in rules too.

An is_a member names a class that every instance of the class carrying
the expression belongs to (the class or an ancestor), that only some
belong to (through a descendant), or that none belongs to, since an
instance belongs to its class and that class's ancestors. Only the
second case checks the instance: sh:class in SHACL, and in JSON Schema,
which carries no type, a reference to the class's definition. Before,
a class inherited its parent's any_of over its own subclasses as sh:class
and $ref to itself, which recursed without end in JSON Schema and, in
SHACL, rejected a node typed only as a subclass.

The slot loop's range dispatch and sh:path computation move into
_add_range and _slot_iri; output for schemas without class expressions
is unchanged.  The compliance tests test_class_any_of and
test_class_any_of_with_required run for SHACL.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>

has_member asks for at least one value of a multivalued slot satisfying
an expression. gen-shacl ignored it. gen-json-schema translated it as
contains, which an absent slot passes but an empty one fails, applied
only the value operators of the expression, and ignored it in
class-level expressions.

Both generators now read a member as one value, which must satisfy the
value operators, range and range_expression of the expression. A slot
without a value has no member, so has_member decides presence. In SHACL
it is an sh:qualifiedValueShape with sh:qualifiedMinCount 1; in JSON
Schema, contains on a required slot, or some value of a dict inlined by
key. In a slot condition the member takes the form of the condition.
SHACL skips a has_member it cannot express with a warning, and both
generators warn about one on a slot that is not multivalued.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>

The rules-to-SHACL-SPARQL converter recognised two named patterns.  This
adds the presence-implies-value pattern: a precondition asserting
`value_presence: PRESENT` on one slot, and a postcondition constraining
another slot with `equals_string` or `equals_string_in`.  It reads as
"if the guard slot is present, the target slot must be present and hold
one of the allowed values", and generalises the boolean guard to
arbitrary enum values.  The boolean guard becomes the case
`equals_string: "true"` of this pattern on an xsd:boolean-typed flag.

`equals_string` / `equals_string_in` compare strings.  The metamodel
defines them for slots of range string; enums and types with datatype
xsd:string are treated alike, and a slot without a range takes the
schema's default_range.  On an xsd:boolean-typed slot each value must be
a lexical form of xsd:boolean (XML Schema 1.1 Part 2 section 3.3.2.2:
true, false, 1, 0) and denotes that boolean.  A rule applying them to
any other range is skipped with a warning: its RDF values are typed
literals or IRIs that equal no string.  Built-in type names resolve
without importing linkml:types, as in the main slot loop; the built-in
name of `curie` was misspelt there and is corrected.

Values are compared with `=`, spelled out as the disjunction SPARQL 1.1
section 17.4.1.9 defines `IN` to be, so the RDF 1.1-identical plain and
xsd:string literal forms both match and booleans compare by value;
rdflib evaluates `IN` and triple-pattern constants by term identity.
COALESCE turns the type error that RDFterm-equal raises for
incomparable literals into false, so a spec-conformant engine reports
such a value instead of dropping the row.  Queries are DISTINCT and
project ?path (and ?value where there is one), so each result names the
property and the offending value.  A value containing a backslash
followed by "u" or "U" is split with CONCAT, since codepoint escapes are
replaced before a query is parsed.

The exclusive-value pattern takes the precondition `has_member:
{equals_string: V}`, which states "V is one of the values". A bare
`equals_string: V` applies to every value of the slot, as the
specification applies a slot constraint to all members of a collection
and as gen-json-schema reads it, so it no longer matches this pattern:
such a rule is skipped with a warning naming has_member. The pattern
gets the same range check and value comparison, and its
maximum_cardinality > 1 form tests HAVING on the COUNT aggregate:
expressions projected by SELECT are not visible to HAVING (SPARQL 1.1
section 18.2.4.2), so `HAVING (?count > N)` never held and the
constraint never fired.

A rule applies to all members of its class, so a class shape now
carries the rules of its ancestors and mixins, as the JSON Schema
generator applies them.  Each rule is translated in the inheriting
class's context, where slot_usage may refine the slots it references.
Classes sharing a class_uri share one shape, which carries each rule
once.  With `open_world: true` "the postconditions may be omitted in
instance data", so an absent target is no violation.

Skip warnings name the rule by its position and its title, or its
description without one.

A rule whose conditions carry any operator beyond what a pattern
translates, or name an unknown or identifier slot, is skipped rather
than partially translated.  Each problem with a rule is logged once as a
warning naming the declaring class, the rule's position and the class
shapes it affects.  The metadata fields the operator accounting ignores
are derived from the metamodel.  Slot resolution goes through induced
slots, so slot_usage overrides, slot_uri overrides and alias-form keys
resolve to the IRI sh:path emits.

Behaviour changes for existing schemas: an exclusive-value rule with a
bare equals_string precondition is skipped (write has_member:
{equals_string: V}); the boolean guard rejects a string "true" on an
xsd:boolean flag and skips a flag whose type has another datatype IRI
(main compared with str()); exclusive-value rules with
maximum_cardinality > 1 now fire; inherited rules are now enforced on
subclass shapes; problems with rules are logged at WARNING instead of
DEBUG; rule results carry sh:resultPath and sh:value; sh:message follows
the rule's in_language or default_language.

Co-authored-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>

…ersion

A rule that no named pattern matches is now composed from the operators
it uses: each precondition becomes SPARQL filters on $this, and each
postcondition slot gets its own constraint, the union of the ways to
violate that slot's condition, binding the offending value to ?value
(SHACL sections 5.3.1 and 5.3.2).  A rule P -> A and B is violated
exactly when P and not A, or P and not B, holds.

A condition may use value_presence, required, equals_string,
equals_string_in, equals_number, pattern, minimum_value, maximum_value,
the slot-level any_of, all_of, exactly_one_of and none_of over those
value tests, a range_expression on the slot's class (nested to any
depth) and has_member.  The semantics follow the JSON Schema
generator's if/then:

- presence: value_presence decides, then required, then a default: a
  precondition requires its slot, a postcondition unless the rule is
  open_world, an inner condition of a nested expression never;
- value operators and range_expression apply to every value of the slot
  (05validation.md: "all members"), has_member to some value;
- equals_string(_in) only on string, enum and boolean slots (the named
  patterns' range check); equals_number and the bounds only on the
  numeric datatypes of SPARQL 1.1 section 17.1, guarded with isNumeric,
  and only finite numbers; pattern as REGEX on the value's string form,
  as sh:pattern reads it (SHACL section 4.4.2);
- a value is one term, so each member of a slot-level boolean is true or
  false, and the operators are those of 05validation.md: exactly_one_of
  counts the members that hold.

Any other operator, or a rule with empty pre- or postconditions, skips
the rule with one warning, so a rule is never partially translated.  A
rule skipped part-way no longer reports the notes of the abandoned
translation.

The compliance suite now runs SHACL in the rule tests, and every rule of
test_preconditions and test_preconditions_combos is translated, so each
instance is checked.

Co-authored-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>

Translate explicit instantiates declarations to rdf:type on the emitted
schema resource through one shared helper. Apply the mapping consistently
to schemas, classes, slots, types, enums, and non-literal permissible values.
Literal permissible values cannot be RDF subjects and produce a warning.

Membership types the schema element itself, not its ordinary data instances,
and does not become subclass inheritance. Existing enum representation and
OWL class/individual punning remain intact. No additional flag is needed
because the schema already declares the relationship.

Cover CURIEs, absolute and minted IRIs, literal values, representation modes,
and the distinction between gen-rdf metamodel serialization and gen-owl
ontology assertions. Document the LinkML and OWL specification references.

A resource in the OWL 2 reserved vocabulary (Structural Specification
section 2.4), such as a type emitted as xsd:integer without type objects,
cannot be an OWL 2 DL individual; its instantiates is not asserted, with
a warning. A value that is neither a CURIE nor an IRI would become a
relative IRI and is skipped with a warning too.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>

…nerator

A value of a type is an IRI, a blank node or a literal according to one rule,
linkml_runtime.utils.rdf_terms.type_term, which gen-jsonld-context, gen-owl,
gen-shacl, gen-shex, gen-yarrrml and the RDF dumper and loader now share:

- uri, uriorcurie and the types derived from them are IRIs;
- objectidentifier is an IRI, and nodeidentifier and types with
  uri: rdfs:Resource are IRIs or blank nodes;
- every other type is a literal of its datatype, including a type derived
  from string with uri: xsd:anyURI, which keeps URI references as text.

The rule resolves the inherited, expanded datatype, so its spelling does not
matter, and recognises uri and uriorcurie by name and definition rather than
by from_schema, which merging imports rewrites. A datatype with an undeclared
prefix is an error (xsd, shex and rdfs are known). The dumper writes the
inherited datatype of a derived type and rejects relative values of IRI types
(RDF 1.1 Concepts §3.2); the loader accepts IRIs for node-kind ranges in
strict mode, keeps uri values complete, compacts uriorcurie values to CURIEs
and keeps blank-node labels. gen-shacl gives sh:in values and sh:defaultValue
the slot's term kind.

The --xsd-anyuri-as-iri option of gen-owl and gen-jsonld-context is
deprecated in 1.13.0, has no effect, and is scheduled for removal in 1.14.0.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>

…rm of their range

An annotation whose tag a metaclass declares through instantiates gets the RDF
term that the slot's range gives a slot value (linkml_runtime.utils.rdf_terms):
nodeidentifier, uri and uriorcurie values are IRIs or blank nodes, string
values are text with the resolved language, curie values are xsd:string
literals, and an enum-ranged slot writes the permissible value's text. gen-owl
and gen-shacl share the rule through linkml.generators.common.annotations;
without a declaration both keep their earlier behaviour, and metaclasses that
declare a tag with different terms are an error.

gen-owl applies the same rule to native metadata, so see_also, source and the
mappings are IRIs by the metamodel's ranges rather than by a name check, and
created_on is an xsd:dateTime literal. A link value that is neither an IRI
nor a CURIE, such as a bare element name in see_also, is written as a plain
literal with a warning: a relative IRI would make the whole graph
non-standard RDF and push it onto the canonicalizer's rdflib fallback. OWL 2 §10.1 admits IRIs, anonymous
individuals and literals as annotation values.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>

…l axioms

A slot that implements owl:AnnotationProperty, or whose property IRI is one of
the built-in annotation properties of OWL 2 (Structural Specification §5.5), is
an annotation property: it is declared as such, produces no class restriction
or cardinality axiom (annotations have no logical meaning, §10.1), and its
domain, datatype or class range and super-properties are emitted as annotation
axioms (§10.2). Built-in properties get neither a declaration nor axioms. The
predicate of a declared annotation whose metaclass slot has the role is
declared too.

A slot that implements two roles is an error (§5.8.1), as is an inverse or a
property characteristic, a unique key, a type designator, or a rule or
class-expression condition on an annotation property.

A super-property must have the role of its sub-property: a sub-property
axiom relates two annotation properties (§10.2.1) or two logical
properties (§9.2.1, §9.3.1), so an annotation property under a data or
object property, or the reverse, is an error.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>

Generated contexts use JSON-LD 1.1 constructs: scoped contexts on enum
slots and class-level range overrides, and @Prefix. A JSON-LD 1.0
processor ignores them silently, so datatype coercions are lost. Emit
"@Version": 1.1 first in every generated context. It makes a JSON-LD 1.0
processor stop (JSON-LD 1.1 §1.7), and a JSON-LD 1.1 processor then
requires 1.1 processing (JSON-LD 1.1 API §4.1.2). gen-jsonld embeds the
declaration with the generated context.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>

Describe in the gen-owl documentation what RDFC-1.0 canonicalization
guarantees for standard RDF, why its sequential blank-node numbering can
still produce noisy diffs, and that --diff-stable reduces that churn
without guaranteeing minimal diffs. Note that the option also exists on
gen-rdf, gen-shacl and gen-shex --format rdf, and that graphs that are not
standard RDF, such as gen-shacl --include-annotations output with a
non-CURIE annotation tag, are serialized by rdflib with a warning, where
--diff-stable has no effect.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>

… names

Add an opt-in --normalize-prefixes flag to the OWL, SHACL, JSON-LD
context and JSON-LD generators that renames non-standard prefix aliases
to their well-known names. The names come from the prefixmaps
linked_data context (curated) layered over its merged context, and where
rdflib binds a namespace by default, from rdflib, which serializes the
output (geo, schema, wgs where prefixmaps says geosparql, sc, WGS84).

- Both http://schema.org/ and https://schema.org/ map to 'schema'.
- OWL and SHACL share normalize_graph_prefixes(): phase 1 renames
  schema-declared prefixes, phase 2 cleans up runtime-injected bindings
  without overwriting HTTPS bindings with HTTP variants.
- A standard name already declared for another namespace is skipped
  with a warning.

The flag defaults to off, preserving existing behaviour, and only the
generators that honour it offer it: gen-rdf writes the schema's class_uri
and prefix declarations as literal data and gen-shex's RDF uses rdflib's
names already, so neither emits an alias to rename. The documentation
for the flag is in docs/generators/owl.rst.

test_csv_limit set csv.field_size_limit(20) without restoring it, so any
later test on the same worker that read prefixmaps' CSV data failed; it
now restores the limit.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>

The RDF canonicalizer in linkml_runtime was extracted into diffable-rdf,
and the two copies diverged. Calling the library fixes nine defects of
the in-tree copy, each asserted against both entry points in
test_rdf_canonicalize_defects.py:

- a base IRI ending in # no longer rewrites every IRI sharing its prefix
- a shared rdf:List tail is no longer duplicated
- N-Triples refuses a relative IRI instead of writing a file its own
  parser rejects; N-Triples allows only absolute IRIs
- literals containing U+2028 and other line separators survive sorting
- degraded RDF/XML, degraded Turtle and JSON-LD are byte-identical
  across processes
- every format ends with exactly one newline
- a Dataset is refused instead of silently flattened

diffable-rdf becomes a required dependency of linkml-runtime. It needs
only rdflib and pyoxigraph, which the runtime already requires. The
diff-stable extras remain as aliases. Turtle output of the four RDF
generators is unchanged.

The library logs degraded output; the runtime re-emits those records as
RDFCanonicalizationWarning so they stay visible without logging
configuration.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>

…r features

Consumer checks for features that meet on the integration branch: an any_of
branch keeps its pattern next to a nested range_expression after RDF
serialization, in both RDF label modes; and one OWL resource keeps its
metaclass membership (instantiates) next to its declared IRI-valued
annotation, while a type keeps its membership (gen-owl writes no annotations
on types).

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>

A separate job resolves the pinned OWLAPI 5.5.0 tree with Maven and runs a
Java probe on gen-owl output: OWL 2 DL profile, no unparsed triples, and the
expected annotations kept through a round trip. It checks the documented
DCMI example in four modes and a schema where metaclass membership, a
declared annotation, annotation properties with a super-property and a
built-in IRI, and an ordinary data property meet.

Both profile tests are strict expected failures: gen-owl writes metadata
annotations (dcterms:title, skos:definition, skos:inScheme,
linkml:permissible_values) without declaring their annotation properties,
and asserts membership in external classes without declaring them, which
upstream main does for any schema too. They fail loudly once the
declarations exist.

Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>