Repository navigation
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3991 +/- ##
==========================================
+ Coverage 81.62% 85.09% +3.46%
==========================================
Files 162 162
Lines 20518 20838 +320
Branches 4366 4442 +76
==========================================
+ Hits 16748 17732 +984
+ Misses 2876 2170 -706
- Partials 894 936 +42
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
608f99c to
7197a7e
Compare
|
This looks good, but I want to hold off on hitting approve until the PRs lower on the stack are ready to go, so that this doesn't just bring in the whole bundle. |
442dd51 to
6a78f26
Compare
|
Thanks @kevinschaper. Since your look, I've rewritten the section to match the final behaviour of #3989 and #3990: the composed translation, the presence and every-value semantics, and numeric bounds. I also corrected four statements, listed in the description. The example is now generated by |
83b92de to
029c4c2
Compare
029c4c2 to
4377a5c
Compare
The rules-to-SHACL-SPARQL converter recognised two named patterns. This
adds the presence-implies-value pattern: a precondition asserting
`value_presence: PRESENT` on one slot, and a postcondition constraining
another slot with `equals_string` or `equals_string_in`. It reads as
"if the guard slot is present, the target slot must be present and hold
one of the allowed values", and generalises the boolean guard to
arbitrary enum values. The boolean guard becomes the case
`equals_string: "true"` of this pattern on an xsd:boolean-typed flag.
`equals_string` / `equals_string_in` compare strings. The metamodel
defines them for slots of range string; enums and types with datatype
xsd:string are treated alike, and a slot without a range takes the
schema's default_range. On an xsd:boolean-typed slot each value must be
a lexical form of xsd:boolean (XML Schema 1.1 Part 2 section 3.3.2.2:
true, false, 1, 0) and denotes that boolean. A rule applying them to
any other range is skipped with a warning: its RDF values are typed
literals or IRIs that equal no string. Built-in type names resolve
without importing linkml:types, as in the main slot loop; the built-in
name of `curie` was misspelt there and is corrected.
Values are compared with `=`, spelled out as the disjunction SPARQL 1.1
section 17.4.1.9 defines `IN` to be, so the RDF 1.1-identical plain and
xsd:string literal forms both match and booleans compare by value;
rdflib evaluates `IN` and triple-pattern constants by term identity.
COALESCE turns the type error that RDFterm-equal raises for
incomparable literals into false, so a spec-conformant engine reports
such a value instead of dropping the row. Queries are DISTINCT and
project ?path (and ?value where there is one), so each result names the
property and the offending value. A value containing a backslash
followed by "u" or "U" is split with CONCAT, since codepoint escapes are
replaced before a query is parsed.
The exclusive-value pattern takes the precondition `has_member:
{equals_string: V}`, which states "V is one of the values". A bare
`equals_string: V` applies to every value of the slot, as the
specification applies a slot constraint to all members of a collection
and as gen-json-schema reads it, so it no longer matches this pattern:
such a rule is skipped with a warning naming has_member. The pattern
gets the same range check and value comparison, and its
maximum_cardinality > 1 form tests HAVING on the COUNT aggregate:
expressions projected by SELECT are not visible to HAVING (SPARQL 1.1
section 18.2.4.2), so `HAVING (?count > N)` never held and the
constraint never fired.
A rule applies to all members of its class, so a class shape now
carries the rules of its ancestors and mixins, as the JSON Schema
generator applies them. Each rule is translated in the inheriting
class's context, where slot_usage may refine the slots it references.
Classes sharing a class_uri share one shape, which carries each rule
once. With `open_world: true` "the postconditions may be omitted in
instance data", so an absent target is no violation.
Skip warnings name the rule by its position and its title, or its
description without one.
A rule whose conditions carry any operator beyond what a pattern
translates, or name an unknown or identifier slot, is skipped rather
than partially translated. Each problem with a rule is logged once as a
warning naming the declaring class, the rule's position and the class
shapes it affects. The metadata fields the operator accounting ignores
are derived from the metamodel. Slot resolution goes through induced
slots, so slot_usage overrides, slot_uri overrides and alias-form keys
resolve to the IRI sh:path emits.
Behaviour changes for existing schemas: an exclusive-value rule with a
bare equals_string precondition is skipped (write has_member:
{equals_string: V}); the boolean guard rejects a string "true" on an
xsd:boolean flag and skips a flag whose type has another datatype IRI
(main compared with str()); exclusive-value rules with
maximum_cardinality > 1 now fire; inherited rules are now enforced on
subclass shapes; problems with rules are logged at WARNING instead of
DEBUG; rule results carry sh:resultPath and sh:value; sh:message follows
the rule's in_language or default_language.
Co-authored-by: jdsika <carlo.van-driesten@vdl.digital>
Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
…ersion A rule that no named pattern matches is now composed from the operators it uses: each precondition becomes SPARQL filters on $this, and each postcondition slot gets its own constraint, the union of the ways to violate that slot's condition, binding the offending value to ?value (SHACL sections 5.3.1 and 5.3.2). A rule P -> A and B is violated exactly when P and not A, or P and not B, holds. A condition may use value_presence, required, equals_string, equals_string_in, equals_number, pattern, minimum_value, maximum_value, the slot-level any_of, all_of, exactly_one_of and none_of over those value tests, a range_expression on the slot's class (nested to any depth) and has_member. The semantics follow the JSON Schema generator's if/then: - presence: value_presence decides, then required, then a default: a precondition requires its slot, a postcondition unless the rule is open_world, an inner condition of a nested expression never; - value operators and range_expression apply to every value of the slot (05validation.md: "all members"), has_member to some value; - equals_string(_in) only on string, enum and boolean slots (the named patterns' range check); equals_number and the bounds only on the numeric datatypes of SPARQL 1.1 section 17.1, guarded with isNumeric, and only finite numbers; pattern as REGEX on the value's string form, as sh:pattern reads it (SHACL section 4.4.2); - a value is one term, so each member of a slot-level boolean is true or false, and the operators are those of 05validation.md: exactly_one_of counts the members that hold. Any other operator, or a rule with empty pre- or postconditions, skips the rule with one warning, so a rule is never partially translated. A rule skipped part-way no longer reports the notes of the abandoned translation. The compliance suite now runs SHACL in the rule tests, and every rule of test_preconditions and test_preconditions_combos is translated, so each instance is checked. Co-authored-by: jdsika <carlo.van-driesten@vdl.digital> Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
The SHACL generator translates LinkML rules into sh:sparql constraints, but its documentation did not mention them. Describe the two named patterns and the composed translation with the operators it supports and one constraint per postcondition slot, the semantics shared with the JSON Schema generator (presence, every value versus has_member, value comparison, numeric bounds), inheritance, open_world, --no-emit-rules and the skip warnings, with an example generated by the generator. Replace the dangling "See above for implementation status" in the rules section of advanced.md with links to the JSON Schema and SHACL generator pages. Signed-off-by: jdsika <carlo.van-driesten@vdl.digital>
4377a5c to
170b2fb
Compare
Summary
This PR documents the translation of LinkML rules into SHACL-SPARQL in
docs/generators/shacl.rst, which did not mention it. It also replaces the dangling "See above for implementation status" in the rules section ofdocs/schemas/advanced.mdwith links to the JSON Schema and SHACL generator pages.The new section covers:
has_member;meaning, the boolean lexical forms);open_world,--no-emit-rules, and the rule that a rule is skipped with a warning rather than partially translated;gen-shaclfrom this stack, and what its results report (sh:resultPath,sh:value).Corrections to the earlier version
open_worldis honoured.=, notNOT IN.sh:sparqlconstraints withoutadvanced=True.Verification
gen-shacland was regenerated on this stack; its documented results were checked with pyshacl.make htmlpasses withSPHINXOPTS="-W --keep-going", as indocs-test.Documentation only.
Stack: #3989 → #3990 → #3991. This PR should merge last; its diff includes the two commits below it.