Attaching rules to shapes

[afs]

Should we include attaching SHACL (1.2) Rules to shapes?
If so, what does it mean given the difference in execution semantics?

Subsidiary question:
In practice, how often are constriants and AF-rules written on the same shape?
If they are, how are the rules being used, in practice, to influence the validation?

Sketch:

[] rdf:type sh:NodeShape ;
    sh:rule 
      [ a srl:SHACLRule ;
        srl:ruleSet "...";    ## SRL syntax
        sh:prefixes ... ;
      ];

[] rdf:type sh:NodeShape ;
    sh:rule
      [ a srl:SHACLRule ;
        srl:ruleSet [ ... RDF syntax ... ] ;
      ];

Ruleset execution has $this bound to the current node shape. This is available to rule body evaluation:

https://www.w3.org/TR/shacl12-rules/#eval-rule

let SEQ0: Solution sequence = { ("this", RDF Term of current NodeShape) }

Well-formedness changes to compensate for this.

(given the more restricted matching, substitution of $this by it's RDF term should also work)

A problem:
sh:rule (AF) has it's own execution model. We would have to either modify that (turn it off) or make attached rules "run once". The latter option seems pointless because it is then no more than a way to write a CONSTRUCT rule in another syntax.

[HolgerKnublauch]

We (and customers) only have experience with attaching rules to shapes because that was the default supported technique in SHACL-AF. I definitely think this is a useful approach, because it makes it possible to naturally organize rules by grouping them to the focus nodes that they fit to, similar to OO programming where rules play the role of functions attached to a class. This is IMHO better than the potential spaghetti code of flat rule lists.

I guess attaching a rule to a shape means that the focusNode/this iterates over the targets of the shape. If that is technically too difficult for all target types (e.g. targetWhere, targetNode with node expressions) then I think it would be fine to limit the attachment to classes and shapes that have targetClass. This is by far the most common case anyway.

I assume SHACL 1.2 Rules uses its own namespace, so the property doesn't need to be sh:rule.

Disclaimer: I have not tracked the details of the evolving SHACL 1.2 Rules work yet, so I cannot comment on the technical challenges. But clearly, if something is called SHACL Rules then it should have shapes as a first class citizen. Otherwise, it could just become a stand-alone W3C spec.

[afs]

I guess attaching a rule to a shape means that the focusNode/this iterates over the targets of the shape. If that is technically too difficult for all target types (e.g. targetWhere, targetNode with node expressions) then I think it would be fine to limit the attachment to classes and shapes that have targetClass.

It is not too technically difficult for rules because that would seem to be the responsibility of the core traversal algorithm to calculate the focus nodes and pass a set of $this, ideally in one unit and not externally iterating. Rules execution shouldn't extend into the core traversal algorithm but treat it as parameterization.

Do rules get attached mostly/always to shapes that have a sh:target* ¹ or is it not unusual that the shapes with rules are some number of sh:path steps away? All the examples in SHACL-AF have the rules at the target-defining node shapes.

Footnotes

1. Is class sh:Target only for extensions? Why not use it for the domain of all sh:target*? ↩

[liviorobaldo]

I would like to recall this discussion. It concerns my research on using SHACL to validate the Time Ontology. My view is that shapes should validate the graph inferred through SHACL rules (referred to in the current definitions as the "evaluation graph", i.e., the union of the base graph and the inferred graph).

There are use cases, such as the Time Ontology, in which SHACL shapes are not expressive enough to fully validate the ontology, or where doing so is undesirable because the shapes become very complex and therefore harder to edit and debug. In such cases, validation needs to be performed over the evaluation graph.

There was also a related discussion with @TallTed (which I can no longer find) that presented an example where the base graph was invalid, while the evaluation graph was valid. This situation can arise, for instance, when checking constraints under a closed-world assumption. In that case, the graph should be considered valid because the knowledge itself is valid: the knowledge we have at our disposal (and therefore the knowledge that should be validated) is the combination of what is explicitly asserted and what can be logically inferred from it.

In light of these, my answers are:

Should we include attaching SHACL (1.2) Rules to shapes?

Yes.

If so, what does it mean given the difference in execution semantics?

First we execute the rules then the shapes.

In practice, how often are constriants and AF-rules written on the same shape?

I don’t think we should attach a set of rules to a single shape; rather, a set of rules should be associated with a set of shapes.

If they are, how are the rules being used, in practice, to influence the validation?

I've explained how in my view it should work above.

[afs]

@liviorobaldo - Isn't your use case addressed by node expressions (which do act during validation)?

[liviorobaldo]

I don’t think so. As far as I understand, node expressions add expressivity for selecting the nodes to which a shape is applied. For example, one can use a SPARQL query to identify a set of nodes S to which the shape applies. However, each node in S is then validated independently of the others, as in standard SHACL.

For the Time Ontology, I would need to check that all nodes in S share the same characteristic, for instance that there are no two nodes in S associated with different xsd:dateTime values. I don’t think the use of node expressions helps much with this kind of constraints; it mainly adds expressivity for selecting the nodes in S.

Even less can I see how node expressions could help with the problem of checking closed-world constraints mentioned above, namely cases in which the base graph is invalid but the evaluation graph is not.

[afs]

For the Time Ontology, I would need to check that all nodes in S share the same characteristic,

There are number of features in SHACL that might make that possible. e.g. sh:targetWhere and shnex:matchAll.

Please could you distil a concise example to discuss (on a different issue)?

[liviorobaldo]

Well, I'm afraid the example cannot be so simple/concise, because for simple examples SHACL shapes are indeed enough expressive. But suppose we have a shape checking that every student has a tutor:

ex:StudentShape
    a sh:NodeShape ;
    sh:targetClass :Student ;
    sh:property [
        sh:path :has-tutor ;
        sh:minCount 1 ;
    ] .

Suppose in the knowledge graph Alice is not connected to any tutor, which of course means we do not know if she has, according to the open world assumption. The above shape flags Alice as invalid.

However, suppose we have many rules that also access other repositories such as the school registry, teachers’ timetables, and even personal emails (let’s set GDPR aside for a moment 😅), all encoded in RDF.

These rules, after about 20 inference steps, are capable to infer :Alice :has-tutor :Livio. So the current scenario is not invalid, Alice does have a tutor.

I don't think SHACL 1.2 shapes are expressive enough to embody about 20 reasoning steps into, for instance, a SPARQL query, like the one we could define in a node expression. If it is possible... this should be proved: can we formally prove that SHACL 1.2 shapes are capable to embody any sequence of any inference steps? I guess this would amount to formally prove that SHACL 1.2 shapes are Turing-complete or similar (but I'm not an expert on these matters).

Even if it is possible... do we want that? The SHACL shape would be tremendously long and complex, difficult to update and debug. Maybe it would be instead better to split it across a set of rules, each of which calculate some temporary results and altogether the 20-step inference.

The above is one of the closed-world examples I was discussing with @TallTed. In the Time Ontology there is a similar problem, although without closed-world shapes. The ontology contains instances of time:Instant. When these instances are connected by certain properties, their associated xsd:dateTime values must be the same. The challenge is that these properties not only connect instants, so if we have a long chain linking two instants, we must also verify that all nodes along the path are instants. With SPARQL property paths, we can find paths connecting two instants, but verifying that all intermediate nodes are also instants requires separate inference steps that cannot be embodied within SPARQL property paths.

Therefore, my paper defines rules that infer which nodes are instants and which are not, so that we can distinguish the ones for which we need to check the xsd:dateTime values. Further details in the paper I linked above.

As I mentioned, I believe this should always be the approach: first we execute the rules, and then the shapes. Because the knowledge that must be validated is the explicitly asserted knowledge plus all knowledge that can be logically inferred from it.

[afs]

As I mentioned, I believe this should always be the approach: first we execute the rules, and then the shapes.

Why always? That is a bold claim.

As noted above - it's GIGO.

Rules - and validation - are building blocks. How building blocks are combined in an application-level workflow is the responsibility of the workflow.

[liviorobaldo]

Ok, not "always", let me change it into "in the most general case".

What does seem to be always true is that:

"the knowledge at disposal consists of the explicitly asserted knowledge plus all knowledge that can be logically inferred from it"

In cases where SHACL shapes alone are sufficient to validate the knowledge at disposal, one can rely solely on them. In all other cases, rules followed by shapes should be used. That said, this is only my approach: as you point out, this is ultimately the responsibility of the workflow, and I do not mean to mandate any particular approach to anyone ☺️

[afs]

Ok, not "always", let me change it into "in the most general case".

The general case is validate-shapes-1 ; run rules ; validate-shapes-2 where validate-shapes-1 validate basic structural assumptions, like required properties are present, values are expected datatype, no unexpected properties if closed ,... so that rules execute on the expected inputs. Then apply rules. Then validate the higher level domain model with validate-shapes-2.

[afs]

We (and customers) only have experience with attaching rules to shapes because that was the default supported technique in SHACL-AF. I definitely think this is a useful approach, because it makes it possible to naturally organize rules by grouping them to the focus nodes that they fit to, similar to OO programming where rules play the role of functions attached to a class.

I agree that attaching rules to shapes makes sense for organising them.

We could provide just as "organisational" - a fragment of rule set where the fragments are collected up and run together. Each rule fragment has as input the associate set of focus nodes.

Another way would be the analogy to functions on a class work in so far the rule(s) are run one time only. How much of this is covered by node expressions?

What I'm not clear on is what it means to run rule sets attached to shapes - how do the rules interact? e.g. one rule makes use of $this but there are also rules that don't mention $this. Do the latter "see" all the data?

[liviorobaldo]

The general case is validate-shapes-1 ; run rules ; validate-shapes-2 where validate-shapes-1 validate basic structural assumptions, like required properties are present, values are expected datatype, no unexpected properties if closed ,... so that rules execute on the expected inputs. Then apply rules. Then validate the higher level domain model with validate-shapes-2.

Ok, but since rules can only add triples (and not remove them), validating every step could be computationally expensive. However, this approach has the advantage that if the knowledge is invalid, this can be detected before computing the entire transitive closure. Ok, we can say that the most general case is the one you describe.