Literal equality fails for datatypes None and XSD.string

Example:

>>> from rdflib import Literal
>>> from rdflib.namespace import XSD
>>> Literal('foo') == Literal('foo', datatype=XSD.string)
False

According to https://www.w3.org/TR/rdf11-concepts/#section-Graph-Literal, every literal has a datatype (and a simple literal without a datatype should be considered syntactic sugar for literals of datatype http://www.w3.org/2001/XMLSchema#string).

The implementation in rdflib seems to follow the original RDF (not RDF 1.1), which allowed missing datatypes.

Related issues:

Literal should always have a datatype #1326 proposes to always set a datatype (which would work, but might break things).

PS:
A similar problem would presumably occur with RDF.langString, except that setting both lang and datatype results in an error (which might also be an issue).

This is a bug, but fixing it without a major version bump may be somewhat controversial.

One option to solve this is to enforce that rdflib.terms.Literal always has a datatype, but then we won't be able to support RDF 1.0 anymore. I'm somewhat okay with this, I think it would be nice to be able to support 1.0 and 1.1 - but I think 1.1. support is more important.

I think the best option is to just not allow null literals.

One option to solve this is to enforce that rdflib.terms.Literal always has a datatype, but then we won't be able to support RDF 1.0 anymore. I'm somewhat okay with this, I think it would be nice to be able to support 1.0 and 1.1 - but I think 1.1. support is more important.

The alternative is to maybe create a Literal abstract class which has optional data type and then have Literal with mandatory Datatype but also RDF10Literal with optional datatype, but I think that is going to be too much work.

I think the best option is to just not allow null literals.

An other option: allow None datatype but for __eq__ and __hash__ use self.datatype or XSD.string to get the datatype. This should ensure proper equality support while keeping the "plain literal" information.

A similar problem would presumably occur with RDF.langString, except that setting both lang and datatype results in an error (which might also be an issue).

It might be nice to tweak the constructor to fail only if datatype is not RDF.langString.

An other option: allow None datatype but for __eq__ and __hash__ use self.datatype or XSD.string to get the datatype.

This may be quite unexpected if someone is not familiar with RDF, and even for me, I would expect that the property has a stable value and that value is used during equality and hashing.

I think eliminating the possibility of None datatypes is simpler for users to understand.

For the constructor, I'm thinking of having 3 overloads:

def __new__(value: str, datatype: str = XSD.string) -> Created with lang=None
def __new__(value: str, lang: str) -> Created with datatype=XSD.langString
def __new__(value: str, lang: str, datatype: str) -> Create with the supplied datatype and lang, potentially with some validation.

One obvious problem with validation is that it sometimes it may be undesirable, but I think we can maybe in future add a static variable on the Literal class to control its strictness (e.g. Literal.strict), even though that is not the nicest thing.

I think at some point we should look at making factories that can control strictness in a way that does not involve global state, but right now, being overly strict may be best as it is easier to make it more lax later, I think.

Another related issue:

Literal.datatype doesn't get initialized for Turtle string #670

This may be quite unexpected if someone is not familiar with RDF, and even for me, I would expect that the property has a stable value and that value is used during equality and hashing.

Indeed. It makes perfect sense.

def __new__(value: str, lang: str, datatype: str) -> Create with the supplied datatype and lang, potentially with some validation.

This variant seems dangerous because it indeed requires some validation. What about only allowing the first two overloads, making sure that datatype can not be set at the same time with lang? This way no validation would be needed.

def __new__(value: str, lang: str, datatype: str) -> Create with the supplied datatype and lang, potentially with some validation.

This variant seems dangerous because it indeed requires some validation. What about only allowing the first two overloads, making sure that datatype can not be set at the same time with lang? This way no validation would be needed.

The only potential use may be if we have a stream of data where we get Literals as value, datatype and language, a bit similar to data from HexT, but I will try to avoid adding this overload as I agree it is best to not have it if we don't need it, and if we really do need it we can add it later.

One problem is that SPARQL 1.1 defines Simple Literal:

18.1.2. Simple Literal

Definition: Simple Literal

The set of Simple Literals is the set of all RDF Literals with no language tag or datatype IRI.

I check RDF4J and Jena, Jena's Literal does support null datatypes:

https://www.javadoc.io/doc/org.apache.jena/jena-core/latest/org.apache.jena.core/org/apache/jena/rdf/model/impl/LiteralImpl.html

getDatatype

public RDFDatatype getDatatype()

Return the datatype of the literal. This will be null in the case of plain literals.

RDF4J on the other hand defaults to xsd:string for datatype as the RDF 1.1 standard defines.

getDatatype

IRI getDatatype()

Gets the datatype for this literal.

If getLanguage() returns a non-empty value than this must return rdf:langString. If no datatype was assigned to this literal by the creator, then this method must return xsd:string.

Probably on balance still right to use the RDF 1.1 spec, but worth highlighting this caveat.

One problem is that SPARQL 1.1 defines Simple Literal:

18.1.2. Simple Literal
Definition: Simple Literal
The set of Simple Literals is the set of all RDF Literals with no language tag or datatype IRI.

I guess the right option is to just defer to the RDF 1.1 spec.

What we can also do is have a simple attribute on Literal which is true if the Literal is initialized with a null datatype 🤔

But we can maybe add that later also.

What we can also do is have a simple attribute on Literal which is true if the Literal is initialized with a null datatype thinking

This would complicate equality about as much as having datatype=None being equal to datatype=XSD.string

I made a draft PR (#2460) that still has some work left, I think some tests will fail, but one thing which is quite apparent is that there are cases where passing datatype=None and having it be interpreted as xsd:string is helpful, mainly because there is quite often cases where the passed parameter is of type Optional[str].

Navigation Menu

Repository navigation

Literal equality fails for datatypes `None` and `XSD.string` #2123

4 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Literal equality fails for datatypes None and XSD.string #2123

Description

Activity

aucampia commented on Mar 20, 2023

aucampia commented on Jun 9, 2023

aucampia commented on Jun 9, 2023

4 remaining items

aucampia commented on Jun 9, 2023

Tpt commented on Jun 10, 2023

aucampia commented on Jun 10, 2023

aucampia commented on Jun 10, 2023

Tpt commented on Jun 10, 2023

aucampia commented on Jun 10, 2023

aucampia commented on Jun 20, 2023

aucampia commented on Jun 20, 2023

aucampia commented on Jun 20, 2023

aucampia commented on Jun 20, 2023

aucampia commented on Jun 20, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Literal equality fails for datatypes `None` and `XSD.string` #2123