SEPA - SPARQL Event Processing Architecture

• 🚀 Introduction
• ⚡ Quick Start
• ⚙️ Configuration
• 🧰 Usage
• 🛠️ Build with Maven
• 🤝 Contributing
• 🙌 Credits

SEPA (SPARQL Event Processing Architecture) is a publish-subscribe architecture designed to support information level interoperability.

The architecture is built on top of generic SPARQL endpoints conformant with the SPARQL 1.1 Protocol. Publishers and subscribers use standard SPARQL 1.1 Updates and Queries.

Notifications about events, such as changes in the RDF knowledge base, are expressed as added and removed SPARQL binding results since the previous notification. To learn more about the SEPA architecture and vision, refer to this paper.

SEPA has been formalized in the following unofficial drafts:

• SPARQL Event Processing Architecture (SEPA) contribute here
• SPARQL 1.1 Secure Event Protocol contribute here
• SPARQL 1.1 Subscribe Language contribute here
• JSON SPARQL Application Profile (JSAP) contribute here

You can play with SEPA on our dashboard. 🕹️

🛠️ To build SEPA from source, see Build with Maven.

Java 25 and Maven are required to build the current project.

Build and install all modules locally with tests skipped:

mvn clean install -DskipTests -Dgpg.skip=true

-DskipTests is needed for the local reactor build because the api-java module includes integration tests that require a running SEPA engine. -Dgpg.skip=true skips artifact signing for local builds.

The build creates executable shaded JARs in the module target directories, including the SEPA engine JAR:

engine/target/sepa-engine-<version>.jar

Run the api-java integration tests with a Maven-started in-memory SEPA engine:

mvn -pl api-java verify -Pwith-sepa-engine -Dgpg.skip=true

The with-sepa-engine profile requires the engine JAR to have already been built by the previous reactor command. It starts engine/target/sepa-engine-<version>.jar during the Maven pre-integration-test phase using engine/src/main/resources/endpoints/jena-in-memory.jpar, waits for ports 8000 and 9000, runs the Failsafe integration tests, and stops the engine when Maven exits. Engine output is written to:

api-java/target/sepa-engine.log

The complete local verification flow is therefore:

mvn clean install -DskipTests -Dgpg.skip=true
mvn -pl api-java verify -Pwith-sepa-engine -Dgpg.skip=true

To know more about Maven, refer to the official documentation.

{
"host":"localhost",
"sparql11protocol":{
  "protocol":"http",
  "port":9999,
  "query":{
    "path":"/blazegraph/namespace/kb/sparql",
    "method":"POST",
    "format":"JSON"},
  "update":{
    "path":"/blazegraph/namespace/kb/sparql",
    "method":"POST",
    "format":"JSON"}}}

The default version of engine.jpar configures the engine to listen for incoming SPARQL 1.1 SE Protocol requests at the following URLs:

1. Query: http://localhost:8000/query
2. Update: http://localhost:8000/update
3. Subscribe/Unsubscribe: ws://localhost:9000/subscribe
4. SECURE Query: https://localhost:8443/secure/query
5. SECURE Update: https://localhost:8443/secure/update
6. SECURE Subscribe/Unsubscribe: wss://localhost:9443/secure/subscribe
7. Registration: https://localhost:8443/oauth/register
8. Token request: https://localhost:8443/oauth/token

{"parameters":{
  "scheduler":{
   "queueSize":100,
   "timeout":5000},
  "processor":{
   "updateTimeout":5000,
   "queryTimeout":5000,
   "maxConcurrentRequests":5,
   "reliableUpdate":true},
  "spu":{"timeout":5000},
  "gates":{
   "security":{
    "tls":false,
    "enabled":false,
    "type":"local"},
   "paths":{
    "secure":"/secure",
    "update":"/update",
    "query":"/query",
    "subscribe":"/subscribe",
    "unsubscribe":"/unsubscribe",
    "register":"/oauth/register",
    "tokenRequest":"/oauth/token"},
   "ports":{
    "http":8000,
    "https":8443,
    "ws":9000,
    "wss":9443}}}}

SEPA uses log4j2 by Apache. A default configuration is stored in the file log4j2.xml provided with the distribution. If the file resides in the engine folder, but it is not used, add the following JVM directive to force using it:

java -Dlog4j.configurationFile=./log4j2.xml -jar engine-x.y.z.jar

By default, the engine implements a simple in-memory OAuth 2.0 client-credential flow. It uses a JKS for storing the keys and certificates for SSL and JWT signing/verification. A default sepa.jks is provided including a single X.509 certificate (the password for both the store and the key is: sepa2017). If you face problems using the provided JKS, please delete the sepa.jks file and create a new one as follows: keytool -genkey -keyalg RSA -alias sepakey -keystore sepa.jks -storepass sepa2017 -validity 360 -keysize 2048 Run java -jar engine-x.y.z.jar -help for a list of options. The Java Keytool can be used to create, access and modify a JKS. SEPA also implements other two security mechanisms:

• LDAP: it extends the default one by storing clients's information into an LDAP server (tested with Apache Directory)
• KEYCLOAK: authentication based on OpenID Connect in managed by Keycloak

Security is configured within the engine.jpar as follows:

{"gates":{
  "security":{
    "tls": false,
    "enabled": true,
    "type": "local"
}}}

where

• type can assume one of the following values: local,ldap,keycloak
• tls is used when type=ldap to enable or not LDAP StartTLS

The SEPA engine is also distributed with a default JMX configuration jmx.properties (including the jmxremote.password and jmxremote.access files for password and user grants). Remember to change password file permissions using: chmod 600 jmxremote.password. To enable remote JMX, the engine must be run as follows: java -Dcom.sun.management.config.file=jmx.properties -jar engine-x.y.z.jar. Using jconsole is possible to monitor and control the most important engine parameters. By default, the port is 5555 and the root:root credentials grant full control (read/write).

java [JMX] [JVM] [LOG4J] -jar SEPAEngine_X.Y.Z.jar [-help] [-secure true] [-engine engine.jpar] [-endpoint endpoint.jpar] [JKS OPTIONS] [LDAP OPTIONS] [ISQL OPTIONS]

• secure : overwrite the current secure option of engine.jpar
• engine : can be used to specify the JSON configuration parameters for the engine (default: engine.jpar)
• endpoint : can be used to specify the JSON configuration parameters for the endpoint (default: endpoint.jpar)
• help : to print this help

[JMX]

• Dcom.sun.management.config.file=jmx.properties : to enable JMX remote managment

[JVM]

• XX:+UseG1GC

[LOG4J]

• Dlog4j.configurationFile=path/to/log4j2.xml

[JKS OPTIONS]

• sslstore : JKS for SSL CA (default: ssl.jks)
• sslpass : password of the JKS (default: sepastore)
• jwtstore : JKS for the JWT key (default: jwt.jks)
• jwtalias : alias for the JWT key (default: jwt)
• jwtstorepass : password for the JKS (default: sepakey)
• jwtaliaspass : password for the JWT key (default: sepakey)

[LDAP OPTIONS]

• ldaphost : host (default: localhost)
• ldapport : port (default: 10389)
• ldapdn : domain (default: dc=sepatest,dc=com)
• ldapusersdn : domain (default: null)
• ldapuser : username (default: null)
• ldappwd : password (default: null)

[ISQL OPTIONS]

• isqlpath : location of isql (default: /usr/local/virtuoso-opensource/bin/)
• isqlhost : host of Virtuoso (default: localhost)
• isqluser : user of Virtuoso (default: dba)
• isqlpass : password of Virtuoso (default: dba)

1. Fork it!
2. Create your feature branch: git checkout -b my-new-feature
3. Check some IDE specific instruction below
4. Do your stuff
5. Provide tests for your features if applicable
6. Commit your changes: git commit -am 'Add some feature'
7. Push to the branch: git push origin my-new-feature
8. Submit a pull request :D

Pull request with unit tests have an higher likelihood to be accepted, but we are not to restrictive. So do not be afraid to send your contribution!

SEPA has been inspired and influenced by Smart-M3. SEPA authors have been involved in the development of Smart-M3 since its origin.

The main differences between SEPA and Smart-M3 are the protocol (now compliant with the SPARQL 1.1 Protocol) and the introduction of a security layer (based on TLS and JSON Web Token for client authentication).

All the SEPA software components have been implemented from scratch.

SEPA Engine is released under the GNU GPL, SEPA APIs are released under the GNU LGPL