Software

The Linked Data Event Streams client

A Linked Data Event Stream is a tree:Collection with immutable members. You can use the client to replicate the full tree:Collection and remain in-sync with it afterwards by following all relations starting from a view. Documentation on Github

Connector architecture

Bucketizers categorize each member on a stream in one or more pages/buckets/fragments. These bucketized members can then be ingested into a LDES server.

To facilitate this orchestration of different processors handling the stream of data, a specialized runner can be used. There exists a nifi-runner and a js runner that both start a chain of processors using some rdf configuration.

Using these runners enables the user to configure one big pipeline, that can be started with each pipeline runner (nifi/js) individually, and configure the processor to talk with each other over channels, like websockets.

Here are some components that should work nicely together

• Bucketizers (js)
• LDES solid server
• SDS storeage writer mongo (LDES server ingestor)
• LDES Server 4J
• LDES client processor (nifi)
• NGSI LD to LDES processor (nifi)
• NGSI V2 to NGSI LD processor (nifi)

Querying tree:Collections with Comunica

Comunica, a Web framework to query Linked Data Fragments and heterogeneous views on top of tree:Collections.

At the moment of writing (dec 2021), TREE support in Comunica is still experimental. A working prototype can be found in this fork. It supports TREE and LDES metadata extraction, and is able to perform autocompletion across one or more tree:Collection that is fragmented using a substring fragmentation.

LDES Snapshot

A snapshot of a versioned LDES at time t is an LDES where only the most recent versions until time t are present. The @treecg/ldes-snapshot library allows everyone to create a snapshot of a versioned LDES.

LDES in LDP package

The LDES in LDP specification describes how to write to a time-based fragmented LDES stored on a Solid pod or Linked Data Platform (LDP).
@treecg/versionawareldesinldp is an npm package that deals with the initialisation of and interaction with a (versioned) LDES in LDP.