Skip to content

Latest commit

Β 

History

609 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

LinkedRecords

LinkedRecords is a Backend-as-a-Service you can connect to directly from your single-page application - no backend code required. Authorization is built into the data model, and real-time collaboration (CRDT/OT) works out of the box.

LinkedRecords Introduction

Quickstart

Run the backend locally with Docker (includes PostgreSQL and a mock OIDC provider with one-click test accounts):

curl -fsSLO https://raw.githubusercontent.com/wolfoo2931/linkedrecords/main/docker-compose.yml
docker compose up

The server is then available at http://localhost:6543. Connect to it from your app:

npm install @linkedrecords/browser
import LinkedRecords from '@linkedrecords/browser';

const lr = new LinkedRecords(new URL('http://localhost:6543'), {
  client_id: 'linkedrecords-dev',
  redirect_uri: window.location.origin + '/callback',
});

// To check if the user is authenticated:
// const isAuth = await lr.isAuthenticated();

// To start the login flow (e.g., on a button click):
// lr.login();

// Create a record only you can access - share it later by creating facts
const todo = await lr.Record.createKeyValue(
  { title: 'Buy groceries', done: false },
  [['$it', 'isA', 'Todo']],
);

Using React? The Getting Started Guide builds a complete collaborative todo app with @linkedrecords/react hooks.

Concept

You can think of LinkedRecords as a bucket where anyone can sign up and insert data. As long as you don't share this data with other users or groups, only you can access what you've written into it.

In theory, any user could use the LinkedRecords API directly to write and retrieve data. However, this would be inconvenient - just as you wouldn't expect your users to write SQL queries, you wouldn't expect them to interact with the LinkedRecords API. A LinkedRecords app is a specialized frontend that hides the API and provides a convenient user interface for accomplishing their tasks.

In the traditional SQL world, inconvenience isn't the only reason you don't let users access the database directly - authorization concerns are an even stronger reason. With LinkedRecords, this is no longer an issue: authorization is built directly into the API. This requires a small mindset shift: Instead of defining universal authorization rules in the backend for all records, the user who inserts a data record specifies who can read it.

For the LinkedRecords API, simplicity, flexibility, and a decoupled architecture are the main qualities we strive to achieve.

  • Simplicity: The API should not have many endpoints or methods; instead, it consists of a few fundamental building blocks.
  • Flexibility: The few available endpoints can be composed to support a variety of use cases. The baked in authorization model should allow to implement different authorization use cases (RBAC, ...).
  • Decoupled: LinkedRecords should be decoupled from the single-page applications which use it as data storage

Think of it as SQL you can call directly from your React app without worrying about permissions; it is easier to read than SQL and provides live updates.

Configuration

LinkedRecords is configured via environment variables. See tables below, or the full configuration reference with examples for every deployment mode.

Environment Variable Name Example Description
PGHOST localhost The hostname of the PostgreSQL server.
PGUSER linkedrecords The PostgreSQL user name.
PGPASSWORD xxxx The PostgreSQL password.
PGDATABASE xxxx The PostgreSQL database name.
PG_POOL_SIZE 10 The maximum number of connections in the PostgreSQL connection pool. Defaults to 10.
CORS_ORIGIN ["https://app.example.com", "https://app.example.app"] The content of the cors origin header. If not provided, the value of FRONTEND_BASE_URL will be used.
SERVER_BASE_URL http://localhost:6543 The public URL of the linkedrecords server.
DEFAULT_STORAGE_SIZE_QUOTA 500 The default storage size quota in MB.
QUOTA_COUNT_KV_RECORDS false If the storage space for KeyValue records is deducted from the accountee quota.
QUOTA_COUNT_LT_RECORDS false If the storage space for LongText records is deducted from the accountee quota.
ENABLE_AUTH_RULE_CACHE false Enable cache for authorization lookups. Might require a lot of memory.
SHORT_LIVED_ACCESS_TOKEN_SIGNING xxxx Configuring this is optional but can reduce load on the database because short lived access tokens will be used for checking access when a client subscribes to record changes.

Confidential Client Mode

The environment variables in this section are all optional if the configuration described in the section "Public Client Mode" are provided.

If LinkedRecords runs in confidential client mode, then a session token will be stored in an HttpOnly cookie. From a security standpoint this is considered the suggested method. However, this is not possible if the LinkedRecord server and the frontend do not share the same domain. Across different domains the cookie becomes a third-party cookie, so this mode cannot be used.

Environment Variable Name Example Description
FRONTEND_BASE_URL http://localhost:3001 The base URL of the frontend. It will be used for the Access-Control-Allow-Origin HTTP header and is also required for the OpenID connect redirection.
AUTH_ISSUER_BASE_URL https://xxx.us.auth0.com/ The URL of the OIDC issuer. Can be any OpenID Connect compliant identity provider (e.g. Auth0, Okta).
AUTH_CLIENT_ID The client id. Can be obtained from the identity provider.
AUTH_CLIENT_SECRET The client secret. Can be obtained from the identity provider.
AUTH_IDP_LOGOUT true When set to true the user session will be destroyed in the application AND within the identity provider.
AUTH_COOKIE_SIGNING_SECRET xxxx The secret used to sign cookies.

Public Client Mode

In case the single-page application is hosted on a different domain than the LinkedRecords server, the single-page application has to store the access token in the browser. In this scenario the following environment variables need to be configured.

Environment Variable Name Example Description
ALLOW_HTTP_AUTHENTICATION_HEADER true Allows public clients to make requests by providing an access token via http authentication header.
AUTH_ISSUER_BASE_URL https://xxx.us.auth0.com/ The URL of the OIDC issuer. Can be any OpenID Connect compliant identity provider (e.g. Auth0, Okta).
AUTH_TOKEN_AUDIENCE your-audience-id LinkedRecords will check the audience specified in the JWT bearer token against the value specified in this field.
AUTH_CLIENT_ID The client id. Can be obtained from the identity provider.

Optional Configuration

S3

If S3 is configured it will be used to store blob record values. If it is not configured they will be stored in the PostgreSQL database. It is recommended to configure S3.

Environment Variable Name Example Description
S3_COPY_FROM_BL_ATTRIBUTE_TABLE false This is used for migrating blob storage from postgresql to S3.
S3_ENDPOINT s3.system.svc.cluster.local The hostname of the S3 endpoint.
S3_BUCKET linkedrecords-blobs The name of a bucket. The bucket must exist already.
S3_ACCESS_KEY xxx The access key id to upload blobs to S3.
S3_SECRET_KEY xxx The secret key id to upload blobs to S3.
S3_USE_SSL false Do not use TLS when uploading/downloading to S3.

Redis

Redis is only needed when running multiple LinkedRecords instances behind a load balancer, so real-time updates reach clients connected to other instances.

Environment Variable Name Example Description
REDIS_HOST redis.example.com The Redis host. Setting this enables the Redis adapter.
REDIS_USERNAME default The Redis username.
REDIS_PASSWORD xxx The Redis password.

Paddle

Environment Variable Name Example Description
PADDLE_NOTIFICATION_SECRET xxxx If paddle is used for upgrading quotas this needs to be the notification secret to verify the signature of the webhook content.
PADDLE_API_URL https://sandbox-api.paddle.com The URL of the paddle api.
PADDLE_API_KEY xxx the paddle API key.

Development

Contributions are welcome - see CONTRIBUTING.md for details.

Prerequisites: Node.js v24 and a PostgreSQL database (configured via the PG* environment variables, see CONTRIBUTING.md; alternatively set USE_PGLITE=true to use an embedded database).

npm install
npm start              # Build and run the server on port 6543
npm test               # Run unit/component tests
npm run wdio           # Run end-to-end tests
npm run lint           # ESLint

The documentation website lives in docs/:

cd docs && npm install && npm run dev

Performance

The chart below shows the performance of core operations as the database grows. This is automatically updated after each merge to main.

The load test simulates a realistic document management scenario. Each createDocument operation creates a blueprint of 8 records (7 KeyValueRecords + 1 LongTextRecord) including the document content, collaborator/reader groups, comments, references, and configuration. The fetchDocuments operation lists all documents for a user, while fetchDocument retrieves a single document with all its associated records.

Load Test Performance Chart

How the Load Test Works

The test simulates a multi-tenant environment with three users:

  • User 1 continuously creates documents (in the current test configuration 5,000 iterations).
  • User 2 is the "user under test" who creates one document for every 10 documents User 1 creates, up to 300 documents. At 300 documents, the creation of documents for this user stops.
  • User 3 creates documents occasionally (every 1,000 iterations)

The x-axis shows the total number of documents in the database (owned by all users combined). The y-axis shows the response time in milliseconds.

What's being measured:

Operation Description
createDocument Time to create a new document with all related records (content, config, comments, references, collaborator/reader groups). You can see that createDocument is independent of the total amount of documents in the database as well as the amount of documents visible to the user.
fetchDocuments Time to fetch User 2's document list (up to 300 documents they see). You can see that this time depends on the amount of documents visible to the user but not on the total amount of documents (the graph flattens around 3000 documents)
fetchDocument Time to fetch a single document with all related data (content, comments, groups, activity state, references). You can see that this time depends on the amount of documents visible to the user but not on the total amount of documents (the graph flattens around 3000 documents)

About

A BaaS (Backend-as-a-Service) solution for web applications, inspired by the flexible nature of triplestores. Featuring real-time collaboration and serverless authorization πŸ”

Topics

Resources

Contributing

Stars

90 stars

Watchers

3 watching

Forks

Releases

Packages

Used by

Contributors

Languages