Skip to content

PaulJPhilp/effect-notion

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

effect-notion

effect-notion is a lightweight, production-ready server that acts as a secure proxy for the Notion API. Built with the powerful Effect library, it provides a robust, type-safe, and efficient way to fetch data from Notion databases and pages.

It includes features for real apps: logical field overrides, dynamic filtering, schema-aware validation, consistent errors, and optional type/code generation.

Who is this for?

This server is ideal for developers building front-end applications (e.g., blogs, documentation sites, personal portfolios) that use Notion as a CMS. It acts as a secure and robust backend layer, abstracting away the complexities of the Notion API and preventing the exposure of API keys on the client-side. Think of it as a "smart bridge" that makes Notion work like a traditional database while keeping all the benefits of Notion's interface and collaboration features.

Key Benefits

Security: API keys never leave your server Type Safety: Full TypeScript support with generated types Flexibility: Decouples your app from Notion's exact schema Performance: Built for production with Effect's concurrent processing Deployment Ready: Works with Vercel and other platforms

Architecture

The server acts as a secure intermediary between your application and the Notion API.

[Your Frontend App] <--> [effect-notion server] <--> [Notion API]

Your frontend talks to this server; the server (holding your NOTION_API_KEY) talks to Notion.

Effect-TS Architecture

This project follows Effect-TS best practices throughout:

  • Service Patterns: All dependencies (NotionClient, NotionService, ArticlesRepository) are modeled as Effect services
  • Layer-Based DI: Dependencies are provided via Layers for testability and composability
  • Immutable State: No global mutable state; uses FiberRef and Ref for safe concurrent state
  • Type-Safe Errors: All errors are tagged with Data.TaggedError for exhaustive handling
  • Effect Chains: Pure Effect pipelines with no Promise conversions
  • Clock Service: Time operations use Clock for deterministic testing
  • Metrics & Tracing: Built-in observability with Effect.Metric and Effect.withSpan

Key Services:

  • RequestIdService - Request correlation IDs (FiberRef-based)
  • LogicalFieldOverridesService - Database field mapping configuration
  • NotionClient - Low-level Notion API client with retry
  • NotionService - Business logic with schema caching
  • ArticlesRepository - High-level CRUD operations with tracing

Schema-driven adapters

See docs/SchemaAdapter.md for the schema-driven adapter pattern used to map Notion property bags to domain entities using Effect Schema, including how to add new field mappings.

Features

  • Secure Notion API Proxy: Safely access the Notion API without exposing your credentials on the client-side.
  • Rich Filtering Capabilities: Dynamically filter Notion database entries using a flexible JSON-based query language.
  • Logical Field Overrides: Decouple your application from your Notion schema by mapping Notion's field names to logical names in your code.
  • Codegen for Type Safety: Generate TypeScript types and Effect Schema from your live Notion database to ensure end-to-end type safety.
  • Built with Effect: Leverages the Effect library for a highly performant, concurrent, and error-resilient server following Effect-TS best practices.
  • Ready to Deploy: Includes configurations for easy deployment to Vercel (Node.js runtime).
  • Consistent Error Model: All errors return normalized JSON with a stable shape and a requestId for tracing.
  • Production Observability: Built-in metrics (Effect.Metric), structured logging, and distributed tracing (Effect.withSpan).
  • Effect-Native Patterns: Proper service patterns with Layers, immutable state management, and type-safe error handling.
  • Intelligent Retry: Automatic retry with exponential backoff using Effect.retry and Schedule.

Runtime and adapters

  • src/router.ts: Effect HttpRouter defining the API surface. It is the source of truth for business logic and error handling.
  • api/index.ts: Vercel Node v3 adapter. It converts the router to a Fetch-style handler and applies logging + CORS. Health and all routes are served via the router (no fast-path bypasses).
  • src/main.ts: Local Bun/Node server entry for development using the Effect Node HTTP server integration.

Logging & CORS:

  • Both entry points enable structured logging via HttpMiddleware.logger.
  • CORS is enabled; configure via CORS_ORIGIN.

Observability & Resilience

The application follows Effect-TS best practices for production observability:

Metrics (Effect.Metric)

  • Endpoint: /api/metrics provides real-time metrics in Prometheus format
  • Fiber-safe: Uses Effect.Metric for concurrent-safe counters and histograms
  • Automatic: Tracks Notion API requests, durations, and errors

Distributed Tracing (Effect.withSpan)

  • OpenTelemetry Ready: All CRUD operations instrumented with Effect.withSpan
  • Rich Context: Spans include operation parameters for debugging
  • Integrations: Works with Jaeger, Datadog, Zipkin, and other OpenTelemetry exporters

Retry & Error Handling

  • Effect.retry: Automatic retry with Schedule-based policies
  • Exponential Backoff: Configurable delays with jitter
  • Type-Safe Errors: Tagged errors with Effect.catchAll and catchTags
  • Request IDs: Every request includes a unique ID for log correlation

Effect Service Architecture

  • Service Patterns: All dependencies modeled as Effect services with Layers
  • Immutable State: No global mutable state, fiber-safe throughout
  • Clock Service: Deterministic time handling for testing
  • Structured Logging: Effect.log with JSON output

Configuration & Security

This server is the sole keeper of your NOTION_API_KEY. Never expose or send this key from the client.

Create a .env file at the project root with the following variables:

# Required
NOTION_API_KEY=your_notion_integration_key_here

# Optional
NODE_ENV=development
PORT=3000
CORS_ORIGIN=*
LOG_LEVEL=Info
NOTION_DB_ARTICLES_BLOG=your_blog_database_id_here

Source Configuration

Sources (databases) are configured via JSON files for flexibility and environment-specific settings:

Default configuration (sources.config.json):

{
  "version": "1.0",
  "sources": [
    {
      "alias": "blog",
      "kind": "articles",
      "databaseId": "${NOTION_DB_ARTICLES_BLOG}",
      "adapter": "blog",
      "capabilities": {
        "update": true,
        "delete": true
      },
      "description": "Public-facing blog posts"
    }
  ]
}

Key features:

  • Environment variable substitution: Use ${VAR_NAME} pattern for database IDs
  • Environment-specific configs: Point to different configs via NOTION_SOURCES_CONFIG env var
  • Per-source capabilities: Control read/write/delete permissions per source
  • Startup validation: Schema validation with clear error messages
  • No code changes needed: Add/remove sources by editing JSON

For production (read-only):

NOTION_SOURCES_CONFIG=./sources.config.production.json bun start

See Source Configuration Guide for complete reference and Migration Guide for upgrading from hardcoded sources.

Env file precedence

Loaded at startup in this order (later overrides earlier):

  1. .env
  2. .env.local
  3. .env.$NODE_ENV
  4. .env.$NODE_ENV.local

Environment variables

  • NODE_ENV (development | test | production). Default: development.
  • PORT: Port for local server. Default: 3000.
  • CORS_ORIGIN: CORS allowed origin(s). Default: *.
  • CORS_ALLOWED_METHODS: Comma-separated list of allowed HTTP methods. Default: POST,GET,OPTIONS.
  • CORS_ALLOWED_HEADERS: Comma-separated list of allowed headers. Default: Content-Type,Authorization.
  • LOG_LEVEL: Effect logger level. Default: Info.
  • NOTION_API_KEY: Your Notion integration key.
  • NOTION_SOURCES_CONFIG: Path to sources configuration file. Default: ./sources.config.json.
  • NOTION_DB_ARTICLES_BLOG: Database ID for the blog source (used via environment variable substitution in sources config).

Quick Start

  1. Install dependencies

    bun install
  2. Run the dev server (Bun)

    bun run dev
  3. Server runs at http://localhost:3000 (or your PORT).

Useful scripts from package.json:

  • bun start — run src/main.ts
  • bun run dev — watch mode
  • bun test — run tests (Vitest via Bun)
  • bun run build — type-check via tsc
  • bun run codegen:notion — run schema codegen (see below)
  • bun run health-check [URL] — test live server health (see docs/HEALTH_CHECK.md)

Diagnostics helper:

bun scripts/diagnose.ts /api/health

You can also POST JSON bodies to exercise routes end-to-end. The script prints request/response details and structured logs.

Usage:

# GET (default method)
bun scripts/diagnose.ts "/api/health"

# POST with JSON body
bun scripts/diagnose.ts "/api/articles/list" POST '{"source":"blog","pageSize":5}'

# Arbitrary path + body
bun scripts/diagnose.ts "/api/your/route" POST '{"key":"value"}'

Notes:

  • Content-Type is set to application/json automatically for POST bodies.
  • Logs include pre-response info and Effect structured logs at Debug level.

API Endpoints

Note on HTTP methods:

  • Use GET for simple, idempotent retrieval by ID (e.g., pageId in query).
  • Use POST when a structured JSON body is required (e.g., filters/sorts for listing; content payload for update).

List Articles (paginated)

  • Endpoint: POST /api/list-articles
  • Description: Retrieves a paginated list of items from a Notion database, with optional filtering and sorting.

Request Body:

{
  "databaseId": "YOUR_NOTION_DATABASE_ID",
  "titlePropertyName": "Name",
  "filter": {
    "property": "Status",
    "select": { "equals": "Published" }
  },
  "sorts": [
    { "property": "Date", "direction": "descending" }
  ],
  "pageSize": 20,            // optional (1-100)
  "startCursor": "..."       // optional
}

Response:

{
  "results": [
    { "id": "...", "title": "..." }
  ],
  "hasMore": true,
  "nextCursor": "..." // null when no more
}

Example Request:

curl -X POST http://localhost:3000/api/list-articles \
  -H "Content-Type: application/json" \
  -d '{
    "databaseId": "<YOUR_DATABASE_ID>",
    "titlePropertyName": "Name",
    "filter": {
      "property": "Status",
      "select": { "equals": "Published" }
    },
    "pageSize": 20
  }'

Get Article Content

  • Endpoint: GET /api/get-article-content
  • Description: Retrieves the content of a single Notion page, formatted as Markdown.

Query Parameters:

  • pageId: The ID of the Notion page to retrieve.

Example Request:

curl "http://localhost:3000/api/get-article-content?pageId=<YOUR_PAGE_ID>"

Update Article Content

  • Endpoint: POST /api/update-article-content
  • Description: Replaces the content of a Notion page with new content provided as a Markdown string.

Request Body:

{
  "pageId": "YOUR_NOTION_PAGE_ID",
  "content": "# New Title\n\nThis is the new content of the page."
}

Example Request:

curl -X POST http://localhost:3000/api/update-article-content \
-H "Content-Type: application/json" \
-d '{
  "pageId": "<YOUR_PAGE_ID>",
  "content": "# My New Page Title\n\nAnd this is the updated content."
}'

Get Database Schema

  • Endpoint: GET /api/get-database-schema
  • Description: Returns the normalized live schema for a Notion database.

Example:

curl "http://localhost:3000/api/get-database-schema?databaseId=<YOUR_DB_ID>"

Get Article Metadata (properties)

  • Endpoint: GET /api/get-article-metadata
  • Description: Returns the raw Notion page properties for a page.

Example:

curl "http://localhost:3000/api/get-article-metadata?pageId=<YOUR_PAGE_ID>"

Error Responses (normalized)

All error responses follow a consistent JSON structure and include a request ID:

{
  "error": "Bad Request",
  "code": "BadRequest",
  "requestId": "abcd1234",
  "detail": "Optional human-friendly text",
  "errors": ["Optional list of validation errors"]
}
  • Codes include: BadRequest, InvalidApiKey, NotFound, InternalServerError.
  • The x-request-id header mirrors requestId for log correlation.

Health (router-based)

  • Endpoint: GET /api/health
  • Description: Reports server health via the router.

Example:

curl "http://localhost:3000/api/health"

Articles (router-based, source-aware)

These endpoints operate on logical "articles" and are parameterized by a source (e.g., blog). Sources are configured in sources.config.json with environment variable substitution.

Example sources:

  • blog → uses ${NOTION_DB_ARTICLES_BLOG} from environment
  • Add more by editing sources.config.json (see Source Configuration Guide)

All POST requests require Content-Type: application/json.

• List

  • Endpoint: POST /api/articles/list
  • Body:
{ "source": "blog", "pageSize": 20 }

• Get by id

  • Endpoint: GET /api/articles/get
  • Query: source=blog&pageId=<PAGE_ID>

• Create

  • Endpoint: POST /api/articles/create
  • Body (partial fields are allowed):
{ "source": "blog", "data": { "name": "New Article" } }

• Update

  • Endpoint: POST /api/articles/update
  • Body (partial fields are allowed):
{
  "source": "blog",
  "pageId": "<PAGE_ID>",
  "patch": { "name": "Updated" }
}

• Delete (archive)

  • Endpoint: POST /api/articles/delete
  • Body:
{ "source": "blog", "pageId": "<PAGE_ID>" }

Notes:

  • Field shapes align with src/domain/logical/Common.ts (BaseEntity, ListParams).
  • The Notion mapping is handled by the source adapter, e.g., src/domain/adapters/articles/blog.adapter.ts.

Advanced Features

Dynamic Filtering

You can construct complex filters based on the Notion API's filter object structure.

Example: Compound filter

This request fetches entries where "Status" is "In Progress" AND "Priority" is "High".

# The JSON body for your POST /api/list-articles request:
{
  "databaseId": "<YOUR_DATABASE_ID>",
  "filter": {
    "and": [
      { "property": "Status", "select": { "equals": "In Progress" } },
      { "property": "Priority", "select": { "equals": "High" } }
    ]
  }
}

Codegen

Optional codegen can emit a static module describing your current Notion database schema to aid compile-time checks.

From scripts/generate-notion-schema.ts:

# Uses env vars if flags are omitted
bun scripts/generate-notion-schema.ts \
  --databaseId <id> \
  [--apiKey <key>] \
  [--out src/generated/notion-schema.ts] \
  [--emitEffectSchema]

Defaults:

  • apiKey: NOTION_API_KEY
  • out: src/generated/notion-schema.ts

Outputs:

  • src/generated/notion-schema.ts (types and data)
  • If --emitEffectSchema is provided, also emits src/generated/notion-schema.effect.ts.

Testing

Tests include live Notion integration. Ensure env vars are set (.env or system).

bun test

The project uses Vitest for testing with the following configuration:

  • Test environment: Node.js
  • Test files: test/**/*.test.ts
  • Excludes compiled JavaScript and node_modules

Deployment

Vercel (Recommended)

The project is configured for Vercel with Node.js runtime (vercel.json):

{
  "version": 2,
  "functions": {
    "api/index.ts": { "runtime": "@vercel/node@3.2.20" }
  },
  "routes": [
    { "src": "/api/(.*)", "dest": "api/index.ts" },
    { "src": "/(.*)", "dest": "api/index.ts" }
  ]
}

Steps:

  1. Push to a Git repo and import into Vercel
  2. Set environment variables (see Configuration section above)
  3. Deploy with vercel --prod
  4. Verify deployment: bun run health-check https://your-app.vercel.app

Note: While local development uses Bun for speed, Vercel deployment uses Node.js runtime for compatibility.

Post-Deployment Verification:

After deploying, run the health check script to verify all systems are operational:

# Test your production deployment
bun run health-check https://your-app.vercel.app

# Test staging environment
bun run health-check https://your-app-staging.vercel.app

The health check validates:

  • Health endpoint responds correctly
  • CORS configuration is working
  • 404 handling for invalid routes
  • Request ID headers are present

See docs/HEALTH_CHECK.md for detailed usage and CI/CD integration.

Local Development

# Development server with watch mode
bun run dev

# Production build
bun run build

# Run tests
bun test

OpenTelemetry Integration (Optional)

To enable distributed tracing in production, add an OpenTelemetry exporter:

bun add @effect/opentelemetry @opentelemetry/exporter-jaeger
# or @opentelemetry/exporter-datadog, etc.

See docs/DEPLOYMENT_READY.md for detailed production deployment checklist and OpenTelemetry configuration examples.

Documentation

Quick Links

Getting Help

Contributing

Contributions are welcome! Please open an issue or PR.

Before submitting a PR, ensure build and tests pass:

bun run build  # type-check
bun test       # run tests

See DEVELOPMENT.md for detailed contribution guidelines.

Modular Services & Import Conventions

  • Services live under src/services/<ServiceName>/ with a consistent layout:

    • api.ts — public interface and Effect tag
    • types.ts — request/response types
    • errors.ts — typed errors (optional)
    • helpers.ts — pure helpers
    • service.ts — implementation and .Default layer
    • __tests__/ — colocated tests
  • Backward compatibility is preserved with re-exports:

    • src/<ServiceName>.ts re-exports from src/services/<ServiceName>/service.ts
    • src/services/<ServiceName>.ts (legacy) also re-exports to the new impl
  • TypeScript NodeNext with verbatim syntax requires explicit extensions:

    • Use .js for runtime imports of values
    • Use .ts for type-only imports

Example:

// runtime
import { NotionService } from "./src/services/NotionService/service.js"

// type-only
import type { ListResult } from "./src/services/ArticlesRepository/types.ts"