Skip to content

Giswater/giswater-api

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

307 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸš€ Giswater FastAPI Service

Versions Release Python Docker Image Version
CI Lint Tests
Status Last commit Commit activity
Meta License: GPL v3

A lightweight, modular FastAPI application with Swagger UI, Docker support, Keycloak authentication, and Gunicorn + Uvicorn for production.

πŸ“š Table of Contents

✨ Overview

Giswater API exposes a clean HTTP surface to query and operate a Giswater database. It wraps existing GIS and network operations behind FastAPI endpoints, so desktop tools, mobile apps, and integrations can run the same workflows without directly touching the database.

How it works

  • Each route validates input with Pydantic models, builds a standard payload, and calls Postgres procedures in the Giswater schema.
  • Modules are toggled via environment variables, so you only expose the domains you need (Basic, OM, CRM, Routing, Water Balance).
  • Optional Keycloak authentication protects routes while keeping Swagger/OpenAPI usable.

Why it is useful

  • Centralizes complex GIS/business logic behind stable HTTP endpoints.
  • Supports automated workflows (field operations, routing) without QGIS client coupling.
  • Keeps API surface modular and controllable per deployment.

How to use it

  • Base path: ${API_ROOT}/v1 (default /giswater/v1; Swagger at ${API_ROOT}/v1/docs on each tenant host). Platform admin API: ${API_ROOT}/admin (Swagger at ${API_ROOT}/admin/docs on the apex host only). Override API_ROOT to remount everything (e.g. /gw-api for legacy URLs).
  • Deploy behind nginx on the host: see deploy/nginx.conf.example. The compose file binds the app to 127.0.0.1:8000.
  • Tenants are resolved from the leftmost host label of <tenant>.<BASE_DOMAIN> (e.g. acme.bgeo360.com).
  • Each tenant owns its own DB pool, Keycloak IDP, file/DB logger, and API_* toggles via config/tenants/<tenant>.env.
  • Use REST endpoints directly or via the autogenerated OpenAPI client.

πŸ”„ Compatibility

This service is versioned alongside the Giswater database. Use matching ranges to avoid subtle schema/API mismatches.

giswater-api version Supported Giswater DB versions
0.1 – 0.7 4.0 – 4.7
0.8 – 1.x 4.8+

When GISWATER_DB_VERSION_CHECK=true, tenant readiness (GET ${API_ROOT}/v1/ready) returns 503 if {DB_SCHEMA}.sys_version does not report a giswater version β‰₯ GISWATER_DB_MIN_VERSION (default 4.8.0).

πŸš€ Quick Start

1️⃣ Clone the Repository

git clone https://github.com/Giswater/giswater-api.git
cd giswater-api

2️⃣ Create local env files

cp .env.example .env
cp config/tenants/example.env config/tenants/test.env

3️⃣ Run with Docker Compose (recommended)

docker compose up --build -d

Check service health:

curl http://127.0.0.1:8000/giswater/health

πŸ“Œ Tenant API docs (host-based): http://<tenant>.<BASE_DOMAIN>:8000/giswater/v1/docs (example: http://test.bgeo360.localhost:8000/giswater/v1/docs)

Tenant routes (including /giswater/v1/docs) require a tenant context. In browser, use host-based routing:

  • BASE_DOMAIN=bgeo360.localhost in .env
  • open http://test.bgeo360.localhost:8000/giswater/v1/docs (for tenant file config/tenants/test.env)

For API clients on localhost/IPs, set DEV_ALLOW_TENANT_HEADER=true and send X-Tenant-ID: <id>.

4️⃣ Stop services

docker compose down

Optional: run without Docker

python3 -m venv venv
source venv/bin/activate  # Windows PowerShell: .\venv\Scripts\activate
pip install -e ".[dev]"
uvicorn app.main:app --reload

CLI (operator scripts)

After pip install -e ., the giswater-api console script reuses the same service layer as the HTTP API:

# List configured tenants
giswater-api admin tenants list

# Check tenant readiness
giswater-api tenant --tenant test --schema ws_40 ready

# Insert a hydrometer (example)
giswater-api tenant --tenant test --schema ws_40 --user postgres crm insert-hydrometer \
  --code H1 --hydro-number HN1

# Database migrations for the API-owned gwapi schema
giswater-api db upgrade --all          # or --tenant <id>
giswater-api db current --tenant test
giswater-api db history

Use --tenants-dir to override TENANTS_DIR when not running via the FastAPI lifespan.

The gwapi schema (basic-auth tables and audit logs) is managed by Alembic and, by default, migrated automatically on startup. See Database migrations.


βš™οΈ Configuration

Copy the env template and customize:

cp .env.example .env

Multi-tenant layout

.env                              # global config only
config/tenants/<tenant_id>.env    # one file per tenant (filename stem = tenant id = subdomain)

Template files named example.env, sample.env, and template.env are ignored by tenant discovery.

Tenant id rules: ^[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$. Reserved (rejected at load): www, api, admin, static, traefik, localhost.

Environment variables

Full reference (tables + explanations): docs/ENVIRONMENT_VARIABLES.md.

Templates: root .env.example, production deploy/.env.prod.example + deploy/main.env.example, per-tenant config/tenants/example.env.

Summary for quick orientation:

BASE_DOMAIN=bgeo360.com
TENANTS_DIR=config/tenants
API_ROOT=/giswater

LOG_DIR=logs
LOG_LEVEL=INFO
LOG_DB_ENABLED=true
LOG_DB_SAMPLE_RATE=1.0
LOG_HTTP_BODY_CAPTURE=true
LOG_DB_MAX_BODY_BYTES=2048

ADMIN_USER=admin
ADMIN_PASSWORD=<set me>
ADMIN_RELOAD_ENABLED=true
ADMIN_WRITE_ENABLED=true

DEV_ALLOW_TENANT_HEADER=false

PLATFORM_KEYCLOAK_ENABLED=false
PLATFORM_KEYCLOAK_URL=
PLATFORM_KEYCLOAK_REALM=
PLATFORM_KEYCLOAK_CLIENT_ID=
PLATFORM_KEYCLOAK_CLIENT_SECRET=

Gunicorn overrides and every variable name are documented in docs/ENVIRONMENT_VARIABLES.md and mirrored in .env.example.

Per tenant (config/tenants/<id>.env) β€” API_* toggles, DB pool, optional Keycloak. See config/tenants/example.env and docs/ENVIRONMENT_VARIABLES.md.

Connection budget

Each tenant has its own pool. Total connections to your Postgres β‰ˆ N_tenants Γ— DB_POOL_MAX_SIZE (defaults to 10). Plan accordingly.

Tenant CRUD / hot reload

  • GET /admin/tenants β€” list tenants (secrets redacted)
  • GET /admin/tenants/{id} β€” read one
  • POST /admin/tenants β€” create + write .env atomically
  • PUT /admin/tenants/{id} β€” full replace; secrets unset = keep existing
  • DELETE /admin/tenants/{id} β€” drains pool, archives .env to _archive/
  • POST /admin/tenants/{id}/reload β€” re-read one .env from disk
  • POST /admin/tenants/reload β€” rescan TENANTS_DIR

All /admin/* endpoints support either auth path: HTTP Basic with ADMIN_USER/ADMIN_PASSWORD, or a Bearer JWT from the platform Keycloak realm with role platform-admin.

Mutations gated by ADMIN_WRITE_ENABLED=true; full-dir reload also gated by ADMIN_RELOAD_ENABLED=true.


πŸ” Authentication

Tenant API auth is controlled per tenant by AUTH_MODE:

Mode Behavior
none (default) Anonymous access β€” same as 1.3.x with KEYCLOAK_ENABLED=false
keycloak Bearer JWT from the tenant Keycloak realm
basic HTTP Basic (Authorization: Basic …) validated against gwapi.users in the tenant DB

Migration from 1.3.x: no config change required. KEYCLOAK_ENABLED=true β†’ keycloak, false β†’ none. Set AUTH_MODE explicitly when convenient; KEYCLOAK_ENABLED is deprecated (removed in 2.0.0).

Basic auth: users live in the tenant Postgres (gwapi schema). Each user has a db_role that must exist as a PostgreSQL role (SET ROLE). Manage users via admin API: ${API_ROOT}/admin/tenants/{id}/users (requires tenant AUTH_MODE=basic).

Optional bootstrap: AUTH_BASIC_BOOTSTRAP_USER / AUTH_BASIC_BOOTSTRAP_PASSWORD create the first user when the table is empty.

Role checks: inject Depends(require_role("role_om", "role_master")) on routes; roles come from Keycloak JWT claims or gwapi.user_roles in basic mode.

Keycloak integration details (unchanged for AUTH_MODE=keycloak):

  • Endpoints require a valid Authorization: Bearer <token> issued by the tenant's realm.
  • Tokens are decoded against the tenant idp's RS256 public key (no realm round-trip per request).

When disabled (AUTH_MODE=none), endpoints accept anonymous requests for that tenant.

/admin/* uses a separate platform Keycloak realm (PLATFORM_KEYCLOAK_*) plus HTTP Basic, with platform-admin as the required role.

Swagger's built-in OAuth2 helper is no longer wired in β€” it can only point at one realm at a time, which is incompatible with multi-tenant. Clients send Authorization: Bearer ... directly.


πŸ”Œ Plugins

Custom plugins can extend the API. Place plugin folders in plugins/:

plugins/
└── my-plugin/
    └── ...

Plugins are auto-loaded at startup on the tenant API surface (${API_ROOT}/v1, default /giswater/v1; same feature toggles and OpenAPI as core routers). See example plugin.


🐳 Running with Docker

Use Compose as the default runtime:

docker compose up --build -d
docker compose ps
docker compose logs -f app

Dev hot reload:

cp docker-compose.override.yml.example docker-compose.override.yml
docker compose up --build

The override enables:

  • bind mount .:/app
  • uvicorn --reload
  • DEV_ALLOW_TENANT_HEADER=true

πŸ—οΈ Deployment Notes

Production installer (single-tenant)

On a Debian/Ubuntu server with Docker:

curl -fsSL https://raw.githubusercontent.com/Giswater/giswater-api/main/deploy/install.sh | sudo bash

Creates /opt/giswater-api with production Compose, .env (SINGLE_TENANT_ID=main), and config/tenants/main.env. See deploy/README.md.

Pin deploy templates: GISWATER_API_REF=v1.3.2 curl -fsSL ... | sudo bash (Docker image tag is prompted separately).

  • Keep proxy_set_header Host $host at the reverse proxy (deploy/nginx.conf.example) because tenant resolution depends on Host.
  • Apex host (BASE_DOMAIN) serves only ${API_ROOT}/admin/*; tenant hosts (<tenant>.<BASE_DOMAIN>) serve ${API_ROOT}/v1/*.
  • Base compose binds to 127.0.0.1:8000; expose publicly through your proxy/TLS layer.
  • Production images start Gunicorn + uvicorn.workers.UvicornWorker (see gunicorn.conf.py, Dockerfile). Override worker count with WEB_CONCURRENCY if needed.
  • Tune container/Kubernetes probes: allow enough start_period for Postgres pool init; ${API_ROOT}/health is fast; tenant ${API_ROOT}/v1/ready validates DB connectivity (and optional DB version check).

Logging (production)

Variable Default Meaning
LOG_HTTP_BODY_CAPTURE true When true, request/response payload text is logged for failed requests (4xx/5xx) with redaction + truncation; binary/multipart payloads are skipped.
LOG_DB_MAX_BODY_BYTES 2048 Max bytes per stored body (0 uses the same safe internal cap).
LOG_DB_ENABLED true Sample API rows into the tenant log table.
LOG_DB_SAMPLE_RATE 1.0 Fraction of tenant requests logged to DB (1.0 = all; typical for QGIS plugin workloads).

Detailed reference: docs/ENVIRONMENT_VARIABLES.md. Production installer: deploy/install.sh. Copy-paste templates: .env.example, deploy/.env.prod.example. Operator checklist: docs/DEPLOYMENT_CHECKLIST.md. Per-tenant template: config/tenants/example.env.

πŸ› οΈ API Endpoints

All surfaces live under ${API_ROOT} (default /giswater; override via the API_ROOT env var).

  • Tenant surface prefix: ${API_ROOT}/v1 (default /giswater/v1)
  • Admin surface prefix: ${API_ROOT}/admin (default /giswater/admin, apex host only)
  • Health checks:
    • ${API_ROOT}/health (global, tenant-independent)
    • ${API_ROOT}/v1/health (tenant scope)
    • ${API_ROOT}/admin/health (admin scope)
  • OpenAPI docs:
    • Tenant: ${API_ROOT}/v1/docs on tenant host
    • Admin: ${API_ROOT}/admin/docs on apex host
  • Module routers are loaded from:
    • basic
    • om (profile, flow, mincut, waterbalance, mapzones)
    • routing
    • crm
    • epa (dscenario)
    • system (ready, schema validation, tenant-scoped logs)

Use OpenAPI as source of truth for the full endpoint list in your running environment.


πŸ“‚ Project Structure

giswater-api/
│── app/
β”‚   │── main.py              # FastAPI app entry point (lifespan, sub-app mounts, middleware)
β”‚   │── api/                 # HTTP layer
β”‚   β”‚   │── deps.py          # Shared FastAPI dependencies (common_parameters, get_schema, require_feature)
β”‚   β”‚   │── exception_handlers.py  # Service error β†’ HTTP response mapping
β”‚   β”‚   │── v1/              # Versioned tenant API
β”‚   β”‚   β”‚   │── router.py    # Router wiring + per-tenant OpenAPI filter
β”‚   β”‚   β”‚   └── endpoints/   # Thin handlers; delegate to services/
β”‚   β”‚   └── admin/           # Platform-admin API (tenants.py, users.py, router.py)
β”‚   β”‚
β”‚   │── services/            # HTTP-agnostic business logic (API + CLI)
β”‚   │── cli/                 # Click CLI (`giswater-api` entry point)
β”‚   β”‚
β”‚   │── core/                # Dependency-free leaf: config.py, constants.py, exceptions.py
β”‚   │── auth/                # session.py, keycloak.py, users.py, schemas.py, constants.py
β”‚   │── db/                  # manager.py, context.py, execution.py, version.py, log_store.py, schema.py, partitions.py, migrate.py
β”‚   │── tenancy/             # registry.py, state.py, host_middleware.py
β”‚   │── middleware/          # request_logging.py
β”‚   │── schemas/             # Pydantic request/response models (basic/, crm/, om/, routing/, epa/, admin.py, common.py)
β”‚   │── utils/               # body.py, version.py, rate_limit.py, plugins.py, log_setup.py, routing.py
β”‚   └── static/              # Static files (favicon, logs UI, etc.)
β”‚
│── alembic/             # gwapi schema migrations (env.py + versions/)
│── alembic.ini          # Alembic config (dev CLI; runtime config is programmatic)
│── config/
β”‚   └── tenants/         # Per-tenant .env files (e.g. test.env, acme.env)
│── deploy/
β”‚   └── nginx.conf.example
│── plugins/             # Plugin directory (see plugins/readme.md)
│── tests/               # Tests
│── .github/workflows/   # CI/CD (ruff, pytest)
│── Dockerfile           # Docker build config
│── docker-compose.yml
│── docker-compose.override.yml.example
│── gunicorn.conf.py     # Gunicorn + Uvicorn worker defaults for production images
│── pyproject.toml       # Project metadata, dependencies, and tooling config
│── docs/
β”‚   β”œβ”€β”€ ARCHITECTURE.md            # Package map, dependency rules, code locations
β”‚   β”œβ”€β”€ VERSIONING.md              # API + per-tenant DB versioning policy
β”‚   β”œβ”€β”€ DATABASE_MIGRATIONS.md     # Alembic gwapi schema migrations + upgrade paths
β”‚   β”œβ”€β”€ DEPLOYMENT_CHECKLIST.md
β”‚   └── ENVIRONMENT_VARIABLES.md   # Human-readable env reference (tables + descriptions)
│── scripts/
β”‚   │── release.sh
β”‚   │── release.ps1
β”‚   └── smoke_test.sh
└── README.md

βœ… Testing & Linting

Run Tests

pytest

Run Linter

ruff check .

CI/CD runs both on push via GitHub Actions (.github/workflows/).


πŸš€ Releasing

  1. Update CHANGELOG.md under [Unreleased] and move the block to a dated ## [X.Y.Z] section; commit.
  2. Follow docs/DEPLOYMENT_CHECKLIST.md for production cutovers.
  3. Run the release script:
# Bash
./scripts/release.sh 1.0.0

# PowerShell
.\scripts\release.ps1 1.0.0

The script will:

  • Abort if there are uncommitted changes
  • Update the version in pyproject.toml
  • Commit, tag vX.Y.Z, create branch release/X.Y
  • Push everything to origin

πŸ“Œ License

This project is free software, licensed under the GNU General Public License (GPL) version 3 or later. Refer to the LICENSE file for details.

About

Fast-API for Giswater

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors