Cloud Engine

Workflow-based Topology Provisioning Orchestrator for Hybrid Cloud Infrastructure

Cloud Engine is a hybrid cloud orchestration platform that provisions and manages network slices — groups of interconnected virtual machines — across heterogeneous compute infrastructure. Organizations using Cloud Engine may run their workloads entirely on-premise, entirely on public cloud, or across both simultaneously: the platform treats on-premise KVM/libvirt clusters, private OpenStack deployments, and public cloud providers (AWS, in progress) as interchangeable backends behind a single unified API. The target infrastructure belongs to the organization; Cloud Engine is the control plane that drives it.

All provisioning is orchestrated through Temporal durable workflows with topological DAG execution and automatic saga-style rollback on failure. A suite of seven FastAPI microservices handles the platform logic: quota management, topology-aware VM placement (via Google OR-Tools), network address allocation, and user/workspace lifecycle. The frontend is a Next.js single-page application with a visual drag-and-drop slice topology editor and an in-browser noVNC console.

Cloud Engine was built to lower the operational barrier of multi-VM network experimentation for the people who need it most but rarely have a purpose-built tool for it: IT teams managing shared lab infrastructure across multiple projects, system administrators who spend hours manually scripting VM and network setup for each tenant, researchers who need reproducible, isolated network topologies without touching the underlying hypervisor, and education centers running networking or distributed systems courses where every student group needs their own isolated environment on demand. Instead of writing provisioning scripts or navigating raw hypervisor CLIs, users describe a topology — which VMs exist, how they connect, what firewall rules apply — and Cloud Engine handles the rest: placement, VLAN assignment, port wiring, image import, and full lifecycle management, with durable retries and rollback built in.

• Dual infrastructure support — same workflow engine provisions VMs on KVM/libvirt (SSH + bash + OVS) and OpenStack (Nova/Neutron/Glance/Keystone), selected per slice at runtime
• AWS driver (in progress) — third infrastructure backend targeting EC2/VPC/ENI/Security Groups; the same Temporal DAG workflow and activity interface apply with a new aws-cluster-activity-worker polling aws-cluster-queue
• Durable workflow orchestration — Temporal-based DAG interpreter runs provisioning steps in parallel phases; saga compensation stack undoes completed steps on any failure
• Topology-aware VM placement — Google OR-Tools bin-packing assigns VMs to physical servers based on real-time CPU/RAM/disk availability queried from Prometheus
• Multi-domain RBAC — platform roles (administrador, coordinador, usuario_estandar) combined with per-workspace roles (propietario, colaborador); admins can act on any workspace without membership
• Visual slice editor — React Flow canvas for drag-and-drop VM and link topology design, with property panels and named template save/load
• In-browser VNC console — noVNC client routed through a VNC token service and per-cluster websockify proxy; tokens are time-limited and connection params are snapshotted at issuance
• Workspace quota system — per-workspace vCPU/RAM/storage/slice-count limits enforced transactionally; workspace types define quota tiers; restricted types require admin approval
• Slice operator assignments — workspace members can delegate slice operation access to specific users outside their workspace
• End-to-end structured logging — all services emit JSON logs (structlog) collected by Grafana Alloy via the Docker socket and stored in Loki; trace_id from Kong's correlation-id plugin spans the full request path
• Real-time cluster metrics — custom ce_exporter (Prometheus, port 9200) on each compute node exposes host CPU/RAM/disk and per-VM metrics at 1-second resolution; pre-built Grafana dashboards per infrastructure
• Kong API gateway — DB-less declarative config with RS256 JWT validation (Keycloak-issued), CORS, per-consumer rate limiting, and correlation ID injection
• TEST_MODE — bypass Temporal entirely; the slice-manager prints the full WorkflowSpec to stdout and immediately simulates completion, enabling end-to-end planning tests without cluster infrastructure

  Browser
     │
     ▼
  Nginx (TLS :443)
     │
     ├──► Next.js Frontend (React Flow editor, noVNC, role-based routing)
     │
     └──► Kong API Gateway (RS256 JWT · CORS · rate-limit · trace-id)
               │
               ▼
        Business Layer — 7 FastAPI microservices
        ┌──────────┬──────────┬────────────┬─────────────────┐
        │  slice-  │ catalog  │   quota-   │   placement     │
        │ manager  │          │ accounting │  (OR-Tools)     │
        ├──────────┴──────────┴────────────┴─────────────────┤
        │  networking-security  │  vnc-proxy  │    admin-     │
        │  (VLAN/MAC/IP/port)   │  (WS relay) │  management   │
        └───────────────────────┴─────────────┴───────────────┘
               │                         │
          PostgreSQL                Temporal gRPC
           (mydb)                       │
                                        ▼
                              Temporal Workflow Engine
                       ┌────────────────────────────────┐
                       │  create-slice  (DAG + saga)     │
                       │  delete-slice  (DAG, no saga)   │
                       │  manage-vm     (power actions)  │
                       └────────────────────────────────┘
                                        │
               ┌─────────────────────── ┴ ───────────────────────────┐
               ▼                                                       ▼
   linux-cluster-queue                               openstack-cluster-queue
   KVM Activity Worker                               OpenStack Activity Worker
   SSH + bash scripts + OVS + libvirt/QEMU           Nova · Neutron · Glance · Keystone
               │                                                       │
   Linux Cluster Headnode API                        OpenStack cluster direct API calls
   (FastAPI + paramiko)
               │
               ├──► business-layer-queue (notify.* activities → slice-manager internal)
               │
               └──► Observability
                    Alloy → Loki → Grafana (logs)
                    ce_exporter → Prometheus → Grafana (metrics)

The workflow worker interprets WorkflowSpec as a topological DAG: it repeatedly selects steps whose declared dependencies are satisfied and runs them in parallel via asyncio.gather. Activities prefixed notify.* are always routed to business-layer-queue (calls back into slice-manager); all other activities go to the infrastructure-specific driver queue (linux-cluster-queue or openstack-cluster-queue) determined by the target infrastructure's configuration.

AWS support (in progress): A third driver (aws-cluster-activity-worker) is planned for EC2/VPC. The workflow DAG is infrastructure-agnostic by design — only the activity implementations change. Network control at the VLAN level is replaced by VPC subnets, ENIs, and Security Groups, preserving the topology abstraction.

• Docker 24+ and Docker Compose v2
• A reachable KVM cluster (headnode IP + SSH private key) and/or OpenStack cluster with admin credentials — or set TEST_MODE=true to skip infrastructure entirely and test the planning layer only
• All .env files populated (see Configuration)
• TLS certificates — generated automatically by scripts/setup.sh (self-signed)

git clone <repo-url>
cd Cloud_Engine_Project

# 1. Copy and fill in the env files
cp business_layer/.env.example business_layer/.env
cp linux-cluster-services/.env.example linux-cluster-services/.env
cp openstack-cluster-services/.env.example openstack-cluster-services/.env
# ... (see Configuration below)

# 2. Bring up the full stack
bash scripts/setup.sh

# Access points:
#   Web UI:        https://localhost
#   Temporal UI:   http://127.0.0.1:8081
#   Grafana:       https://localhost/grafana

Setup script flags:

To rebuild a single service without restarting the whole stack:

docker compose up -d --build slice-manager

Note: After recreating any Kong-fronted service, run docker compose restart kong — Kong caches the upstream IP and will return 502s until restarted.

Each component reads from its own .env file. scripts/validate.sh checks all of them for unfilled placeholder values.

Cloud_Engine_Project/
├── docker-compose.yml                   # Canonical unified compose (26 services)
├── scripts/
│   ├── setup.sh                         # First-time bring-up: validate → TLS → Kong config → compose up
│   └── validate.sh                      # .env completeness and cross-service consistency checks
│
├── business_layer/                      # Internal microservices + shared DB + Grafana
│   ├── database/                        # PostgreSQL schema (db_cloud_v1_postgres.sql) + seed data
│   ├── slice_manager/                   # Orchestrator — sole external API entry point
│   ├── image_catalog/                   # VM image and flavor management
│   ├── quota_accounting/                # Workspace quota enforcement
│   ├── placement/                       # VM-to-server bin-packing (OR-Tools)
│   ├── networking_security/             # VLAN/MAC/IP/VNC-port allocation
│   ├── vnc_proxy/                       # WebSocket relay to cluster websockify
│   ├── admin_management/                # User CRUD + Keycloak sync
│   └── grafana/                         # Pre-provisioned dashboards (metrics + logs)
│
├── workflow_engine/
│   ├── temporal_server/                 # Temporal server config + namespace init scripts
│   ├── workflow_worker/                 # DAG interpreter: create-slice, delete-slice, manage-vm
│   ├── linux_cluster_activity_worker/   # KVM driver activities (polls linux-cluster-queue)
│   ├── openstack_cluster_activity_worker/ # OpenStack driver activities (polls openstack-cluster-queue)
│   └── business_layer_activity_worker/  # notify.* activities (polls business-layer-queue)
│
├── linux-cluster-services/              # Headnode REST API (FastAPI + paramiko + OVS/libvirt)
├── linux_driver_scripts/                # Bash scripts executed on cluster nodes via SSH
├── openstack-cluster-services/          # VNC token service + websockify + metrics for OpenStack
├── observability/                       # Loki config + Grafana Alloy log pipeline
├── frontend/
│   ├── keycloak/                        # Realm JSON import (fixed RSA key pair)
│   ├── kong/                            # DB-less declarative gateway config
│   └── webui/                           # Next.js 16 App Router application
└── documentation/                       # API spec, flow examples, changelogs

All requests are routed through Kong (https://localhost/api/...). Kong validates the RS256 JWT issued by Keycloak and injects user_id and system_role claims.

See business_layer/documentation/API.md for the full request/response schema reference.

A custom ce_exporter (port 9200) runs on each compute node and exposes:

• Host metrics — ce_host_cpu_total, ce_host_memory_total_bytes, ce_host_filesystem_size_bytes, and their utilization counterparts
• VM aggregate metrics — ce_host_vm_cpu_seconds_total, ce_host_vm_vcpu_total, ce_host_vm_memory_rss_bytes, ce_instances_used_bytes

Prometheus scrapes at 1-second resolution. Grafana dashboards are provisioned per infrastructure at startup; the Cloud Engine — Infraestructura dashboard provides CPU/RAM/disk timeseries alongside scheduler reservation data from PostgreSQL.

All services emit structured JSON logs with a shared schema:

Grafana Alloy collects logs from all containers via the Docker socket, parses the JSON, and pushes to Loki. The Cloud Engine — Logs Grafana dashboard enables log exploration filtered by service, level, trace_id, or slice_id.

Set TEST_MODE=true in business_layer/.env to run the full planning path (quota check, placement, network allocation, WorkflowSpec construction) without a live Temporal server or cluster infrastructure:

# business_layer/.env
TEST_MODE=true

The slice-manager will print the complete WorkflowSpec or DeleteWorkflowSpec JSON to stdout and immediately transition the slice to activo / eliminado, skipping all Temporal and driver interactions. This is useful for iterating on the planning layer in isolation.

1. Fork the repository and create a feature branch off main
2. Keep changes scoped — one logical change per pull request
3. Open a pull request against main with a clear description of the change and how to test it
4. No formal CLA required; contributions are accepted under Apache 2.0

Licensed under the Apache License 2.0.

Copyright 2026 Tony F. & Christian F.