This is mainly a [Django] web app for ARRC radar archive. There is also a [Django]-[Channels] interface for live radar operations.

Three main parts:

• [Django] backend
• [React] frontend
• [Python] backhaul

Some design constraints:

• Keep backend-frontend responsive, only async functions
• All input/output gets picked up by backhaul
• Backhaul serves as the middle party to facilitate one-to-one or one-to-many messaging
• Payloads from a radar get broadcast to all users in a specific group by worker backhaul
• A radar reporter does not receive anything from the radar itself (different behavior than a chat room)

Currently, the RadarHub is almost like a chat program with an exception that the messages are not echoed back. The radars do not recieve the payload they send home and the users do not see the command they issue. Everyone connects through the frontend websocket, either join as a radar (frontend.consumers.Radar) or join as a user (frontend.consumers.User). When a radar joins and sends in proper greeting, it receives a welcome message from the hub. When a user joins through a web browser, the frontend javascript main script ensures the connection is properly made, the user is assigned to a group, named after the radar name.

When a user request is issued, it is first received by the frontend, which checks for the required fields. If failed, nothing happens. Otherwise, it is routed to the backhaul asynchronously. Frontend immediately regain control, GUI should always be responsive. Backhaul decides which radar to send the request to, awaits for the radar response, and then routes it to the user.

When a radar joins the RadarHub, it reports its name. Backhaul launches a runloop to collect data streams from the radar. This runloop also sends whatever data stream available from the radar to the group. All users in that group receive the same data stream. This will change in the future for a more controlled fashion but kept simple at the moment for progressing towards the subsequent milestones.

Here are some assumptions that developers should be aware of.

Currently defined in reporter/types.h, which is arguably sub-optimal. Perhaps the project root folder is better. Will revisit.

In the C space, obviously you just include the header and you are done.

In the Python space, a parser is made to retrieve everythin in the C header to a Python enums through reporter/enums.py. All the types will be in the convention of

• RadarHubType.Handshake
• RadarHubType.Control
• RadarHubType.Health
• RadarHubType.Scope
• ...

In the Javascript space, the definition is passed to the frontend upon a successful connection, the data ingest frontend/src/components/ingest.js expects keys like Control, Health, Scope, etc., which nicely maps to the enum names in the C and Python spaces.

frontend.consumers.User

• message from web UI is always in text form (JSON)
• message to web UI is always in binary form (bytearray)

frontend.consumers.Radar

• message from radar is always in binary form ([type][payload])
• message to radar is always in text form (plain text)

This section details the file setup and actions required to setup a deployment of RadarHub on a Linux host using Docker Compose. Docker Compose simplifies the deployment process by taking advantage of container technology to provision a runtime environment for the various services (RadarHub, Redis, Nginx, and Postgres) without polluting host system with additional libraries or runtimes.

The host is the machine running Docker on which the RadarHub services will be deployed using Docker Compose. These steps have been tested on a system running Ubuntu 24.04. It is assumed Docker Engine has been installed on the host system and the Linux post-install steps have been applied.

The Redis service requires the host machine's kernel be configured to overcommit memory. This is achieved by setting vm.overcommit_memory=1 in /etc/sysctl.conf. See overcommit-accounting in the kernel docs for more info.

It might be prudent to check /etc/sysctl.conf for a line containing vm.overcommit_memory, but if the system is fresh it probably isn't configured. Use the commands below to set overcommit to mode 1.

# Add vm.overcommit_memory=1 to sysctl
echo "vm.overcommit_memory=1" | sudo tee -a /etc/sysctl.conf

# Reload without reboot
sudo sysctl -p

There are two approaches to deploying RadarHub via Docker Compose. The minimal approach does not require cloning the repository, but does require Both approaches assume the host machine is online, connected to the internet, and its shell can be accessed (either locally or remotely via SSH).

• Deploy Only: download the docker-compose.yml file and use Docker Compose to pull the service images and deploy.
• Build and Deploy: clone the RadarHub repo and use Docker Compose to build the images and deploy.

Coming Soon. (Need to push built images to git.arrc.ou.edu container registry to make this work.)

The following steps should be performed on the machine which will host the RadarHub deployment. The user account that performs the following operations should have sudo access and be added to the docker group (See Host Configuration above).

On the deployment host, clone the repository:

git clone https://git.arrc.ou.edu/radar/radarhub.git

The remaining instructions assume the radarhub repository directory is the working directory. Move into the radarhub directory:

cd radarhub

Create a .env file with the appropriate values for the deployment environment (development vs. production). See .env file below for more information.

touch .env
# {vi, vim, nano, emacs} .env

Create a settings.json file and set the value of RH_SETTINGS_JSON in the .env file to the path to this file. A settings.json file looks something like this:

{
    "radars": {
      "px1000": { "name": "PX-1000", "prefix": "PX", "folder": "PX1000", "summary": "E4.0" },
      "raxpol": { "name": "RaXPol", "prefix": "RAXPOL", "folder": "RaXPol", "summary": "E4.0" },
    }
  }

TODO This section should be flushed out when an file share (NFS?) for instances of RadarHub is documented. This will allow developers to load data in RadarHub without having to copy files from the production server to a development machine.

Create a mount location (network share) on the host machine which has access to the radar data for RadarHub or copy some data to the host machine.

Set the value of RH_RADARHUB_DATA and RH_DATASHOP_DATA in the .env file to the directory the radar data is located. This directory should have subdirectories for the various radar systems like so:

$ tree -L 1 /mnt/data
/mnt/data
├── PX1000
├── PX10k
└── RaXPol

To build and run the RadarHub deployment, run the following command:

docker compose up -d

• RH_DATABASE_HOST - the hostname for the SQL database server (Postgres, Docker Postgres), this can be the name of the service that the docker-compose.yml file creates "db" or an external host address, if the DBMS is hosted elsewhere. This will pull and build the necessary images, configure the services (based on .env), and start the deployment. This deployment will persist and resume across reboots and persist database data. Database data will persist either to a docker volume on the local host or to the remote DBMS depending on the values specified in .env for RH_DATABASE_HOST, RH_DATABASE_NAME, RH_DATABASE_USER, and RH_DATABASE_PASS (See .env file below for more information).

NOTE The deployment process via Docker Compose builds the required images based on the DockerFile found in the RadarHub repository. This process requires access to the internet, specifically hub.docker.com and arrc.ou.edu. If these sites are not available via DNS, then the build cannot successfully complete. You may use normal Docker configuration settings to proxy Docker Hub for the image pulls. The ARRC website is used to download static maps files and must be available.

If the local database instance created by docker-compose.yml is used, then the data will need to be imported into the database. If a remote DBMS instance is used, then it is likely the data is already in it. To import the radar data files into the database, run:

docker compose exec radarhub bash

Now, inside the radarhub service container, run:

python dbtool.py -i /mnt/data/PX1000/2025/20250414

For each folder of radar data to be imported, modify the above command to import the required folder.

TODO Make the dbtool.py import bulk data with a single, user-friendly command.

Congratulations! RadarHub should now be live at the port specified by RH_FE_HTTP_PORT or RH_FE_HTTPS_PORT on the host machine (e.g. http://localhost:8080).

If the deployment of RadarHub is not online, use the docker compose logs and docker compose exec commands to troubleshoot.

The following values should be added to a file called .env in the same directory as the docker-compose.yml file on the host. They will be used by Docker Compose as secrets passed to the services.

• COMPOSE_PROJECT_NAME - the project name for Docker compose.
• RH_DJANGO_INIT - initialize the database when the project up for the first time.
• RH_DJANGO_DEBUG - controls whether the RadarHub Django app is run in Debug mode. Should be omitted or set to "False" for production environments. Set to "True" for additional logs and debug webpages in development environments.
• RH_DJANGO_SECRET - sets the secret key used by Django for various secure web operations. See the Django secret key docs for more information. This should be set to a strong and secure value for production environments.
• RH_DATABASE_HOST - the hostname of the SQL database server (Postgres, Docker Postgres), this is the name of the service that the docker-compose.yml file creates "db" or an external host address, if the DBMS is hosted elsewhere.
• RH_DATABASE_NAME - the name of the RadarHub database in the DBMS.
• RH_DATABASE_USER - the username of a user with access to the RadarHub database in the DBMS.
• RH_DATABASE_PASS - the password of the user specified by RH_DATABASE_USER in the DBMS.
• RH_FE_HTTP_PORT - the port to bind on the host machine at which RadarHub will be served via HTTP.
• RH_FE_HTTPS_PORT - the port to bind on the host machine at which RadarHub will be served via HTTPS.
• RH_RADARHUB_DATA - a directory location on the host machine to mount which provides radar data access to the radarhub service.
• RH_REDIS_HOSTPORT - the host and port of the redis service.
• RH_DATASHOP_HOST - the hostname of the datashop service.
• RH_DATASHOP_PORT - the port to bind on the host machine for the datashop service.
• RH_DATASHOP_DATA - a directory location on the host machine to mount which provides radar data access to the datashop service.
• RH_PGADMIN_EMAIL - an email address to associate with the default account for PG Admin.
• RH_PGADMIN_PASS - the password for the default account for PG Admin.
• RH_PGADMIN_PORT - the port ot bind on the host machine at which PG Admin will be served via HTTP.
• RH_SETTINGS_JSON - the file location on the host machine that contains the settings for the radarhub service.
• RH_LDM_PORT - the port to bind on the host machine for the LDM service.
• RH_LDM_DATA - a directory location on the host machine to mount which provides radar data acess to the ldm service.

NOTE: RH_DATASHOP_DATA and RH_RADARHUB_DATA should be the same if both radarhub and datashop services are running on the same host.

Example .env file:

COMPOSE_PROJECT_NAME="radarhub"
RH_DJANGO_INIT=False
RH_DJANGO_DEBUG=False
RH_DJANGO_SECRET="django-insecure-5zk9_rg=98@@h+e6*iw63l9h*v_bo9+_xum)"
RH_DATABASE_HOST="db"
RH_DATABASE_NAME="radarhub"
RH_DATABASE_USER="radarhub"
RH_DATABASE_PASS="verysecurepassword1"
RH_REDIS_HOSTPORT="redis://redis:6379"
RH_FE_HTTP_PORT=8080
RH_FE_HTTPS_PORT=8443
RH_RADARHUB_DATA="./test-volumes/radarhub"
RH_DATASHOP_DATA="./test-volumes/datashop"
RH_DATASHOP_HOST="datashop"
RH_PGADMIN_EMAIL="arrcengineering@groups.ou.edu"
RH_PGADMIN_PASS="verysecurepassword2"
RH_PGADMIN_PORT=9080
RH_SETTINGS_JSON="./config/settings.json"

A Dev Container configuration is provided to enable easy configuration of a development environment which closely matches the environment configuration for production while allowing for additional tooling. With the Dev Containers extension for Visual Studio Code, use Ctrl/Cmd + Shift + P or F1 to open the command palette and select Dev Containers: Rebuild and Reopen in Container to launch the Dev Container environment.

The file .devcontainer/devcontainer.json contains the configuration for the environment. See the Dev Containers documentation for instructions to customize the environment.

The Dev Container environemnt uses the same container as the RadarHub service which provides Python, Node.js, and NPM. The RadarHub service container is based on the Debian Bookworm distro container. So, you can always apt-get install whatever additional tools you would like. The Dev Container configuration specification provides a convenient mechanism for automating additional tool/library installs as well. Just be sure to add any system packages to radarhub.Dockerfile if RadarHub will need them to function.

For the old instructions on setting up a machine to develop RadarHub without Docker Compose or Dev Containers, see READMEDEV.md.