Skip to content

dominikschulz/rest-server

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

627 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Rest Server

Status badge for CI tests Go Report Card GoDoc License Powered by

Rest Server is a high performance HTTP server that implements restic's REST backend API. It provides secure and efficient way to backup data remotely, using restic backup client via the rest: URL.

Requirements

Rest Server requires Go 1.24 or higher to build. The only tested compiler is the official Go compiler.

The required version of restic backup client to use with rest-server is v0.7.1 or higher.

Build

For building the rest-server binary run CGO_ENABLED=0 go build -o rest-server ./cmd/rest-server

Usage

To learn how to use restic backup client with REST backend, please consult restic manual.

$ rest-server --help

Run a REST server for use with restic

Usage:
  rest-server [flags]

Flags:
      --append-only                  enable append only mode
      --cpu-profile string           write CPU profile to file
      --debug                        output debug messages
      --group-accessible-repos       let filesystem group be able to access repo files
  -h, --help                         help for rest-server
      --htpasswd-file string         location of .htpasswd file (default: "<data directory>/.htpasswd)"
      --listen string                listen address (default ":8000")
      --log filename                 write HTTP requests in the combined log format to the specified filename (use "-" for logging to stdout)
      --max-size int                 the maximum size of the repository in bytes
      --no-auth                      disable .htpasswd authentication
      --no-verify-upload             do not verify the integrity of uploaded data. DO NOT enable unless the rest-server runs on a very low-power device
      --path string                  data directory (default "/tmp/restic")
      --private-repos                users can only access their private repo
      --prometheus                   enable Prometheus metrics
      --prometheus-no-auth           disable auth for Prometheus /metrics endpoint
      --proxy-auth-username string   specifies the HTTP header containing the username for proxy-based authentication
      --tls                          turn on TLS support
      --tls-cert string              TLS certificate path
      --tls-key string               TLS key path
      --tls-min-ver string           TLS min version, one of (1.2|1.3) (default "1.2")
  -v, --version                      version for rest-server

By default the server persists backup data in the OS temporary directory (/tmp/restic on Linux/BSD and others, in %TEMP%\\restic in Windows, etc). If rest-server is launched using the default path, all backups will be lost. To start the server with a custom persistence directory and with authentication disabled:

rest-server --path /user/home/backup --no-auth

To authenticate users (for access to the rest-server), the server supports using a .htpasswd file to specify users. By default, the server looks for this file at the root of the persistence directory, but this can be changed using the --htpasswd-file option. You can create such a file by executing the following command (note that you need the htpasswd program from Apache's http-tools). In order to append new user to the file, just omit the -c argument. Only bcrypt and SHA encryption methods are supported, so use -B (very secure) or -s (insecure by today's standards) when adding/changing passwords.

htpasswd -B -c .htpasswd username

If you want to disable authentication, you must add the --no-auth flag. If this flag is not specified and the .htpasswd cannot be opened, rest-server will refuse to start.

NOTE: In older versions of rest-server (up to 0.9.7), this flag does not exist and the server disables authentication if .htpasswd is missing or cannot be opened.

By default the server uses HTTP protocol. This is not very secure since with Basic Authentication, user name and passwords will be sent in clear text in every request. In order to enable TLS support just add the --tls argument and add a private and public key at the root of your persistence directory. You may also specify private and public keys by --tls-cert and --tls-key and set the minimum TLS version to 1.3 using --tls-min-ver 1.3.

Signed certificate is normally required by the restic backend, but if you just want to test the feature you can generate password-less unsigned keys with the following command:

openssl req -newkey rsa:2048 -nodes -x509 -keyout private_key -out public_key -days 365 -addext "subjectAltName = IP:127.0.0.1,DNS:yourdomain.com"

Omit the IP:127.0.0.1 if you don't need your server be accessed via SSH Tunnels. No need to change default values in the openssl dialog, hitting enter every time is sufficient. To access this server via restic use --cacert public_key, meaning with a self-signed certificate you have to distribute your public_key file to every restic client.

The --append-only mode allows creation of new backups but prevents deletion and modification of existing backups. This can be useful when backing up systems that have a potential of being hacked.

To prevent your users from accessing each others' repositories, you may use the --private-repos flag which grants access only when a subdirectory with the same name as the user is specified in the repository URL. For example, user "foo" using the repository URLs rest:https://foo:pass@host:8000/foo or rest:https://foo:pass@host:8000/foo/ would be granted access, but the same user using repository URLs rest:https://foo:pass@host:8000/ or rest:https://foo:pass@host:8000/foobar/ would be denied access. Users can also create their own subrepositories, like /foo/bar/.

Rest Server uses exactly the same directory structure as local backend, so you should be able to access it both locally and via HTTP, even simultaneously.

Systemd

There's an example systemd service file included with the source, so you can get Rest Server up & running as a proper Systemd service in no time. Before installing, adapt paths and options to your environment.

Docker

Rest Server works well inside a container, images are published to Docker Hub.

Start server

You can run the server with any container runtime, like Docker:

    docker pull restic/rest-server:latest
    docker run -p 8000:8000 -v /my/data:/data --name rest_server restic/rest-server

Note that:

  • contrary to the defaults of rest-server, the persistent data volume is located to /data.
  • By default, the image uses authentication. To turn it off, set environment variable DISABLE_AUTHENTICATION to any value.
  • By default, the image loads the .htpasswd file from the persistent data volume (i.e. from /data/.htpasswd). To change the location of this file, set the environment variable PASSWORD_FILE to the path of the .htpasswd file. Please note that this path must be accessible from inside the container and should be persisted. This is normally done by bind-mounting a path into the container or with another docker volume.
  • It's suggested to set a container name to more easily manage users (--name parameter to docker run).
  • You can set environment variable OPTIONS to any extra flags you'd like to pass to rest-server.

Customize the image

The published image is built from the Dockerfile available on this repository, which you may use as a basis for building your own customized images.

    git clone https://github.com/restic/rest-server.git 
    cd rest-server
    docker build -t restic/rest-server:latest .

Manage users

Add user
docker exec -it rest_server create_user myuser

or

docker exec -it rest_server create_user myuser mypassword
Delete user
docker exec -it rest_server delete_user myuser

Proxy Authentication

See above for no authentication (--no-auth) and basic authentication.

To delegate authentication to a proxy, use the --proxy-auth-username flag. The specified header name, for example X-Forwarded-User, must be present in the request headers and specifies the username. Basic authentication is disabled when this flag is set.

Warning: rest-server trusts the username in the header. It is the responsibility of the proxy to ensure that the username is correct and cannot be forged by an attacker.

Prometheus support and Grafana dashboard

The server can be started with --prometheus to expose Prometheus metrics at /metrics. If authentication is enabled, this endpoint requires authentication for the 'metrics' user, but this can be overridden with the --prometheus-no-auth flag.

This repository contains an example full stack Docker Compose setup with a Grafana dashboard in examples/compose-with-grafana/.

Group-accessible Repositories

Rest-server supports making repositories accessible to the filesystem group by setting the --group-accessible-repos option. Note that permissions of existing files are not modified. To allow the group to read and write file, use a umask of 007. To only grant read access use 027. To make an existing repository group-accessible, use chmod -R g+rwX /path/to/repo.

Disk Space Reservation

When the underlying filesystem fills up completely, restic clients can no longer create lock files, causing backups to hang or fail with confusing errors. Rest Server can reserve a minimum amount of free disk space exclusively for lock file writes, so that lock operations always succeed even when the disk is nearly full.

The reservation is controlled with the --disk-reserve flag (default: 100 MiB):

# Reserve 200 MiB for lock files (non-lock writes rejected below this threshold)
rest-server --path /data --disk-reserve $((200 * 1024 * 1024))

# Disable the reservation entirely
rest-server --path /data --disk-reserve 0

When free space drops below the configured threshold:

  • Lock file writes (/locks/…) are always allowed — lock files are tiny (a few hundred bytes) and are essential for safe concurrent access.
  • All other writes (data, index, snapshots, keys, config) are rejected with HTTP 507 Insufficient Storage.

The free-space check uses a short cache (refreshed every 5 seconds) to avoid a statfs call on every request. When Prometheus metrics are enabled, two additional gauges are exported:

Metric Description
rest_server_disk_free_bytes Current free disk space in bytes on the data volume
rest_server_disk_reserve_active 1 when the reserve is blocking non-lock writes, 0 otherwise

The disk reservation and --max-size quota are independent and complementary: --max-size limits one repository's tracked size, while --disk-reserve guards against the filesystem filling up from any cause.

Per-User and Per-Repo Quotas

When running rest-server with multiple users or repositories you may want to limit how much space each individual user or repository can consume. The --quota-file flag enables file-based per-user and per-repo size limits.

Quota file format

The quota file uses a simple line-based format:

# Lines starting with # are comments; blank lines are ignored.

# Default limits applied to any user/repo without a specific entry
default:user  50G
default:repo  25G

# Per-user limits (total across all repos owned by that user)
user:alice    200G
user:bob      100G
user:intern   5G

# Per-repo limits (specific repo path)
repo:alice/photos     100G
repo:alice/desktop    80G
repo:ci-bot/nightly   50G

Size suffixes: K (KiB), M (MiB), G (GiB), T (TiB). Plain integers are bytes.

Key types:

Type Format Meaning
user:<name> user:alice 200G Total limit for all repos owned by alice
repo:<path> repo:alice/laptop 20G Limit for the specific repo alice/laptop
default:user default:user 50G Fallback user limit when no specific entry exists
default:repo default:repo 25G Fallback repo limit when no specific entry exists

Enabling per-user/per-repo quotas

rest-server --path /srv/restic --quota-file /etc/rest-server/quota.conf

At startup, rest-server tallies existing on-disk usage and initializes counters for each user and repo it finds. Usage counters are updated on every write and delete.

How limits interact

Both user and repo limits can apply simultaneously. A write is allowed only if it passes all active checks:

  1. Disk reserve check (--disk-reserve)
  2. Global size limit (--max-size)
  3. Per-user quota (user:<name> or default:user)
  4. Per-repo quota (repo:<path> or default:repo)

If a user alice has a 50 GiB total limit and a specific repo alice/laptop has a 20 GiB limit, a write to that repo will be rejected as soon as either threshold is exceeded.

All rejected writes return HTTP 507 Insufficient Storage.

Hot-reloading

Send SIGHUP to the rest-server process to reload the quota file without restarting:

kill -HUP $(pidof rest-server)

Limits are updated immediately; usage counters are preserved across reloads.

Prometheus metrics

When --prometheus is enabled, four additional gauges are exported per user and per repo:

Metric Labels Description
rest_server_user_quota_used_bytes user Current space used in bytes
rest_server_user_quota_limit_bytes user Configured limit in bytes (0 = no limit)
rest_server_repo_quota_used_bytes repo Current space used in bytes
rest_server_repo_quota_limit_bytes repo Configured limit in bytes (0 = no limit)

Why use Rest Server?

Compared to the SFTP backend, the REST backend has better performance, especially so if you can skip additional crypto overhead by using plain HTTP transport (restic already properly encrypts all data it sends, so using HTTPS is mostly about authentication).

But, even if you use HTTPS transport, the REST protocol should be faster and more scalable, due to some inefficiencies of the SFTP protocol (everything needs to be transferred in chunks of 32 KiB at most, each packet needs to be acknowledged by the server).

One important safety feature that Rest Server adds is the optional ability to run in append-only mode. This prevents an attacker from wiping your server backups when access is gained to the server being backed up.

Finally, the Rest Server implementation is really simple and as such could be used on the low-end devices, no problem. Also, in some cases, for example behind corporate firewalls, HTTP/S might be the only protocol allowed. Here too REST backend might be the perfect option for your backup needs.

Contributors

Contributors are welcome, just open a new issue / pull request.

About

Rest Server is a high performance HTTP server that implements restic's REST backend API.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages

  • Go 98.4%
  • Other 1.6%