-
-
Notifications
You must be signed in to change notification settings - Fork 27
External Secrets
External secrets are secrets that are stored in an external secret management service and fetched during a deployment by Doco-CD. This allows you to keep your secrets out of your version control system and manage them in a secure way.
Additional external secret providers may be supported in the future. If you have a specific provider in mind, please open an issue or submit a pull request.
To use an external secret provider, you need to configure the environment variables for the provider you want to use.
To use AWS Secrets Manager, you need to set the following environment variables:
Create an access token via IAM, see https://repost.aws/knowledge-center/create-access-key
| Key | Value |
|---|---|
SECRET_PROVIDER |
aws_sm |
SECRET_PROVIDER_REGION |
AWS Region to use, e.g. eu-west-1
|
SECRET_PROVIDER_ACCESS_KEY_ID |
Access key ID of an IAM user with access to AWS Secrets Manager |
SECRET_PROVIDER_SECRET_ACCESS_KEY |
Secret access key of an IAM user with access to AWS Secrets Manager |
SECRET_PROVIDER_SECRET_ACCESS_KEY_FILE |
Path to the file containing the secret access token inside the container |
Deployment configuration
Add a mapping/reference between the environment variable you want to set in the docker compose project/stack and the ARN of the secret in AWS Secrets Manager.
Secrets can be retrieved in two ways:
- As clear text/plain string:
arn:aws:secretsmanager:region:account-id:secret:secret-name - With a path to the value (Used if the secret contains a JSON):
arn:aws:secretsmanager:region:account-id:secret:secret-name/item
For example in your account 1234567890, the secret myapp in the region eu-west-1 contains this JSON value: {"username":"foo","password":"bar"}
If you want to get the secret value of the password field, use the ARN in addition with a slash (/) and the field name/key as the path: arn:aws:secretsmanager:eu-west-1:1234567890:secret:myapp/password.
Note
Without specifying a path, the entire JSON gets returned as a single string (see example below).
For example in your .doco-cd.yml:
external_secrets:
JSON_STRING: "arn:aws:secretsmanager:eu-west-1:1234567890:secret:myapp" # '{"username":"foo","password":"bar"}'
APP_PASSWORD: "arn:aws:secretsmanager:eu-west-1:1234567890:secret:myapp/password" # barTo use Bitwarden Secrets Manager, you need to set the following environment variables:
| Key | Value | Default |
|---|---|---|
SECRET_PROVIDER |
bitwarden_sm |
|
SECRET_PROVIDER_API_URL |
US: https://vault.bitwarden.com/api EU: https://vault.bitwarden.eu/api
|
https://vault.bitwarden.com/api |
SECRET_PROVIDER_IDENTITY_URL |
US: https://vault.bitwarden.com/identity EU: https://vault.bitwarden.eu/identity
|
https://vault.bitwarden.com/identity |
SECRET_PROVIDER_ACCESS_TOKEN |
Access token of a machine account, see the docs for machine accounts and access-tokens | |
SECRET_PROVIDER_ACCESS_TOKEN_FILE |
Path to the file containing the access token inside the container |
Deployment configuration
Add a mapping/reference between the environment variable you want to set in the docker compose project/stack and the ID of the secret in Bitwarden Secrets Manager.
For example in your .doco-cd.yml:
name: myapp
external_secrets:
DB_PASSWORD: 138e3a97-ed58-431c-b366-b35500663411To use 1Password, you need to set the following environment variables:
| Key | Value |
|---|---|
SECRET_PROVIDER |
1password |
SECRET_PROVIDER_ACCESS_TOKEN |
Access token of a service account, see the docs and here |
SECRET_PROVIDER_ACCESS_TOKEN_FILE |
Path to the file containing the access token inside the container |
Note
The start time and memory usage of the doco-cd container, as well as the runtime of a job, can increase significantly when using this secret provider.
Deployment configuration
Add a mapping/reference between the environment variable you want to set in the docker compose project/stack and the URI to the secret in 1Password.
See their docs for the correct syntax and how to get a secret reference of your secret: https://developer.1password.com/docs/cli/secret-reference-syntax/
A valid secret reference should use the syntax:
op://<vault>/<item>/[section/]<field>
To get a one-time password, append the ?attribute=otp query parameter to a secret reference that points to a one-time password field in 1Password:
op://<vault>/<item>/[section/]one-time password?attribute=otp
Important
Machine accounts can only access vaults for which you have granted read permissions during creation. The default Personal vault can't be access by machine accounts!
For example in your .doco-cd.yml:
name: myapp
external_secrets:
DB_PASSWORD: "op://vault/item/field"To use Infisical, you need to set the following environment variables:
| Key | Value |
|---|---|
SECRET_PROVIDER |
infisical |
SECRET_PROVIDER_SITE_URL |
The URL of the Infisical site (e.g. https://app.infisical.com, https://eu.infisical.com or your self-hosted instance URL) |
SECRET_PROVIDER_CLIENT_ID |
The Client ID of a machine account, see the docs for machine accounts |
SECRET_PROVIDER_CLIENT_SECRET |
The Client Secret of a machine account (Universal Auth) |
SECRET_PROVIDER_CLIENT_SECRET_FILE |
Path to the file containing the client secret inside the container |
Deployment configuration
Add a mapping/reference between the environment variable you want to set in the docker compose project/stack and the reference to the secret in Infisical.
A valid secret reference should use the syntax:
projectId:env:[/some/path/]key
Important
Machine accounts can only access projects for which you have granted read permissions.
For example in your .doco-cd.yml:
name: myapp
external_secrets:
TEST_PASSWORD: 0db45926-c97c-40d4-a3aa-fefd5d5fb492:dev:DATABASE_URL
OTHER_PASSWORD: "0db45926-c97c-40d4-a3aa-fefd5d5fb492:dev:/Test/Sub/TEST_SECRET"
USERNAME: 0db45926-c97c-40d4-a3aa-fefd5d5fb492:dev:Test/Sub/TEST_SECRETTo use OpenBao, you need to set the following environment variables:
| Key | Value |
|---|---|
SECRET_PROVIDER |
openbao |
SECRET_PROVIDER_SITE_URL |
The URL of the OpenBao instance |
SECRET_PROVIDER_ACCESS_TOKEN |
Access token for authenticating with the secret provider |
SECRET_PROVIDER_ACCESS_TOKEN_FILE |
Path to a file containing the access token inside the container |
Deployment configuration
Add a mapping/reference between the environment variable you want to set in the docker compose project/stack and the reference to the key-value secret in OpenBao.
By default, the root namespace is used (root or /), but you can specify a different namespace by adding it as the first part of the reference.
- A valid key-value secret reference should use the syntax:
kv:<namespace(optional)>:<secretEngine>:<secretName>:<key> - A valid PKI certificate reference should use the syntax:
pki:<namespace(optional)>:<secretEngine>:<commonName>
Examples of valid references:
-
kv:prod-secrets:db-prod:username→ Fetches theusernamekey from thedb-prodkey-value secret in theprod-secretssecret engine in therootnamespace. -
kv:root:prod-secrets:db-prod:username→ Same as above, explicitly specifying therootnamespace. -
kv:my-namespace:secret:api-keys:stripe→ Fetches thestripekey from theapi-keyssecret in thesecretkey-value secret engine in themy-namespacenamespace. -
pki:certs:myapp.example.com→ Fetches the certificate for the common namemyapp.example.comfrom thecertspki secret engine in therootnamespace. -
pki:my-namespace:certs:myapp.example.com→ Fetches the certificate for the common namemyapp.example.comfrom thecertspki secret engine in themy-namespacenamespace.
For example in your .doco-cd.yml:
name: myapp
external_secrets:
DB_USERNAME: kv:secret:db-prod:username
DB_PASSWORD: kv:secret:db-prod:password
CERT: pki:pki:myapp.example.comTo use the certificate in your compose file, you can pass the value to a compose config:
# docker-compose.yml
configs:
myapp-example-com.crt:
#environment: CERT # Either pass the variable via the environment like this (without a $ sign)
content: $CERT # Or use the content field to directly inject the variable value to the config content
services:
app:
image: myapp:latest
environment:
DB_USERNAME: $DB_USERNAME
DB_PASSWORD: $DB_PASSWORD
configs:
- source: myapp-example-com.crt
target: /etc/ssl/certs/example.crtDoco-CD uses variable interpolation to replace variables in your Compose files with the values fetched from the external secret provider, see the Compose file reference for more information and examples.
For example with Bitwarden Secrets Manager, if you want to use secrets named DB_PASSWORD and LABEL_SECRET in your Compose file, you can reference it like this:
#.doco-cd.yml
name: myapp
external_secrets:
DB_PASSWORD: a8f1e4eb-d76d-47b4-aa3c-103733e77fce
LABEL_SECRET: cfd0c4a9-16d4-44c8-9a80-c6143a7c7b71Then you can use the variable in your Compose file like this:
Important
External secrets have a higher priority than variables set in a .env file or in the environment.
If a variable is set in both an external secret and in a .env file, the value from the external secret will be used.
#.env
DB_PASSWORD=testpassword # This will be overridden by the external secret
DOMAIN=example.com#docker-compose.yml
services:
app:
image: myapp:latest
environment:
DATABASE_HOST: db
DATABASE_USER: ${$DB_USER:-postgres} # You can also set a default value if the secret is missing
DATABASE_PASSWORD: $DB_PASSWORD
labels:
- "traefik.enable=true"
- "traefik.http.routers.myapp.rule=Host(`myapp.${DOMAIN}`)" # Note that DOMAIN is set in a local .env file and not fetched from the secret provider
db:
image: postgres:latest
environment:
POSTGRES_USER: ${$DB_USER:-postgres}
POSTGRES_PASSWORD: ${DB_PASSWORD}This will result in the following docker-compose configuration being used during deployment:
#docker-compose.yml
services:
app:
image: myapp:latest
environment:
DATABASE_HOST: db
DATABASE_USER: postgres
DATABASE_PASSWORD: supersecretpassword123 # Value of external secret fetched from Bitwarden Secrets Manager
labels:
- "traefik.enable=true"
- "traefik.http.routers.myapp.rule=Host(`myapp.example.com`)" # Value from .env file
db:
image: postgres:latest
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: supersecretpassword123 # Value of external secret fetched from Bitwarden Secrets Manager