-
Notifications
You must be signed in to change notification settings - Fork 2
Add troubleshooting and misc to local dev docs #29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
I have a few notes that may be helpful for troubleshooting the application. I also included some details for guidance on getting and preparing maxmind license key credentials Signed-off-by: Matt Johnson <m@ttjohnson.com>
WalkthroughA new "Troubleshooting and Misc" section was added to the Changes
Possibly related PRs
Suggested reviewers
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
🧹 Nitpick comments (5)
docs/local_dev.md (5)
64-64: Fix typo & punctuation in dry-run explanationSpelling (
recipie→recipe) and a missing comma break the flow.-From the shell, to see what command a just recipie would call use the `-n` flag with the `just` command. +From the shell, to see what command a `just` recipe would call, use the `-n` flag with the `just` command.
78-87: One-liner is brittle – consider a helper script/just-recipeThe nested-container
docker … runc exec …command is impressive but hard to type, depends onjq, and will break if the output formats change.
Suggest adding a small helper (just shell-into-nested) or a bash wrapper script so users copy/run a stable alias instead of re-pasting this wall of shell.
111-111: Vendor name is “MaxMind”, not “Max Mind”-## Max Mind GeoIP Database for Log Enrichment +## MaxMind GeoIP Database for Log Enrichment
113-118: Tighten wording & capitalisation in MaxMind intro-If you want to work with the geoip data used by vector to enrich log data, you can get a license key from maxmind in order to download the geoip database files. +If you want to work with the GeoIP data Vector uses for log enrichment, obtain a license key from MaxMind to download the GeoIP database files.Also convert the bare URLs on lines 115-116 to Markdown links to satisfy MD034:
-https://dev.maxmind.com/geoip/geolite2-free-geolocation-data/#sign-up-for-a-maxmind-account-to-get-geolite -https://www.maxmind.com/en/geolite-free-ip-geolocation-data +[MaxMind GeoLite2 sign-up](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data/#sign-up-for-a-maxmind-account-to-get-geolite) +[MaxMind GeoLite Free IP Geolocation Data](https://www.maxmind.com/en/geolite-free-ip-geolocation-data)
138-141: Hard tabs trigger MD010 – silence or convertMarkdown-lint flags tabs inside the Go snippet. Either:
- Replace leading tabs with four spaces, or
- Precede the snippet with
<!-- markdownlint-disable MD010 -->to keep code fidelity while silencing the warning.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/local_dev.md(2 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/local_dev.md
[grammar] ~64-~64: Ensure spelling is correct
Context: ...m the shell, to see what command a just recipie would call use the -n flag with the `...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
[style] ~113-~113: Consider using an alternative to strengthen your wording.
Context: ...oIP Database for Log Enrichment If you want to work with the geoip data used by vec...
(WANT_KEEN)
[style] ~113-~113: Consider a more concise word here.
Context: ... you can get a license key from maxmind in order to download the geoip database files. htt...
(IN_ORDER_TO_PREMIUM)
🪛 Gitleaks (8.27.2)
docs/local_dev.md
149-149: Discovered a potential basic authorization token provided in a curl command, which could compromise the curl accessed resource.
(curl-auth-user)
🪛 markdownlint-cli2 (0.17.2)
docs/local_dev.md
115-115: Bare URL used
(MD034, no-bare-urls)
116-116: Bare URL used
(MD034, no-bare-urls)
138-138: Hard tabs
Column: 1
(MD010, no-hard-tabs)
139-139: Hard tabs
Column: 1
(MD010, no-hard-tabs)
140-140: Hard tabs
Column: 1
(MD010, no-hard-tabs)
144-144: Bare URL used
(MD034, no-bare-urls)
154-154: Spaces inside code span elements
(MD038, no-space-in-code)
and prevent dumb credential scanners from tripping needlessly over example docs with demonstrated placeholder values Signed-off-by: Matt Johnson <m@ttjohnson.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (4)
docs/local_dev.md (4)
64-65: Fix typo in command description
recipie→recipe-From the shell, to see what command a just recipie would call use the `-n` flag with the `just` command. +From the shell, to see what command a just recipe would call, use the `-n` flag with the `just` command.
113-118: Consistently capitalise “MaxMind” and convert bare URLs to Markdown linksThis section mixes the spellings “Max Mind”, “maxmind”, and uses bare URLs that violate MD034. A quick tidy-up avoids distraction and linter noise.
-If you want to work with the geoip data used by vector to enrich log data, you can get a license key from maxmind in order to download the geoip database files. - -https://dev.maxmind.com/geoip/geolite2-free-geolocation-data/#sign-up-for-a-maxmind-account-to-get-geolite -https://www.maxmind.com/en/geolite-free-ip-geolocation-data -My Account -> Manage License Keys +If you want to work with the GeoIP data used by Vector to enrich log data, obtain a licence key from **MaxMind** to download the GeoLite2 database files. + +* <https://dev.maxmind.com/geoip/geolite2-free-geolocation-data/#sign-up-for-a-maxmind-account-to-get-geolite> +* <https://www.maxmind.com/en/geolite-free-ip-geolocation-data> + +MaxMind dashboard → **My Account → Manage License Keys**
149-152: Re-format thecurlexample to silence secret-scannersGitleaks still flags the
-u user:passpattern even with placeholders. Passing credentials via environment variables avoids the false positive and mirrors best practice.-curl -v -u YOUR_ACCOUNT_ID:YOUR_LICENSE_KEY https://download.maxmind.com/ 2>&1 | grep Authorization -# > Authorization: Basic <BASE64_ENCODED_AUTH> -echo "<BASE64_ENCODED_AUTH>" | base64 -d +# ACCOUNT_ID and LICENSE_KEY can be exported or sourced from your shell profile. +curl -v -u "${ACCOUNT_ID}:${LICENSE_KEY}" https://download.maxmind.com/ 2>&1 | grep Authorization +# > Authorization: Basic <BASE64_ENCODED_AUTH> +echo "<BASE64_ENCODED_AUTH>" | base64 -d
138-141: Replace hard tabs inside fenced block with spaces to satisfy MD010Although it’s a code snippet, replacing the literal tab characters with spaces keeps markdown-lint happy and doesn’t alter the Go example.
- geoLite2CityArchive := dag.HTTP("https://download.maxmind.com/geoip/databases/GeoLite2-City/download?suffix=tar.gz", dagger.HTTPOpts{ - AuthHeader: maxMindAuth, - }) + geoLite2CityArchive := dag.HTTP("https://download.maxmind.com/geoip/databases/GeoLite2-City/download?suffix=tar.gz", dagger.HTTPOpts{ + AuthHeader: maxMindAuth, + })
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/local_dev.md(2 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/local_dev.md
[grammar] ~64-~64: Ensure spelling is correct
Context: ...m the shell, to see what command a just recipie would call use the -n flag with the `...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
[style] ~113-~113: Consider using an alternative to strengthen your wording.
Context: ...oIP Database for Log Enrichment If you want to work with the geoip data used by vec...
(WANT_KEEN)
[style] ~113-~113: Consider a more concise word here.
Context: ... you can get a license key from maxmind in order to download the geoip database files. htt...
(IN_ORDER_TO_PREMIUM)
🪛 Gitleaks (8.27.2)
docs/local_dev.md
149-149: Discovered a potential basic authorization token provided in a curl command, which could compromise the curl accessed resource.
(curl-auth-user)
🪛 markdownlint-cli2 (0.17.2)
docs/local_dev.md
115-115: Bare URL used
(MD034, no-bare-urls)
116-116: Bare URL used
(MD034, no-bare-urls)
138-138: Hard tabs
Column: 1
(MD010, no-hard-tabs)
139-139: Hard tabs
Column: 1
(MD010, no-hard-tabs)
140-140: Hard tabs
Column: 1
(MD010, no-hard-tabs)
144-144: Bare URL used
(MD034, no-bare-urls)
154-154: Spaces inside code span elements
(MD038, no-space-in-code)
| The raw authorization header for basic authentication is a base64 encoded value that contains the username separated by a `:` and then the password. | ||
|
|
||
| ```bash | ||
| curl -v -u YOUR_ACCOUNT_ID:YOUR_LICENSE_KEY https://download.maxmind.com/ 2>&1 | grep Authorization |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This part bugged me a little bit, trying to demonstrate what the raw header ends up looking like with curl because curl has to establish a valid connection to a server in order to show the request headers it generates, and I'm not sure I like using a real public web server to demonstrate this with.
I messed with curl a bit more and figured out how to get it to produce what I wanted without making a remote request, but it involves a few more commands and is a bit more convoluted. A netcat process opens and listens on port 9999, then a curl command is executed in the background after waiting a second for netcat to get a port open, and netcat will timeout after 2 seconds.
( exec 2>/dev/null; ( sleep 1 && curl -s -u YOUR_ACCOUNT_ID:YOUR_LICENSE_KEY http://127.0.0.1:9999 ) & timeout 2 nc -l 127.0.0.1 9999 | grep Authorization )While this does what I was initially looking for, and avoids sending requests to some poor web server just to demonstrate how the header is constructed, I don't know that I like all the hoops that I have to jump through to get the same kind of results without connecting to a remote server. It's also a lot less clear what's going on in order to have it all happen on the localhost, so maybe leaving the remote server (download.maxmind.com) in there is better for the documentation example?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Keeping it simple seems best to me 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've read this section a few times and I don't understand how it works.
If you want to work with the geoip data used by vector to enrich log data, you can get a license key from maxmind…
That makes complete sense. 👍🏼
When some
justcommands call dagger they may pass in an option with the location of maxmind auth credentials to be retrieved from a 1Password vault
Conditional on what? What sorts of commands? Can they emit a warning that the credential is missing, and thus logging is degraded?
There is a section of the dagger code that downloads the maxmind database using a license key.
Great 👍🏼
The raw authorization header for basic authentication is a base64 encoded value that contains the username separated by a
:and then the password.
OK. An example might help.
curl -v -u YOUR_ACCOUNT_ID:YOUR_LICENSE_KEY https://download.maxmind.com/ 2>&1 | grep Authorization
Is this how to retrieve it? What's the difference between YOUR_ACCOUNT_ID:YOUR_LICENSE_KEY and the Authorization header? Or are you just using curl -u to convert the plaintext to base64? If so, there's surely a more straightforward way to do that.
Either way, this seems like it belongs inside the Dagger code.
Signed-off-by: Gerhard Lazu <gerhard@changelog.com>
9943a36 to
3d0be9a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 🚀
| ```bash | ||
| # Exec into nested container (broken down) | ||
| docker_container_name="$(docker ps --format json | jq --slurp -r '[.[] | select((.Command | contains("dagger")) and (.Image | contains("dagger")) and (.Names | contains("dagger")))][0].Names')" | ||
| nested_containers="$(docker exec "${docker_container_name}" runc list -f json)" | ||
| nested_container_id="$(echo -E "${nested_containers}" | jq -r '[.[] | select(.status=="running" and (.bundle | contains("dagger/worker/executor")))][0].id')" | ||
| docker exec -it "${docker_container_name}" runc exec -t "${nested_container_id}" bash | ||
|
|
||
| # Crazy One-Liner to shell into nested container | ||
| docker exec -it "$(docker ps --format json | jq --slurp -r '[.[] | select((.Command | contains("dagger")) and (.Image | contains("dagger")) and (.Names | contains("dagger")))][0].Names')" runc exec -t "$(echo -E "$(docker exec "$(docker ps --format json | jq --slurp -r '[.[] | select((.Command | contains("dagger")) and (.Image | contains("dagger")) and (.Names | contains("dagger")))][0].Names')" runc list -f json)" | jq -r '[.[] | select(.status=="running" and (.bundle | contains("dagger/worker/executor")))][0].id')" bash | ||
| ``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This feels like it deserves to be a script somewhere: maybe in the justfile.
| # to dynamically resolve the domain for the backends. If the varnish config has an | ||
| # acl for only allowing IPv6 or IPv4 addresses, you will see errors when it gets a | ||
| # response from the dns query that is not part of the acl. | ||
| varnishlog -g raw -q '* ~ vmod-dynamic' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you expect this to be a pretty common incantation, why not put it in the containerized justfile?
| The raw authorization header for basic authentication is a base64 encoded value that contains the username separated by a `:` and then the password. | ||
|
|
||
| ```bash | ||
| curl -v -u YOUR_ACCOUNT_ID:YOUR_LICENSE_KEY https://download.maxmind.com/ 2>&1 | grep Authorization |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've read this section a few times and I don't understand how it works.
If you want to work with the geoip data used by vector to enrich log data, you can get a license key from maxmind…
That makes complete sense. 👍🏼
When some
justcommands call dagger they may pass in an option with the location of maxmind auth credentials to be retrieved from a 1Password vault
Conditional on what? What sorts of commands? Can they emit a warning that the credential is missing, and thus logging is degraded?
There is a section of the dagger code that downloads the maxmind database using a license key.
Great 👍🏼
The raw authorization header for basic authentication is a base64 encoded value that contains the username separated by a
:and then the password.
OK. An example might help.
curl -v -u YOUR_ACCOUNT_ID:YOUR_LICENSE_KEY https://download.maxmind.com/ 2>&1 | grep Authorization
Is this how to retrieve it? What's the difference between YOUR_ACCOUNT_ID:YOUR_LICENSE_KEY and the Authorization header? Or are you just using curl -u to convert the plaintext to base64? If so, there's surely a more straightforward way to do that.
Either way, this seems like it belongs inside the Dagger code.
I have a few notes that may be helpful for troubleshooting the application. I also included some details for guidance on getting and preparing maxmind license key credentials
Summary by CodeRabbit