-
Notifications
You must be signed in to change notification settings - Fork 77
Add SecretManagerProvider to Ops Agent #1953
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Won't this cause the test to fail when run on any other project?
Also, why is there not a $ at the start of the value?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When does the test run on other projects?
"Also, why is there not a $ at the start of the value?" -> fixed, ty.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When does the test run on other projects?
When it's run directly from a developer's workstation with go test
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The secrets are accessible as long as the project hosting the secrets grants the roles/secretmanager.secretAccessor to the test VM Service Account.
If the test is not running from within a VM, then the deverlopers @google.com email needs to be granted the iam role.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And if the developer does not have an @google.com account? Ops Agent is open source software.
047f2da to
47ce017
Compare
apps/oracledb.go
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should probably add a test for this case specifically.
apps/oracledb.go
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit. Wouldn't we want secretPassword to start with ${googlesecretmanager? (i.e. HasPrefix)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently, we are trying to prevent users from using the secret provider in OracleDB-related configuration entries.
This is because the password will be URI-escaped before being passed to the OTel collector, which results in the provider identifier being altered and consequently unable to map back to a secret entry in Google Secret Manager.
We will remove this check later when we implement the fix in the sqlquery receiver.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here's the upstream issue: open-telemetry/opentelemetry-collector-contrib#40117
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This isn't an upstream issue; an upstream user can and should just put an escaped string into the secret.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copied from my comment in the upstream issue:
I agree that the "ideal" solution is to have a sharable component that can be called to URL encode or escape sections of the configuration.
From a "where this belongs" perspective between the two existing components though, I think it's closer to the sql receiver than the secret manager. This escaping issue is not specific to the case of wanting to use the secret manager. It also applies to using env variable (in the example Braydon provided), or any other provider in the future.
The sql receiver also knows exactly what type of escaping it needs to do (in this case URL encoding).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on my previous comment, I don't think deciding whether to implement escaping in a new provider or sql receiver should be blocking for the Secret Manager integration.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also copied from the upstream issue:
This isn't a question of the "ideal" solution - the proposed solution here can not work. Once the string is already concatenated into a single URL, it's no longer possible to know which part of that string needs encoding applied to it.
Also, the exact same problem applies to environment variable substitution, not just googlesecretmanager, so if you want to report a configuration error until this is supported upstream, you should be checking for ${ and not just ${googlesecretmanager
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is a good point. This would also not work with env variable substitution. Maybe we do a regex match for something like${.*}
Let's also change the error message to say that this integration does not support confmap providers.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
fd4e37a to
80c6391
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can leave it for this PR, but in the future a PR like this that does two distinctly different things (upgrades otelopscol, adds new test cases for googlesecretmanager) should be done in two separate PRs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why was the \EOF added in every enable script here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding \ so that the provider identifier is not interpreted as a shell variable.
This does not explain why EOF has a \, I'm still confused about this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The change in this file (adding \ in front of EOF) has been reverted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not a fan of the testing strategy in this PR because:
- It overrides the plaintext password testcases reducing our coverage
- It buries
googlesecretmanagertests in random database app tests. It should really have its own dedicated test cases in the main test framework.
I think this should be redone in a way that addresses both of those points.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with this, and I think it should be replaced with an integration test that also creates the secrets in the current project, so the test is not hardcoded to a single project.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah that would make more sense given how other tests in this repo work, given that none of the others hardcode a project.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
…vider identifiers
…he datasource field and becomes irrecognizable
2a11942 to
5888926
Compare
c8b25c5 to
a4ed0da
Compare
Description
Related issue
b/417214181
How has this been tested?
integration tests
Checklist: