Description

The changes presented implement a new binary in the folder cmd/google_cloud_ops_agent_diagnostics that will run after the google_cloud_ops_agent_engine binary successfully generates and validates the required configurations for fluent-bit and otelopscol. This binary will run as a service, in Windows and Linux, along side the execution of fluent-bit and otelopscol.

The purpose of the google-cloud-ops-agent-diagnostics service is to have a place to run diagnostics checks and actions during the execution of the Ops Agent. A non exhaustive list of this are :

• Report Ops Agent Self Metrics related to the currently set configuration :
  • agent.googleapis.com/agent/ops_agent/enabled_receivers (Implemented in this PR)
  • agent.googleapis.com/agent/ops_agent/feature_tracking (Future Work)
• Log Rotation for Fluent-bit Logs (Possible use case)
• Report Ops Agent Health Metrics (Possible use case)

Related issue

b/232815588 | b/245344014

How has this been tested?

• Integration Tests
  • Implemented a new test TestDiagnosticsCrashRestart to verify the diagnostics service restarts after manually stopping (pkill) the binary.
  • Added agent.googleapis.com/agent/ops_agent/enabled_receivers to the expected metrics verified by the TestDefaultMetricsNoProxy test.
  • In a following PR we can add the diagnosticsLivenessChecker to the opsAgentLivenessChecker. It can't be done in the same PR since it's used in TestUpgradeOpsAgent which compares to a previous version that doesn't have the diagnostics service.
• Unit Tests
  • Added TestEnabledReceiversDefaultConfig unit test to verify the count of receivers in the default configurations is what we expect.
• Manual Verification
  • Verified the correct functionality of the Diagnostics Service in a Windows VM ( windows-2019 ) and a Linux VM ( debian 10 ). The metrics were sent to the Monitoring API correctly and the service executes correctly.

Windows 2019

Debian 10

Checklist:

• Unit tests
  • Unit tests do not apply.
  • Unit tests have been added/modified and passed for this PR.
• Integration tests
  • Integration tests do not apply.
  • Integration tests have been added/modified and passed for this PR.
• Documentation
  • This PR introduces no user visible changes.
  • This PR introduces user visible changes and the corresponding documentation change has been made.
• Minor version bump
  • This PR introduces no new features.
  • This PR introduces new features, and there is a separate PR to bump the minor version since the last release already.
  • This PR bumps the version.