Skip to content

fix(tempo): add sensible tempo limits#1160

Merged
glottologist merged 2 commits into
masterfrom
jason/limit_tempo
Feb 24, 2026
Merged

fix(tempo): add sensible tempo limits#1160
glottologist merged 2 commits into
masterfrom
jason/limit_tempo

Conversation

@glottologist

@glottologist glottologist commented Feb 24, 2026

Copy link
Copy Markdown
Contributor

Describe the changes
A clear and concise description of what the bug fix, feature, or improvement is.

Related Issue(s)
Please link to the issue(s) that will be closed with this PR.

Checklist

  • Tests have been added/updated for the changes.
  • Documentation has been updated for the changes (if applicable).
  • The code follows Rust's style guidelines.

Additional Context
Add any other context about the pull request here.

Summary by CodeRabbit

  • Chores
    • Updated Tempo ingester settings: reduced maximum block duration and introduced a max block bytes limit to control ingestion behavior.
    • Set Tempo service memory resource limit to 4 GB to constrain runtime memory allocation.

@coderabbitai

coderabbitai Bot commented Feb 24, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 249af35 and c44eb9b.

📒 Files selected for processing (1)
  • docker/observation/configs/tempo/tempo.yaml

📝 Walkthrough

Walkthrough

Adjusted Tempo observability configuration and Docker deployment: decreased ingester max_block_duration, added ingester max_block_bytes, and added a 4G memory limit to the Tempo service in docker-compose.

Changes

Cohort / File(s) Summary
Tempo Configuration
docker/observation/configs/tempo/tempo.yaml
Decreased ingester.max_block_duration from 5m to 2m and added ingester.max_block_bytes: 104857600.
Docker Compose Deployment
docker/observation/docker-compose.yaml
Added deploy.resources.limits.memory: 4G to the tempo service.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main changes: adding memory limits to Tempo and adjusting ingester configuration settings for optimization.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch jason/limit_tempo

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docker/observation/configs/tempo/tempo.yaml`:
- Line 60: The tempo processors list currently omits the local-blocks processor
which TraceQL metrics require; either re-add local-blocks to the processors
array in tempo.yaml (restore "local-blocks" alongside "service-graphs" and
"span-metrics") to keep TraceQL metrics working, or disable the traceqlEditor
toggle in the Grafana service's GF_FEATURE_TOGGLES_ENABLE environment value in
docker-compose.yaml (remove "traceqlEditor" from that comma-separated list) so
the editor is not exposed when local-blocks is not enabled.

In `@docker/observation/docker-compose.yaml`:
- Around line 57-60: The compose file currently sets a hard memory cap via
deploy.resources.limits.memory: 4G but should also declare CPU limit and a
memory reservation to provide a guaranteed floor; update the service's deploy
block (deploy.resources) to add limits.cpus (or cpus) and reservations.memory so
the scheduler has both a ceiling and a reservation — modify the same
deploy.resources.limits and add a deploy.resources.reservations.memory entry and
a deploy.resources.limits.cpus entry for the service to enforce CPU and
reservation alongside the 4G memory limit.

ℹ️ Review info

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8e430bf and 249af35.

📒 Files selected for processing (2)
  • docker/observation/configs/tempo/tempo.yaml
  • docker/observation/docker-compose.yaml

Comment thread docker/observation/configs/tempo/tempo.yaml Outdated
Comment thread docker/observation/docker-compose.yaml

@JesseTheRobot JesseTheRobot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@glottologist glottologist merged commit a0c22f6 into master Feb 24, 2026
18 checks passed
@glottologist glottologist deleted the jason/limit_tempo branch February 24, 2026 22:14
@coderabbitai coderabbitai Bot mentioned this pull request Feb 25, 2026
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants