Skip to content

Script to compare prom output with m3 - #857

Merged
nikunjgit merged 3 commits into
masterfrom
compareProm
Aug 27, 2018
Merged

Script to compare prom output with m3#857
nikunjgit merged 3 commits into
masterfrom
compareProm

Conversation

@nikunjgit

Copy link
Copy Markdown
Contributor

No description provided.

@@ -0,0 +1,15 @@
#!/bin/bash

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: maybe add a check at the beginning to make sure m3db/prom are running?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also rename to curl_diff.sh or curl-diff.sh

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a check for running and renamed to prom-m3-diff.sh

@@ -0,0 +1,15 @@
#!/bin/bash
# This script can be used to compare prometheus query output with m3query output

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps making this into a function so that it could be sourced into other scripts in the future. Just a thought

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a good idea but I'm not sure how useful it would be to call it as a function vs calling this script completely and then working off generated files. Also, note that this is a simple script which just generates output from prom and m3. We'll be working on a more complex way to compare prom output with m3 output.

start=${2?"Missing start, usage: $0 target start"}
duration=${3:1000}
end=$((start + duration))
m3command="localhost:7201/api/v1/prom/native/read?start=$start&end=$end&step=15s&debug=true --data-urlencode target=$target"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps let users specify step size? and then have the default as 15s

promcommand="localhost:9090/api/v1/query_range?start=$start&end=$end&step=15s --data-urlencode query=$target"
echo $m3command
echo $promcommand
curl -G $m3command > m3out

@benraskin92 benraskin92 Aug 24, 2018

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: extra space after -G

@codecov

codecov Bot commented Aug 24, 2018

Copy link
Copy Markdown

Codecov Report

Merging #857 into master will decrease coverage by 0.13%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #857      +/-   ##
==========================================
- Coverage    78.4%   78.26%   -0.14%     
==========================================
  Files         384      384              
  Lines       33060    33060              
==========================================
- Hits        25920    25876      -44     
- Misses       5363     5403      +40     
- Partials     1777     1781       +4
Flag Coverage Δ
#dbnode 81.37% <ø> (-0.15%) ⬇️
#m3ninx 71.99% <ø> (ø) ⬆️
#query 66.54% <ø> (-0.14%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eca65a6...7b02f6f. Read the comment docs.

start=${2?"Missing start, usage: $0 target start"}
duration=${3:1000}
end=$((start + duration))
m3command="localhost:7201/api/v1/prom/native/read?start=$start&end=$end&step=15s&debug=true --data-urlencode target=$target"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Take m3 endpoint and prom endpoint as parameters

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't want to make the script too complicated. We can definitely add that when needed

curl -G $m3command > m3out
curl -G $promcommand > promout
jq ".[]|.tags,.datapoints" m3out > m3result
jq ".data.result|.[]|.metric,.values" promout > promresult

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably don't want to overwrite existing results you have here

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually I like overwritting the results. This keeps the workflow simple where I don't have to constantly deleted created files. Also, the query should be idempotent.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's more to deal with different but similar queries, e.g. if you want to compare two results for the same query but different start/end to see where the points actually line up. Take a param to see if you should overwrite?

@arnikola arnikola left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a little useful but it doesn't actually run any comparisons on the outputs

@nikunjgit

Copy link
Copy Markdown
Contributor Author

The main purpose of this script is to create an easy way to quickly run a query through both prom and m3. It doesn't intend to be a way for comparisons but rather for manual inspection.

@codecov

codecov Bot commented Aug 27, 2018

Copy link
Copy Markdown

Codecov Report

Merging #857 into master will decrease coverage by 0.18%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #857      +/-   ##
==========================================
- Coverage    78.4%   78.21%   -0.19%     
==========================================
  Files         384      384              
  Lines       33060    33060              
==========================================
- Hits        25920    25858      -62     
- Misses       5363     5406      +43     
- Partials     1777     1796      +19
Flag Coverage Δ
#dbnode 81.28% <ø> (-0.24%) ⬇️
#m3ninx 71.99% <ø> (ø) ⬆️
#query 66.62% <ø> (-0.06%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eca65a6...83db426. Read the comment docs.

@benraskin92 benraskin92 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@arnikola

Copy link
Copy Markdown
Collaborator

Maybe printout file size post comparison to have a quick indicator to see if they're equal? Obviously this won't be perfect, but could be a good canary

@nikunjgit

Copy link
Copy Markdown
Contributor Author

Added file size

@nikunjgit
nikunjgit merged commit 21e8012 into master Aug 27, 2018
@nikunjgit
nikunjgit deleted the compareProm branch August 30, 2018 17:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants