Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Format json with JQ_COLORS #14

Merged
merged 15 commits into from
Feb 8, 2024
Merged

Format json with JQ_COLORS #14

merged 15 commits into from
Feb 8, 2024

Conversation

thaliaarchi
Copy link
Collaborator

This PR adds support for colors via JQ_COLORS and enabling/disabling with NO_COLORS, --color-output/-C, and --monochrome-output/-M. The formatting matches jq (not gojq).

I've tested this and it has identical behavior to jq. If we want to add tests, it would need a new harness, because --run-tests uses value-based equality, not formatted textual equality.

I also removed the tojson/1 intrinsic, because neither jq nor gojq define it.

If jqlang/jq#3034 is merged, I'll update _tojson here to match.

@thaliaarchi
Copy link
Collaborator Author

thaliaarchi commented Feb 7, 2024

Commit 231277f moves _tojson without changing it, after the previous commits modify it. If you're reviewing with a diff viewer that doesn't detect text moves (like GitHub), then you can review it easier in three chunks: before (master...808b031), move (231277f), and after (231277f...d7cea3c)

jqjq.jq Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
jqjq.jq Show resolved Hide resolved
jqjq.jq Show resolved Hide resolved
jqjq.jq Show resolved Hide resolved
jqjq.jq Show resolved Hide resolved
jqjq.jq Outdated
end;
( [_f]
| add
);

# get the ANSI color codes for printing values
# corresponds to jv_set_colors and its usage in main
def parse_colors($opts; $env):
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fq lacks JQ_COLORS support atm (has it's own color config with more stuff) maybe can steal some of this code :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're welcome to use it for that :). It should represent the C parsing and formatting exactly.

From my tests, it appears the gojq ignores JQ_COLORS and it formats differently. Maybe it would be worth coordinating a decision with gojq? I figure you probably inherited that from using gojq as a library.

I submitted jqlang/jq#3034, which fixes some usually-invisible inconsistencies with colors in jq. I think that should be merged regardless, but in addition it might worth bringing jq more into line with gojq. In particular, I don't really like how in jq, {, :, ,, and } in objects get the object color and [, ,, and ] in arrays get the array color. I think : and , punctuation should be colored uniformly across objects and arrays. Personally, I don't see much use in coloring braces and brackets, but many people like coloring them by nesting depth in VS Code, so I can see why this simplistic differentiation could be somewhat useful.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're welcome to use it for that :). It should represent the C parsing and formatting exactly.

👍

From my tests, it appears the gojq ignores JQ_COLORS and it formats differently. Maybe it would be worth coordinating a decision with gojq? I figure you probably inherited that from using gojq as a library.

Kind-of, fq uses a modified version of cli/encoder.go from gojq https://github.com/wader/fq/blob/master/internal/colorjson/encoder.go that makes the options non-global and adds some things needed by fq (how to format binary strings etc)

I noticed now that gojq seems to only read GOJQ_COLORS, was thinking if i added support in fq i would probably read FQ_COLORS and fallback to JQ_COLORS? hmm

I submitted jqlang/jq#3034, which fixes some usually-invisible inconsistencies with colors in jq. I think that should be merged regardless, but in addition it might worth bringing jq more into line with gojq. In particular, I don't really like how in jq, {, :, ,, and } in objects get the object color and [, ,, and ] in arrays get the array color. I think : and , punctuation should be colored uniformly across objects and arrays. Personally, I don't see much use in coloring braces and brackets, but many people like coloring them by nesting depth in VS Code, so I can see why this simplistic differentiation could be somewhat useful.

I see, hmm haven't really thought about it much so no string opinion, but making them more consistent make sense i think. Let's see what the other maintainers think

@wader
Copy link
Owner

wader commented Feb 7, 2024

Commit 231277f moves _tojson without changing it, after the previous commits modify it. If you're reviewing with a diff viewer that doesn't detect text moves (like GitHub), then you can review it easier in three chunks: before (master...808b031), move (231277f), and after (231277f...d7cea3c)

👍

@thaliaarchi
Copy link
Collaborator Author

Thanks for the review! I'll go through your comments in the morning.

Shortly after your review, I implemented the raw output options, which I've now pushed. They touch the _tojson filter after eval, so are fairly related to this PR. I'd prefer to have it reviewed here, but I could split it from this PR and rebase it onto master once this lands, if you'd like.

This reduces the missing features needed for wsjq even more!

@wader
Copy link
Owner

wader commented Feb 7, 2024

Thanks for the review! I'll go through your comments in the morning.

No stress!

Shortly after your review, I implemented the raw output options, which I've now pushed. They touch the _tojson filter after eval, so are fairly related to this PR. I'd prefer to have it reviewed here, but I could split it from this PR and rebase it onto master once this lands, if you'd like.

Either way works for me so let's keep it here

This reduces the missing features needed for wsjq even more!

🥳

jqjq.jq Show resolved Hide resolved
jqjq.jq Show resolved Hide resolved
jqjq.jq Show resolved Hide resolved
Updates to match the changes in jqlang/jq#3034.
@thaliaarchi
Copy link
Collaborator Author

Since jqlang/jq#3034 looks like it'll be merged, I've pushed a commit to match it.

- [x] Run jqjq with jqjq
- [x] Bugs

### jq's test suite

```
```sh
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to rerun this, still 307, but i did notice that something seems strange with the output when running tests. Have a look at make test guessing it's the switch to --join-output that require some change?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just changed ./jqjq to replace --join-output with --raw-output for --run-tests.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get 308 on jq.test from jq origin/main.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh made sure i had master also now, still 307, weird, some locale thing? anyways, that's for another time

@wader
Copy link
Owner

wader commented Feb 7, 2024

BTW talking about ANSI output, maybe https://github.com/wader/ansisvg is interesting? think my main use of it has been producing colorful output from jq and fq for presentations :)

@wader
Copy link
Owner

wader commented Feb 8, 2024

So ready to merge?

@thaliaarchi
Copy link
Collaborator Author

Yep!

@wader wader merged commit ebbce4c into wader:master Feb 8, 2024
@wader
Copy link
Owner

wader commented Feb 8, 2024

Great work! and so happy someone else wants to help out with this weird thing :)

@wader
Copy link
Owner

wader commented Feb 8, 2024

Hmm noticed --help looks weird, maybe the last minute fix for test output messed up? wonder if there is a neat we to have output tests

@thaliaarchi
Copy link
Collaborator Author

Your ansisvg project looks useful. I've used asciinema for that kind of thing before, but this would be usable outside the browser when it's not animated.

It reminds me of a tiny project I did, terminal-palette. It computes the nearest color in a palette for RGB colors, by using the perceptual distance between colors in the CIELAB color space (logic borrowed from james-oldfield/nearest-colour). I used it to make fallback color schemes for my Tetris game, programmed in Whitespace. It uses the official piece colors when the terminal supports 24-bit color, or uses similar colors from the 256 colors if that's supported, or otherwise uses the basic 16 colors.

@wader
Copy link
Owner

wader commented Feb 8, 2024

Neat! think i will have to have a closer look at the whitespace language

@wader
Copy link
Owner

wader commented Feb 8, 2024

Getter a bit late here, feel tree to merge a fix for help if you look into it. Otherwise will fix or review tomorrow

@thaliaarchi
Copy link
Collaborator Author

Good catch for --help. I had picked --join-output as the unconditional output format (except for tests), because it gives the most flexibility to jqjq. jqjq can insert LF or NUL as appropriate, according to its own --raw-output/--raw-output0/--join-output options. That means, though, that it needs to be more conscious of how everything is formatted.

In this case, the fix is to add \n to each line of help. I'd go further to even concatenate it into one string. I have a fix. Can I just push it to master?

@wader
Copy link
Owner

wader commented Feb 8, 2024

Nice, yeap push is fine too!

@thaliaarchi thaliaarchi mentioned this pull request Mar 8, 2024
@thaliaarchi thaliaarchi deleted the colors branch March 8, 2024 11:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants