feat(parser): support regex matching on JSON arrays with scalar elements by ognis1205 · Pull Request #1163 · orhun/git-cliff

ognis1205 · 2025-06-04T00:54:42Z

Description

This PR extends the commit parser to support regex matching on JSON arrays, where each element is a scalar value (e.g., strings, numbers, booleans). When a commit field (like remote.pr_labels) is a JSON array, all entries are evaluated according to the specified match_mode (any or all). Fixes #1157

Motivation and Context

Previously, only scalar JSON values (e.g., strings, numbers) were matched against regex patterns. This change extends support to JSON arrays, where each element is a scalar value. All values in the array are checked based on the configured match mode (any/all).

How Has This Been Tested?

Clone the mta-solutions/fsharp-data-validation GitHub Repo.
Create a cliff.toml file in the root of the repo with the following contents:

[remote.github]
owner = "mta-solutions"
repo = "fsharp-data-validation"

[changelog]
output = "CHANGELOG.md"

header = """
# Changelog

"""

body = """
{% if version %}\
    ## [{{ version | trim_start_matches(pat="v") }}](https://github.com/{{ remote.github.owner }}/{{ remote.github.repo }}/tree/{{ version }}) ({{ timestamp | date(format="%Y-%m-%d") }})
{% else %}\
    {% if commits | length > 0 %}\
        \n## [unreleased]
    {% endif %}\
{% endif %}\

{% for group, commits in commits | unique(attribute="id") | filter(attribute="merge_commit", value=true) | group_by(attribute="group") %}
    ### {{ group | upper_first }}
    {% for commit in commits %}
        - {{ commit.message | split(pat="\n") | nth(n=2) | trim_end }} \
        ([PR #{{ commit.remote.pr_number }}](https://github.com/{{ remote.github.owner }}/{{ remote.github.repo }}/pull/{{ commit.remote.pr_number }})) \
        ([{{ commit.remote.username }}](https://github.com/{{ commit.remote.username }}))\
    {% endfor %}
{% endfor %}\n
"""

footer = """
\n\n*This changelog was automatically generated by [git-cliff](https://github.com/orhun/git-cliff)*
"""

trim = true

[git]
conventional_commits = false
filter_unconventional = false
split_commits = false
commit_parsers = [
  { field = "remote.pr_labels", pattern = "duplicate|invalid|wontfix|skip changelog", skip = true },
  { field = "remote.pr_labels", pattern = "breaking change", group = "<!-- 0 -->🏗️ Breaking Changes" },
  { field = "remote.pr_labels", pattern = "feature|deprecation", group = "<!-- 1 -->🚀 Features" },
  { field = "remote.pr_labels", pattern = "enhancement|refactor", group = "<!-- 1 -->🛠️ Enhancements" },
  { field = "remote.pr_labels", pattern = "bug|regression", group = "<!-- 2 -->🐛 Bug Fixes" },
  { field = "remote.pr_labels", pattern = "security", group = "<!-- 3 -->🔐 Security" },
  { field = "remote.pr_labels", pattern = "documentation", group = "<!-- 4 -->📝 Documentation" },
  { field = "remote.pr_labels", pattern = ".*", group = "<!-- 5 -->🌀 Miscellaneous" },
]
filter_commits = true
sort_commits = "newest"

Run this command from a console:

path/to/my/local/git-cliff/repository/target/debug/git-cliff -v

View the CHANGELOG.md:

# Changelog

## [2.2.0](https://github.com/mta-solutions/fsharp-data-validation/tree/v2.2.0) (2024-12-13)

### <!-- 1 -->🛠️ Enhancements

- Add additional VCtx functions ([PR #24](https://github.com/mta-solutions/fsharp-data-validation/pull/24)) ([RJSonnenberg](https://github.com/RJSonnenberg))

## [2.1.0](https://github.com/mta-solutions/fsharp-data-validation/tree/v2.1.0) (2024-12-10)

### <!-- 1 -->🛠️ Enhancements

- Remove internal scoping and add Async extension methods for validation contexts ([PR #22](https://github.com/mta-solutions/fsharp-data-validation/pull/22)) ([RJSonnenberg](https://github.com/RJSonnenberg))

### <!-- 4 -->📝 Documentation

- Prep v2.1.0 Release ([PR #23](https://github.com/mta-solutions/fsharp-data-validation/pull/23)) ([RJSonnenberg](https://github.com/RJSonnenberg))

## [2.0.0](https://github.com/mta-solutions/fsharp-data-validation/tree/v2.0.0) (2024-10-08)

### <!-- 1 -->🛠️ Enhancements

- Use FSharpPlus ([PR #20](https://github.com/mta-solutions/fsharp-data-validation/pull/20)) ([devinlyons](https://github.com/devinlyons))

### <!-- 4 -->📝 Documentation

- Prep for Release v2.0.0 ([PR #21](https://github.com/mta-solutions/fsharp-data-validation/pull/21)) ([devinlyons](https://github.com/devinlyons))

## [1.0.1](https://github.com/mta-solutions/fsharp-data-validation/tree/v1.0.1) (2023-11-17)

### <!-- 1 -->🛠️ Enhancements

- Convert to NuGet ([PR #18](https://github.com/mta-solutions/fsharp-data-validation/pull/18)) ([devinlyons](https://github.com/devinlyons))
- Add Giraffe model validation library and examples ([PR #17](https://github.com/mta-solutions/fsharp-data-validation/pull/17)) ([RJSonnenberg](https://github.com/RJSonnenberg))

## [1.0.0](https://github.com/mta-solutions/fsharp-data-validation/tree/v1.0.0) (2022-03-14)

### <!-- 1 -->🛠️ Enhancements

- Add utility functions to help with using NonEmptyLists [MBI-53] ([PR #14](https://github.com/mta-solutions/fsharp-data-validation/pull/14)) ([mleone11](https://github.com/mleone11))



*This changelog was automatically generated by [git-cliff](https://github.com/orhun/git-cliff)*

Screenshots / Logs (if applicable)

N/A

Types of Changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Documentation (no code change)
Refactor (refactoring production code)
Other

Checklist:

My code follows the code style of this project.
I have updated the documentation accordingly.
I have formatted the code with rustfmt.
I checked the lints with clippy.
I have added tests to cover my changes.
All new and existing tests passed.

Signed-off-by: Shingo OKAWA <shingo.okawa.g.h.c@gmail.com>

ognis1205 · 2025-06-04T00:55:26Z

cc: @RJSonnenberg

codecov-commenter · 2025-06-04T00:59:08Z

Codecov Report

Attention: Patch coverage is 94.44444% with 1 line in your changes missing coverage. Please review.

Project coverage is 42.30%. Comparing base (9970402) to head (c0335b6).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
git-cliff-core/src/commit.rs	94.45%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1163      +/-   ##
==========================================
+ Coverage   41.95%   42.30%   +0.35%     
==========================================
  Files          21       21              
  Lines        1981     1991      +10     
==========================================
+ Hits          831      842      +11     
+ Misses       1150     1149       -1

Flag	Coverage Δ
unit-tests	`42.30% <94.45%> (+0.35%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: Shingo OKAWA <shingo.okawa.g.h.c@gmail.com>

ognis1205 · 2025-06-06T01:56:54Z

Hey @RJSonnenberg (cc: @orhun ) ,

I generated the CHANGELOG.md on my local working branch build using the repository you provided (for details, please refer to the “How Has This Been Tested?” section of the PR). Grouping by remote.pr_labels is working, but there's one difference in behavior compared to the existing setup.

{ field = "remote.pr_labels", pattern = ".*", group = "<!-- 5 -->🌀 Miscellaneous" },

Previously, I believe this served as the fallback group setting. However, with the default MatchMode::Any, this does not match when remote.pr_labels is empty. Instead, using:

{ message = ".*", group = "<!-- 6 --> 🌀 Miscellaneous" },

continues to work as a fallback as before.

Note: The original rule ends up matching an empty JSON array ([]) as a string, which is probably not the intended behavior. It might be worth adding a comment or warning about this to avoid confusion.

Any thoughts on this?

…-json-array-field-matcher Signed-off-by: Shingo OKAWA <shingo.okawa.g.h.c@gmail.com>

RJSonnenberg · 2025-06-06T17:41:17Z

Hey @RJSonnenberg (cc: @orhun ) ,

I generated the CHANGELOG.md on my local working branch build using the repository you provided (for details, please refer to the “How Has This Been Tested?” section of the PR). Grouping by remote.pr_labels is working, but there's one difference in behavior compared to the existing setup.
{ field = "remote.pr_labels", pattern = ".*", group = "🌀 Miscellaneous" },
Previously, I believe this served as the fallback group setting. However, with the default MatchMode::Any, this does not match when remote.pr_labels is empty. Instead, using:
{ message = ".*", group = " 🌀 Miscellaneous" },
continues to work as a fallback as before.

Note: The original rule ends up matching an empty JSON array ([]) as a string, which is probably not the intended behavior. It might be worth adding a comment or warning about this to avoid confusion.

Any thoughts on this?

I think that is a fine compromise. Like you said, the original only worked as a fallback, so the new fallback group setting looks fine to me. It might throw people off, but as long as it is documented well, we should be good.

ognis1205 · 2025-06-06T17:51:50Z

@RJSonnenberg

Thanks for your feedback!

orhun · 2025-06-07T17:18:41Z

+	/// Mode to match regex against fields: any or all.
+	#[serde(default)]
+	pub match_mode:    MatchMode,


From reading the changes, it's not clear to me why this was added to fix this problem. Can you share some details?

Also, we should document this if it is a necessary change :)

Thanks for the question!

To clarify with a concrete case: imagine we want to match PRs where at least one of the labels matches a pattern like "enhancement|refactor". If a commit has remote.pr_labels = ["feature", "refactor"], then:

With match_mode = "any", this is a match (since "refactor" matches).

With match_mode = "all", it's not a match (since "feature" doesn't match).

Detailed Example

commit_parsers = [ { field = "remote.pr_labels", pattern = "enhancement|refactor", group = "🛠️ Enhancements" }, ]

Given:

remote.pr_labels = ["feature", "refactor"]

✅ MatchMode::Any

"feature" → ❌ (does not match)

"refactor" → ✅ (matches)

→ Since at least one label matches, the whole condition is satisfied.

❌ MatchMode::All

"feature" → ❌

"refactor" → ✅

→ Since not all labels match, the match fails.

Explanation

Previously, CommitParser implicitly treated the remote.pr_labels field as if MatchMode::Any were applied, except when the field was empty.
With this internal changes where the field is now handled as a Vec, I introduced match_mode to make the aggregation behavior of regex matching across vector elements explicit at the API level — i.e., whether the matches should be interpreted as an AND or an OR.
This change ensures that the matching semantics are both predictable and configurable moving forward.

Hmm I see... I still think that we can avoid this new field though. Maybe loop through each value in Value::Array and add them to the regex_checks array, wouldn't that work?

If we are going to go forward with it then we should document that this is useful for array types and also figure out if there are any other edge cases.

You're right, and I think the behavior you're describing is actually quite similar to what I implemented with MatchMode::Any. It maintains backward compatibility while enabling more flexible matching for array values.

I'll try rewriting the implementation with this in mind and see if it simplifies the design. Thanks for the input!

Cool, let me know about your progress :)

Thanks! I’ve implemented the changes as you suggested.
One thing to note—as I mentioned in this comment—is that the following fallback:

{ field = "remote.pr_labels", pattern = ".*", group = "🌀 Miscellaneous" },

no longer works as expected. Instead, you’ll need to use:

{ message = ".*", group = " 🌀 Miscellaneous" },

The core logic update is complete, and the only remaining task is to add fixtures. I plan to address that as discussed in this thread.

@orhun (cc: @RJSonnenberg)

Also, as an initial fixture test, I’ve added one based on the commit history from this repository:

https://github.com/mta-solutions/fsharp-data-validation

That said, if you think it’s better for maintainability to have a dedicated test case, as we discussed in the discussion thread, I can also add a commit history that uses PR labels explicitly for testing purposes. Let me know what you think :)

Nice!

We can update/expand the fixtures after that discussion is resolved :)

Thanks for your work on this!

Sure — we’ll continue the discussion there and figure out the best approach.
Thanks again for your support!

orhun · 2025-06-07T17:22:40Z

Hey guys! I just had some time catch up with #1157 and this PR - thanks for investigating and possible fix!

I'd like to take a step back and see a minimal example what's broken for understanding this fix better. @ognis1205 can you add a new fixture test? Preferably for grouping pr_labels which is know allegedly broken. I also think that adding that fixture will also help us to catch these regressions better in the future.

orhun · 2025-06-10T16:30:37Z

For the fixtures, you can create a dummy repository on GitHub and simply use that, like we do here or here

ognis1205 · 2025-06-11T04:11:17Z

@orhun

For the fixtures, you can create a dummy repository on GitHub and simply use that, like we do here or here

Thanks! I will add a fixture test that uses a dummy GitHub repository as suggested.

Signed-off-by: Shingo OKAWA <shingo.okawa.g.h.c@gmail.com>

orhun

LGTM, thanks!

ognis1205 · 2025-06-14T16:54:34Z

@orhun

I really appreciate your patience and thoroughness in the reviews. Thank you!

orhun · 2025-06-16T19:44:58Z

Thanks for implementing!

feat(parser): support regex matching on JSON arrays with scalar elements

5e6b7ab

Signed-off-by: Shingo OKAWA <shingo.okawa.g.h.c@gmail.com>

ognis1205 requested a review from orhun as a code owner June 4, 2025 00:54

RJSonnenberg approved these changes Jun 4, 2025

View reviewed changes

ognis1205 added 4 commits June 4, 2025 11:31

test: improve code coverage

4b76197

Signed-off-by: Shingo OKAWA <shingo.okawa.g.h.c@gmail.com>

refactor: improve code readability

edd54b0

Signed-off-by: Shingo OKAWA <shingo.okawa.g.h.c@gmail.com>

test: fix broken unit test

b5e0e55

Signed-off-by: Shingo OKAWA <shingo.okawa.g.h.c@gmail.com>

refactor: improve code readability

edb8ee5

Signed-off-by: Shingo OKAWA <shingo.okawa.g.h.c@gmail.com>

ognis1205 commented Jun 4, 2025

View reviewed changes

Comment thread git-cliff-core/src/commit.rs Outdated

chore: merge remote-tracking branch 'upstream/main' into feat/support…

3236143

…-json-array-field-matcher Signed-off-by: Shingo OKAWA <shingo.okawa.g.h.c@gmail.com>

ognis1205 mentioned this pull request Jun 6, 2025

Git-cliff does not parse changes after breaking change (tag sha-4c3c946) #1157

Closed

1 task done

orhun reviewed Jun 7, 2025

View reviewed changes

This comment was marked as resolved.

Sign in to view

ognis1205 mentioned this pull request Jun 7, 2025

feat: add commit statistics to the changelog #1151

Merged

7 of 12 tasks

ognis1205 added 4 commits June 12, 2025 12:25

refactor: revert regex_checks to String and inline scalar array handling

656630f

Signed-off-by: Shingo OKAWA <shingo.okawa.g.h.c@gmail.com>

test: add fixture test

ae5c8a7

Signed-off-by: Shingo OKAWA <shingo.okawa.g.h.c@gmail.com>

fix(test): add new line

f4bf984

Signed-off-by: Shingo OKAWA <shingo.okawa.g.h.c@gmail.com>

test: update fixture

1576aea

Signed-off-by: Shingo OKAWA <shingo.okawa.g.h.c@gmail.com>

ognis1205 requested review from RJSonnenberg and orhun June 13, 2025 07:15

RJSonnenberg approved these changes Jun 13, 2025

View reviewed changes

refactor: remove unnecessary comma

c0335b6

orhun approved these changes Jun 14, 2025

View reviewed changes

orhun merged commit dc458ea into orhun:main Jun 14, 2025

ognis1205 deleted the feat/support-json-array-field-matcher branch June 14, 2025 16:54

This was referenced Jun 18, 2025

chore(fixture): update test-regex-json-array fixture #1178

Merged

Using github.pr_labels removes all commits silently #1189

Closed

BrewTestBot mentioned this pull request Jul 27, 2025

git-cliff 2.10.0 Homebrew/homebrew-core#231368

Merged

Conversation

ognis1205 commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

How Has This Been Tested?

Screenshots / Logs (if applicable)

Types of Changes

Checklist:

Uh oh!

ognis1205 commented Jun 4, 2025

Uh oh!

codecov-commenter commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

ognis1205 commented Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RJSonnenberg commented Jun 6, 2025

Uh oh!

ognis1205 commented Jun 6, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ognis1205 Jun 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Detailed Example

Explanation

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

orhun commented Jun 7, 2025

Uh oh!

This comment was marked as resolved.

orhun commented Jun 10, 2025

Uh oh!

ognis1205 commented Jun 11, 2025

Uh oh!

orhun left a comment

Choose a reason for hiding this comment

Uh oh!

ognis1205 commented Jun 14, 2025

Uh oh!

orhun commented Jun 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ognis1205 commented Jun 4, 2025 •

edited

Loading

codecov-commenter commented Jun 4, 2025 •

edited

Loading

ognis1205 commented Jun 6, 2025 •

edited

Loading

ognis1205 Jun 7, 2025 •

edited

Loading