Skip to content

Conversation

@DmitrySharabin
Copy link
Member

@DmitrySharabin DmitrySharabin commented Nov 18, 2025

The code from the simplify branch, authored mainly by @LeaVerou.

Summary

  • Add tokenizeByNamedGroups() and camelToKebabCase() utils
  • resolve() now accepts functional tokens
  • Adjust _matchGrammar() to not choke on functional tokens
  • Transform the Markdown language to use named token groups and adjust existing tests accordingly (as a proof of concept)
  • In the pattern and coverage tests, add support for named capture groups and regexp flags
  • Tweak TS types accordingly

With #4018 and #4029 merged, the changes in this PR don't break any tests; all of them still pass.


This is part 2 of 2 in a stack made with GitButler:

@github-actions
Copy link

github-actions bot commented Nov 18, 2025

No JS Changes

Generated by 🚫 dangerJS against 2fe2161

@DmitrySharabin DmitrySharabin force-pushed the functional-inside branch 2 times, most recently from c7f907f to 3eb1e71 Compare November 18, 2025 22:14
!grammar.hasOwnProperty(token) ||
token.startsWith('$') ||
!tokenValue ||
typeof tokenValue === 'function'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Functional tokens ($inside for now) are handled on L170, and we should ignore them in all other cases.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be a comment?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, it should. I'll add it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Base automatically changed from simplify-js to v2 December 4, 2025 12:36
@netlify
Copy link

netlify bot commented Dec 4, 2025

Deploy Preview for dev-prismjs-com ready!

Name Link
🔨 Latest commit 2fe2161
🔍 Latest deploy log https://app.netlify.com/projects/dev-prismjs-com/deploys/6943d5643b28f70008fa7ac1
😎 Deploy Preview https://deploy-preview-4032--dev-prismjs-com.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

…objects

The regex coverage test was failing to track pattern matches because Prism creates new `RegExp objects` when adding flags (e.g., `g` or `d`) during tokenization. The original approach intercepted `exec()` on individual regex objects, which missed matches on the newly created `RegExp` instances.

Changes:
- Replace `String(regex)` key with normalized source+flags key to match patterns even when Prism creates new `RegExp` objects with different flags
- Switch from per-regex interception to global `RegExp.prototype.exec` interception to catch all pattern matches, including on new `RegExp` objects
- Use a simple loop instead of `String.replace` in flag normalization to avoid triggering our own `RegExp.exec` interception (which caused infinite loops)

This fixes the tracking issue for patterns like the markdown code block pattern that were previously reported as untested despite being used in tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants