Some updates

• I ended up working with types, moving the complex ones to types.d.ts and importing them where needed
• All .d.ts files we generate automatically from JSDoc seem to be generated fine. However, there's an issue with the TypeScript compiler: if we have both @private and @type on class properties, it ignores either the property type (and doesn't emit it) or the property modifier. I would suggest using real JS private properties in such cases. For now, I think the correct types are more important. In most cases, we follow the underscore convention for naming such properties in the code base.

• All NPM scripts in package.json adjusted accordingly (no more script for checking TS)

I think it may be good to have something that catches inconsistencies in types, as long as it's opt-in and doesn't block CI.

This is the final step that should be taken. @LeaVerou, could you elaborate a bit? Do we want to check JS files using the TS compiler and show errors? Should it work the same way the lint script does (when we run ESLint in the terminal)?

may I ask why the Prism project is moving from TS to JS? It'd be great to have reasoning in the PR body since JSDoc is considered less ergonomic than Typescript.

• All NPM scripts in package.json adjusted accordingly (no more script for checking TS)

I think it may be good to have something that catches inconsistencies in types, as long as it's opt-in and doesn't block CI.

This is the final step that should be taken. @LeaVerou, could you elaborate a bit? Do we want to check JS files using the TS compiler and show errors? Should it work the same way the lint script does (when we run ESLint in the terminal)?

Some more updates:

• Restore tsconfig.json but allow implicit any
• Add the typecheck NPM script (like lint but for types)

Now, all JS files (except demo.js, which comes with plugins) are type-checked, but it's not blocking the CI.

@scarf005

may I ask why the Prism project moving from TS to JS? It'd be great to have reasoning in the PR body since JSDoc is considered less ergonomic than Typescript.

Please note that there has never been a released Prism version using TS, so the Prism project is not moving to JS, it is simply not moving to TS. A past maintainer rewrote the codebase to use TS as it would help him move faster, and I agreed in the interest of making progress. Since he left and I took over again (with my apprentice @DmitrySharabin's help) we really gave TS a good shot since the v2 codebase was already using it and it was cheaper than switching at that point.

However, the DX has been a constant uphill battle, and we found we were spending more time dealing with TS than actually making progress on v2. Besides the sheer time spent jumping through TS hoops to simply express how things worked, hoops that provided absolutely no benefit in terms of finding bugs more easily, there were also the pointless TS assertions for obvious (to a human) things that added noise all over the codebase. Go through the code this PR removes (e.g. in the grammar files) for many samples of this.

TS is fantastic for getting folks from classical strongly typed / statically typed languages into JS. But for JS-native folks, it means that many of the dynamic coding and metaprogramming paradigms that JS allows (and that Prism takes good advantage of) are very challenging to express in TS.

Additionally, Prism is the type of project where performance can be critical, so there is an advantage to avoiding such heavy abstractions unless absolutely necessary. Staying closer to the metal (at least as much as one can be with JS) allows to better optimize highlighting.

Do note that using JS doesn't mean the project is untyped, or that all types are now defined in JSDoc. JSDoc is used for simple types, whereas complex types are defined in .d.ts files that are imported in JSDoc. Type checking is there if we need it to, it's just not on the critical path for contributions or runtime. Many projects have been using this architecture, e.g. Svelte.

If you're a TS user, you could help by trying out this PR and reporting back on the experience!

All issues with types (which appeared after allowing TS checking of JS files) are now fixed. If we still had a CI task for checking types, we would pass it. That means the changes in this PR don't make the codebase worse than it was before.

I think the PR is ready for review.

@DmitrySharabin I just went through the PR, though not very deeply. All LGTM, except the comment I left: why do we still need a build step given that lazy grammars are (supposed to be) evaluated when they're first used? That's the whole point of lazy grammars — otherwise they're not saving anything if we pre-build them.

(thinking @getify would approve of the spirit of this PR 😀)

Seconded 😃

nicely done! 💯