You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
fix(scripts): Fix process exit code type error (#512)
Since the node20 update process.exit(!!err) is not working anymore and
throwing `TypeError [ERR_INVALID_ARG_TYPE]: The "code" argument must be
of type number. Received type boolean (false)`, because the boolean is
not converted to a number anymore.
feat(schema): add beta ICU tokenizer
* Use ICU tokenizer to improve some Asian languages support
* Remove unused import
* Add more chinese test cases
* Add icuTokenizer flag
* Implement ICU tokenizer test
* Run unit tests for both ICU = true/false
* Run tests for both ICU = true/false
* add fixtures
* Fix bug in settings
* Fix tests
* Fix tests
* Fix tests
* Fix tests
* feat(schema): add beta ICU tokenizer
---------
Co-authored-by: Peter Johnson <insomnia@rcpt.at>