🌿 NodeJS PHP Parser - extract AST or tokens
-
Updated
Jul 18, 2025 - JavaScript
#
tokenizer
A grammar describes the syntax of a programming language, and might be defined in Backus-Naur form (BNF). A lexer performs lexical analysis, turning text into tokens. A parser takes tokens and builds a data structure like an abstract syntax tree (AST). The parser is concerned with context: does the sequence of tokens fit the grammar? A compiler is a combined lexer and parser, built for a specific grammar.
Here are 111 public repositories matching this topic...
Optimised tokenizer/lexer generator! 🐄 Uses /y for performance. Moo.
-
Updated
May 16, 2023 - JavaScript
Multilingual tokenizer that automatically tags each token with its type
multilingual german tokenizer tagging latin french hindi wink devanagari marathi tokenization konkani
-
Updated
Mar 4, 2023 - JavaScript
POS Tagger, lemmatizer and stemmer for french language in javascript
-
Updated
Sep 13, 2017 - JavaScript
Collection of developer toolkits
parser tokenizer devtools developer-tools lexer development-workflow development-environment developer-toolkit
-
Updated
May 25, 2018 - JavaScript
JavaScript Reverse Engineering Toolkit (JSRETK) - Experimental tools for analyzing (minified/obfuscated) JavaScript
javascript node js strings tokenizer static-analysis ecmascript reverse-engineering ast esprima node-js minified reversing abstract-syntax-tree
-
Updated
Dec 12, 2025 - JavaScript
Converts a string into an array of tokens, with useful methods for looking ahead and behind, capturing, matching, et cetera.
-
Updated
Nov 19, 2018 - JavaScript
A small ECMAScript parser, tokenizer and minifier written in JavaScript.
-
Updated
Apr 21, 2016 - JavaScript
- Followers
- 11k followers
- Website
- github.com/topics/parsing
- Wikipedia
- Wikipedia