A grammar describes the syntax of a programming language, and might be defined in Backus-Naur form (BNF). A lexer performs lexical analysis, turning text into tokens. A parser takes tokens and builds a data structure like an abstract syntax tree (AST). The parser is concerned with context: does the sequence of tokens fit the grammar? A compiler is a combined lexer and parser, built for a specific grammar.
This is the result of my scientific work on the creation of a question-answer system for the Kazakh language. Google drive presents 4 models with different learning depths, as well as with different datasets on which we trained the models.
📄 | Recursive descent parser | Abstract Syntax Trees | Tokenizer
Lightweight tokenizer and syntax highlighter
A platform-agnostic Mash creation algorithm
An implementation of Keras Tokenizer in JavaScript.
Parser for programming languages, used to analyze their structures, axiomatically you can use it to parse math expressions.
A secure, tokenized mobile wallet system optimized for vault transactions and revenue growth. Open Wallet is an open-source wallet infrastructure that transforms how businesses handle payments by leveraging tokenized vaults to maximize revenue while maintaining PCI DSS compliance.
JavaScript text obfuscator
This project is a full-stack authentication system built using the MERN stack. It offers user registration, login with email or Google OAuth, profile management, and token-based authentication. Key features include error handling, password hashing, and JWT implementation. The frontend is developed with React and Redux for state management.
Simplified token definition and lexer creation library for use with Chevrotain.
Tokenizes Chinese texts into words.
Simple synchronous string tokenizer using Regex
😸 A LOLCODE interpreter written in JavaScript
react-structured-query-search-suggestions is a javascript library that provides autocomplete search queries with dynamic/static suggestions.