tokenizaiton

FPE - Format Preserving Encryption with FF3 in Java

Build a model to classify text as positive, negative, or neutral. Apply NLP techniques for preprocessing and machine learning for classification. Aim for accurate sentiment prediction on various text formats.

Debt Algorand Standard Application

Decentralized application for vehicle sharing using tokenization

API for CRUD using Token Authentication(Customised) & SQLAlchemy

This project focuses on sentiment analysis of movie reviews using the IMDB dataset. The goal is to classify reviews as positive or negative using TF-IDF techniques and Word2Vec.

Coming Soon! Tokenize anything: assets, identities, contacts, goals, data, etc. - only sky is the limit in the new token economy!

2nd version/part of this project

KMer level tokenizer for DNA & Proteins sequences

C++ function to tokenize the string.

This repository provides a hands-on exploration of SentencePiece tokenization and Byte-Pair Encoding (BPE) .The code demonstrates data preprocessing steps like NFKC normalization and lossless tokenization, followed by a practical implementation of the BPE algorithm from scratch.