Byte-Level BPE: Unicode-agnostic tokenization. Handles any character and out-of-vocabulary words. Balances efficiency and representation.

Stochastic tokenization improving robustness. Applicable in BPE pre-training and fine-tuning.Balances consistency and variability.

Dynamic BPE: Adaptive tokenization for pre-training and fine-tuning. Balances flexibility and consistency.

BPE Dropout: Stochastic subword segmentation. Applies dropout to merges during tokenization. Improves model robustness and generalization.

Word Piece Tokenization: Subword segmentation for NLP. Builds vocab from frequent subwords & handles rare words

Byte Pair Tokenization: Efficient subword segmentation. Merges frequent character pairs, handles unseen words, scales to sentences.