Dynamic BPE
Dynamic BPE: Adaptive tokenization for pre-training and fine-tuning. Balances flexibility and consistency.
top of page
Jun 304 min read
BPE Dropout
BPE Dropout: Stochastic subword segmentation. Applies dropout to merges during tokenization. Improves model robustness and generalization.
Jun 2811 min read
WordPiece Tokenization: A BPE Variant
Word Piece Tokenization: Subword segmentation for NLP. Builds vocab from frequent subwords & handles rare words
bottom of page