BPE Dropout
BPE Dropout: Stochastic subword segmentation. Applies dropout to merges during tokenization. Improves model robustness and generalization.
top of page
Jun 2811 min read
WordPiece Tokenization: A BPE Variant
Word Piece Tokenization: Subword segmentation for NLP. Builds vocab from frequent subwords & handles rare words
Jun 2212 min read
Mastering Masked Language Models: Techniques, Comparisons, and Best Practices
Explore the evolution of Masked Language Modeling: from BERT to XLNet, uncover how to choose the right technique for your NLP tasks.
bottom of page