The Lexical Imprint of Pre-training Data
This study shows that lexical alignment between pre-training data and benchmarks, measured by unigram cross-entropy, predicts zero-shot accuracy and guides data selection and prompt routing.
ls ./papers | grep "#language model pre-training"
Trending AI research papers from Hugging Face, summarized in plain language: what each paper does, why it matters and where to read more.
1 paper · #language model pre-training
This study shows that lexical alignment between pre-training data and benchmarks, measured by unigram cross-entropy, predicts zero-shot accuracy and guides data selection and prompt routing.