问题
This is from manual of Lingpipe doc in building a language model. But I only partly understand the theory behind it.
I especially do not know the base probability.


Here, how to get base p(d). If below is portion of token and their freq in unigram file.
ab 20
aba 3
abd 2
abef 2
abkk 3
Under such condition, what is lamda(),1-lamda(), extcount, numExtentions and Base P(ab)? This is one question but they are chained.
Thanks a lot.
来源:https://stackoverflow.com/questions/10797878/how-to-understand-this-formula-in-lingpipe-language-model