How to understand this formula in Lingpipe language model?

跟風遠走 提交于 2020-01-15 11:54:19

问题


This is from manual of Lingpipe doc in building a language model. But I only partly understand the theory behind it.

I especially do not know the base probability.

Here, how to get base p(d). If below is portion of token and their freq in unigram file.

ab  20
aba 3
abd 2
abef 2
abkk 3

Under such condition, what is lamda(),1-lamda(), extcount, numExtentions and Base P(ab)? This is one question but they are chained.

Thanks a lot.

来源:https://stackoverflow.com/questions/10797878/how-to-understand-this-formula-in-lingpipe-language-model

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!