Perplexity in "ngram"

[Mats Svenson]

Hi,
 I have tried to use "ngram" to count perplexity of my
LMs. However, I am not sure how does the srilm
implementation treat OOVs in terms of counting
perplexity. Is it that "log P(<unk>|history) != 0" or
OOVs are just ignored? If a model with a higher number
of OOVs has a lower perplexity than another LM, does
it mean that it is "better" in this -ppl
implementation?

Second, in some discussions, I have heard about -ppl1
option, but the current version does not seem to have
it. In what -ppl1 differs from -ppl?

Third, is there a way how to meaningfully compute
perplexity for a hidden event LM? Or another way how
to evaluate hidden event LM quality?

Thanks for your help,
 Mats


 
____________________________________________________________________________________
Get your own web address.  
Have a HUGE year through Yahoo! Small Business.
http://smallbusiness.yahoo.com/domains/?p=BESTDEAL