[SRILM User List] big difference between ppl and ppl1

[Burkay Gur]

Is your Dutch model arranged so that there is one sentence on each line? Also which command are you using? I recommend using -gt1max 1 -gt2max 1 -gt3max 1 and -ukndiscount for kneser ney smoothing. These will give you more accurate perplexities.

-Burkay

Sent from my iPad

On Dec 27, 2011, at 6:26 AM, Saman Noorzadeh <saman_2004 at yahoo.com> wrote:

> 
> I  made 2 models of 2 languages, Dutch and English, to make a language recognition.
> I got the following perplexities:
> 
> Model: Dutch    Test: English    ppl:55    ppl2: 2* 10^18
> Model: Dutch    Test: Dutch    ppl:303    ppl2: 400
> Model: English    Test: Dutch    ppl: 600   ppl2: 3122ses n
> Model: English   Test: English    ppl: 227    ppl2: 1897
> 
> I think it is reasonable if I have a large perplexity when my model and test are different but why ppl=55 when having a Duch model and an English test?
> and
> Why is there a BIG difference in their ppl and ppl1 ?
> 
> Thanks in advance
> 
> 
> _______________________________________________
> SRILM-User site list
> SRILM-User at speech.sri.com
> http://www.speech.sri.com/mailman/listinfo/srilm-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.speech.sri.com/pipermail/srilm-user/attachments/20111227/e0c542b8/attachment.html>