<html><head></head><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px"><div id="yui_3_16_0_ym19_1_1485258924481_3113">Hello. I have a question regarding perplexity. I am using srilm to compute the perplexity of some sentences using a LM trained on a big corpus. Given a sentence and a LM, the perplexity tells how well that sentence fits to the language (as far as i understood). And the lower the perplexity, the better the sentence fits.</div><div id="yui_3_16_0_ym19_1_1485258924481_3163"><br></div><div id="yui_3_16_0_ym19_1_1485258924481_3159">$NGRAMCOUNT_FILE -order 5 -interpolate -kndiscount -unk -text Wikipedia.en-es.es -lm lm/lmodel_es.lm</div><div id="yui_3_16_0_ym19_1_1485258924481_3160"><br id="yui_3_16_0_ym19_1_1485258924481_3161"></div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3162">$NGRAM_FILE -order 5 -debug 1 -unk -lm lm/lmodel_es.lm -ppl testlabeled.en-es.es > perplexity_es_testlabeled.ppl</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3162"><br></div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3162">I did the same on EN and on ES and here are some results I got:</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3162"><br></div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3267">Sixty-six parent coordinators were laid off," the draft complaint says, "and not merely excessed.</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3268">1 sentences, 14 words, 0 OOVs</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3269">0 zeroprobs, logprob= -62.106 ppl= 13816.6 ppl1= 27298.9</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3269"><br></div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3305">Mexico's Enrique Pena Nieto faces tough start</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3306">1 sentences, 7 words, 0 OOVs</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3307">0 zeroprobs, logprob= -39.1759 ppl= 78883.7 ppl1= 394964</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3307"><br></div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_6907">The NATO mission officially ended Oct. 31.</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_6908">1 sentences, 7 words, 0 OOVs</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_6909">0 zeroprobs, logprob= -29.2706 ppl= 4558.57 ppl1= 15188.6</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3307"><br></div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3462">Sesenta y seis padres coordinadores fueron despedidos," el proyecto de denuncia, dice, "y no simplemente excessed.</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3463">1 sentences, 16 words, 0 OOVs</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3464">0 zeroprobs, logprob= -57.0322 ppl= 2263.79 ppl1= 3668.72</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3374"><br></div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3381">México Enrique Peña Nieto enfrenta duras comienzo</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3382">1 sentences, 7 words, 0 OOVs</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3383">0 zeroprobs, logprob= -29.5672 ppl= 4964.71 ppl1= 16744.7</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3384"><br id="yui_3_16_0_ym19_1_1485258924481_3385"></div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3384"><br></div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3384">Why are the perplexities for the EN sentences so big? The smallest ppl i get for an EN sentence is about 250. The spanish sentences have some errors, so i was expecting big ppl numbers. Should i change something in the way i compute the lms?</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3384"><br></div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3384">Thank you very much!!</div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3233"><br></div><div dir="ltr" id="yui_3_16_0_ym19_1_1485258924481_3233"><br></div></div></body></html>