Comparision between SRILM and CMU

[Tolos, Marta]

Hi all,

I have a general question about the toolkit. I have just started using this
SRILM toolkit, before I always used CMU toolkit, so I wanted to do a
comparision between the language models created with one and with the other
toolkit. So I created language models with the same corpus using both
toolkits, and I compute the perplexities with each toolkit (I mean, that I
use the same toolkit for creation and evaluation of the perplexity) and the
perplexities were quite different always better for the SRILM, so then I
tried to compute the perplexities of the CMU language models with the SRILM
toolkit and then I got strange results, since most of the time the
performance of the same CMU language model was better when computing the
perplexity with SRILM instead of CMU, except for one case were the value
that the SRILM gave was extremely high. After this, I did it the other way
arround, I used the CMU to evaluate the SRILM language models, and after
some trouble because of the format and some special requeriments of the CMU
toolkit, I got worse results when using the CMU toolkit for evaluating the
perplexity of the SRILM language models (and when the text used for
evaluating perplexity contained OOV words, CMU gave an error.) My question
is what is the difference in the computation of perplexity in the two
toolkits. And also what is the meaning of the "ppl1" that SRILM toolkit
gives.

Thanks a lot,

Marta