[SRILM User List] Models bigger than 3-gram issue

Lucian Georgescu lucgeo92 at gmail.com
Mon Dec 19 07:17:49 PST 2016


I use SRI-LM toolkit and I have two questions:

1. I created two different 4-gram models using "ngram-count" command. Then I wanted to interpolate them using "ngram" command with "mix-lm" option. The final language model obtained contains only 1-grams, 2-grams and 3-grams. Does SRI-LM support interpolation for models bigger than 3-gram?

2. I tried to evaluate perplexity for a 4-gram model. For this task I used this command: "ngram -debug 2 -lm 4-gram.model -ppl testCorpus.txt > results.txt". In console output I got these messages:
reading 100002 1-grams
reading 18094791 2-grams
reading 24314260 3-grams
skipping 31712365 4-grams

The command is running fine and I got the results file, but it seems the 4-gram terms were not taken into account (skiping 4-grams). It is possible to evaluate models bigger than 3-gram?

Thank you.

More information about the SRILM-User mailing list