[SRILM User List] ngram: sentence boundary markers in text file used with -ppl?

Sander Maijers S.N.Maijers at student.ru.nl
Sat May 25 09:27:45 PDT 2013


Hi,

Should one the sentences in the sentences file for ngram's '-ppl' with 
<s> sos and </s> eos tokens? They are in the LM.

I have tested it just now, and it seems that the sentence boundary 
markers are inferred by ngram when left out, and adopted when put in. 
Where is this documented?

Best,
Sander


More information about the SRILM-User mailing list