[SRILM User List] ngram: sentence boundary markers in text file used with -ppl? [edit]

Sander Maijers S.N.Maijers at student.ru.nl
Sat May 25 13:37:12 PDT 2013


Hi,

Should one surround the sentences in the sentences file for ngram's 
'-ppl' with <s> sos and </s> eos tokens? They are in the LM.

I have tested it just now, and it seems that the sentence boundary 
markers are inferred by ngram when left out, and adopted when put in. 
Where is this documented?

Best,
Sander
_______________________________________________
SRILM-User site list
SRILM-User at speech.sri.com
http://www.speech.sri.com/mailman/listinfo/srilm-user


More information about the SRILM-User mailing list