make-big-lm out of memory

Alexy Khrabrov deliverable at gmail.com
Sat Oct 4 08:46:56 PDT 2008


I've got a couple corpora where make-big-lm is killed after a couple  
days of swapping.  I've originally started it after merge-batch-counts  
generated the total counts fine, and the kncounts file is produced OK  
as a part of the make-big-lm run.  I've used a -max-per-file option,  
but once the kncounts is done, I understand it no longer applies?   
Basically trying to rerun the same command line again reuses the  
kncounts and the thing gets killed again.  Which strategies do we have  
to overcome this?

Cheers,
Alexy



More information about the SRILM-User mailing list