make-big-lm out of memory
Alexy Khrabrov
deliverable at gmail.com
Sat Oct 4 08:46:56 PDT 2008
I've got a couple corpora where make-big-lm is killed after a couple
days of swapping. I've originally started it after merge-batch-counts
generated the total counts fine, and the kncounts file is produced OK
as a part of the make-big-lm run. I've used a -max-per-file option,
but once the kncounts is done, I understand it no longer applies?
Basically trying to rerun the same command line again reuses the
kncounts and the thing gets killed again. Which strategies do we have
to overcome this?
Cheers,
Alexy
More information about the SRILM-User
mailing list