[SRILM User List] unexplained failure in ngram-count

[Andrew Philpot]

ngram-count failed on a largish file (but not the largest I was
considering applying it to).  The file contained 325 million short
sentences, and was 10 GB in total size.

I am running on a 64-bit machine with 16GB physical memory (2 2.3 Ghz
CPU x 4 cores) running RH EL5.

The precise invocation was:

./bin/i686-m64/ngram-count -text all_lm5 -no-sos -no-eos -unk 
-write-vocab vocab5.txt -write counts5.txt -order 5 -lm lm5.txt

The precise error message was:

ngram-count: /home/eh-01/philpot/srilm/include/LHash.cc:138: void 
LHash<KeyT, DataT>::alloc(unsigned int) [with KeyT = unsigned int, DataT 
= Trie<unsigned int, long unsigned int>]: Assertion `body != 0' failed.

Is this likely to be a problem with my data, with my machine's memory,
with the code, or what?

Andrew