[SRILM User List] Unexpected error using make-big-lm

[Andreas Stolcke]

jian zhang wrote:
>
> Hi there,
>
> I am very new to srilm tool, i will do my best to explain my problem.
> I am playing around with the feature of building a large language at 
> the moment. The batch counts have been built using make-batch-counts 
> script. But when i am building the lm, I have got unexpected error like,
> + ngram-count -read - -read-with-mincounts -order 5 -kn1 biglm.kn1 
> -kn2 biglm.kn2 -kn3 biglm.kn3 -kn4 biglm.kn4 -kn5 biglm.kn5 -lm 
> testlm.lm -interpolate -unk -meta-tag __meta__ -kn-counts-modified
> Unexpected error.
> /home/srilm/bin/make-big-lm: line 225: 27562 Aborted                 
> ngram-count -read - -read-with-mincounts -order $order $gtflags $options

The ngram-count process exited with an "abort" signal, meaning it 
probably ran out of memory.
As a sanity check, try building your LM on a small subset of the data 
(say, using no more than a million words), using the same procedure.

Assuming that works, please check the FAQ page on strategies for dealing 
with insufficient memory.

Andreas