[SRILM User List] ngram crashing

[Oldrich Kruza]

Eh, yes. It was a 32-bit executable.
Thank you.

On Tue, Mar 18, 2014 at 7:21 PM, Andreas Stolcke
<stolcke at icsi.berkeley.edu> wrote:
> On 3/18/2014 8:20 AM, Oldrich Kruza wrote:
>>
>>
>> Hello everybody.
>>
>> I'm trying to reduce the vocabulary of a huge (64GB) trigram language
>> model.
>>
>> I ran the script change-lm-vocab, and the ngram process died with this
>> error message:
>>
>> include/LHash.cc:141: void LHash<KeyT, DataT>::alloc(unsigned int) [with
>> KeyT = unsigned int, DataT = Trie<unsigned int, BOnode>]: Assertion `body !=
>> 0' failed.
>>
>> I'm positive this is not due to insufficient memory.
>>
> This is the error message when SRILM fails to allocate more memory. The
> reasons could be
>
> - you are using a 32bit binary and running up against the 4GB limit of the
> architecture
> - you have a memory resource limit in force (set by you or your sysadmin) -
> check the ulimit or limit (csh) command
> - your system is actually, really out of memory (which also depends on what
> other users are doing)
>
> By running top or some similar tool concurrently you can see how big your
> ngram process actually grows before crashing, and this can give you
> additional clues.
>
> Andreas
>