7z as a much better archiver than gz/bz2

[Alexy Khrabrov]

(Kamadev -- I think you misunderstood my message.)  I was wondering  
whether folks manage use 7z to speed up their access to their LMs.   
By default, ngram would read the gzipped files as well as the  
originals.  Yet gzipped versions are still much larger than the  
7z'ipped.  7z is an Open Source package with which I have no  
affiliation...

By looking over 7z options, I found that one can extract a file to  
stdout with it  too, e.g.

7z e archive.7z -so

It would be possible to do that for a huge LM and feed that to it by  
piping to

ngram -lm -

-- yet the problem is, I use 

ngram -ppl -

already to serve perplexities.

Would appreciate other folks' experiences with speeding up loading of  
huge LM.  Same could be applied to bz2 as well, and any other  
archiver better than gz.

On Nov 10, 2007, at 11:59 PM, Kamadev Bhanuprasad wrote:
[...]