[SRILM User List] Right way to build LM
stolcke at icsi.berkeley.edu
Tue Apr 29 23:39:04 PDT 2014
On 4/28/2014 7:38 PM, Ismail Rusli wrote:
> Thanks for the answer, Andreas.
> As i read paper by
> Chen and Goodman (1999), they used held-out data
> to optimize parameters in language model. How do i
> do this in SRILM? Does SRILM optimize parameters
> when i use -kndiscount?
SRILM just uses the formulas for estimating the discounts from the
count-of-counts, i.e., equations (26) in the Chen & Goodman technical
> I tried -kn to save
> parameters in a file and included this file
> when building LM but it turned out
> my perplexity is getting bigger.
You can save the discounting parameters using:
1) ngram-count -read COUNTS -kndiscount -kn1 K1 -kn2 K2 -kn3 K3
(no -lm argument!)
Then you can read them back in for LM estimation using
2) ngram-count -read COUNTS -kndiscount -kn1 K1 -kn2 K2 -kn3 K3 -lm LM
and the result will be identical to the second command when run without
Now, if you want you can manipulate the discounting parameters before
invoking command 2.
For example, you could perform a search over the D1, D2, D3 parameters
optimizing perplexity on a held-out set, just like C&G did. But you
have to implement that search yourself by writing some wrapper scripts.
Also consider the interpolated version of KN smoothing. Just add the
ngram-count -interpolate option, it usually gives slightly better results.
> And just one more,
> do you have a link to good tutorial in using
> class-based models with SRILM?
There is a basic tutorial at
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the SRILM-User