Optimizing Weights in Log-Linear Interpolation

[Sibel Yaman]

Hello,
I was wondering how I can train the weights in log-linear interpolation of
several language models (as in Klakow's paper). 
I have successfully used "compute-best-mix" script to use linear
interpolation weights but do not see how to modify the process to optimize
log-linear interpolation weights so that the perplexity is minimized on a
cross-validation set.
Thank you,
Sibel Yaman
From: Andreas Stolcke <stolcke at ADDRESS HIDDEN>
Date: Mon, 18 Jul 2005 07:28:40 PDT
In message <32809.213.58.88.69.1081673875.squirrel at ADDRESS HIDDEN>you
wrote:
> 
> Hi!
> 
> Does anyone know a program or toolkit allowing to do log-linear
> interpolation of different language models? since SRILM only permit to do
> linear interpolation.
> Thanks for your help,
> 
> Ciro Martins

Ciro,

sorry for the late response ;-)

There is now, in the current development version of SRILM, an
implementation of log-linear interpolation. The class name is
LoglinearMix, and the ngram -loglinear-mix option triggers its use.
Note that log-linear interpolation is much slower to evaluate than
linear interpolation, due to the need to normalize the combined LM.
This is done somewhat efficiently in SRILM by caching the normalizers
for previously seen contexts.

You might also want to try using log-linear combination of LM scores
without normalization. This can be done in the nbest or lattice
rescoring framework implemented by the toolkit, simply by computing
scores from multiple LMs.

The latest version of the toolkit can by downloaded in the usual way
by choosing the "1.4.5 (Beta)" version in the web form.

--Andreas 

Click here </projects/srilm/>  to go to the SRILM home page.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.speech.sri.com/pipermail/srilm-user/attachments/20080325/3a0bc6c6/attachment.html>