Factored LMs and interpolated models

[Andreas Stolcke]

Hi,

you SHOULD be able to do this with

	ngram -factored -bayes 0 ...
 
followed by the usual options to specify mixtures of LMs.   This is because
the -factored option causes all LM components to be interpreted as factored
LMs, and this causes the standard interpolation mechanism to be wrapped
around them.

So, all you have to do is implement your standard word ngram and class
ngrams each separately as FLMs.   This is not quite what you asked for
but it should be equivalent.

This is the theory.  I don't think we ever tested this, so there might be 
glitches.  But those could be fixed if necessary, the basic machinery is
there.

There might also be a different approach.  You could engineer the FLM
definition so that at the highest level you always back off.  Then you specify
interpolation as the backoff strategy, emulating word and class ngrams
as two parallel backoff paths.  I'm not sure this will work with the current
functionality, it's just an idea.  Katrin or Jeff should be able to tell you
if it's feasible.

--Andreas

In message <409A784F.8060305 at ee.washington.edu>you wrote:
> There is no easy way to interpolate word and class ngram models in the 
> factored language model framework. Factor language models support only 
> interpolation of an N-gram probability estimate and its corresponding 
> lower-order estimate, which is similar to the "interpolate" option in 
> "ngram-count."
> 
> You could conceivably treat the word and the class as your factors and 
> perform interpolation whenever you back off from one set of these 
> conditioning variables to a subset. However, this backoff nature makes 
> the interpolation different from the traditional interpolation of 
> parallel n-grams. Probably the best thing to do is to use the usual 
> SRILM tools for this.
> 
> Hope this helps,
> Kevin Duh
> 
> Tanel Alumäe wrote:
> 
> >Hello,
> >
> >I'm experimenting with factored language modeling implementation in
> >SRILM. I got some nice results and now want to compare them with the
> >traditional approach where a word-trigram LM is interpolated with the
> >parallel class trigram. Is it possible to create a factored LM that
> >actually implements such traditional interpolation? 
> >
> >Thanks in advance,
> >Tanel A.
> >  
> >
>