Inquiry about expanding interpolated LM (class model + word model) to word model

[Andreas Stolcke]

Xiaodan Zhuang wrote:
> Dear Andreas,
>
> Thanks for your input.
>
> It looks to me that the output class ngram (either just
> -expand-classes or further interpolated with some word ngram) is a
> normal class ngram followed by the probability of a class emitting a
> particular word. Is that just the class ngram appended by the class
> definition file?
Correct.

>
> If I need to convert the class-ngram or the interpolated LM into a
> pure word ngram model for use elsewhere, shall I replace for example
> the lines in the following 2-grams as indicated:
>
> 2-grams:
> pp1(log) class-1 class-2
>>>>>> change to all possible word pairs, such as "pp1+log(p11)+log(p29)
> word-1 word-9" and "pp1+log(p12)+log(p29) word-2 word-9"
>
> pp2(log) class-1 word-5
>>>>> change to "pp2+log(p11) word-1 word-5" and "pp2+log(p12) word-2 
>>>>> word-5"
>
>
> [[class definition:
> class-1 p11 word-1
> class-1 p12 word-2
> class-2 p29 word-9
> ]]
Yes, except you need to take care to sum probabilities of word ngrams 
that are generated through
multiple distinct class expansions.  Also, in SRILM, classes may have 
multi-word strings as members,
further complicating the situation.

The good news is that ngram -expand-classes already does all this for you.
Beware that expanding large or even moderate-sized class ngrams may not 
be feasible computationally depending
on the cardinality of your classes

Andreas