class-SLM

[Andreas Stolcke]

Hongqin,

there is no guarantee that a class-based LM will have lower perplexity
than a word-based one.  For small, task-oriented domains with little
training data (think ATIS), you can usually get a good improvement
with hand-defined word classes that reflect the properties of 
the domain.  For large-vocabulary, unconstrainted domains (such as
Switchboard or Broadcast News), a class-based LM by itself will usually 
have higher perplexity.  However, you can usually get a nice 
perplexity reduction by interpolating the word and the class-based LMs.
Mostly, the class-based LM helps with the prediction of unseen word ngrams.

It is pure laziness that the make-ngram-pfsg script cannot handle 
4-gram and higher-order LMs at this point.  It shouldn't be hard to 
do.  If anybody wants to contribute a generalized version I'd be happy 
to incorporate it.

--Andreas

In message <3C911CC9.C47D16B1 at inzigo.com>you wrote:
> Hi,
> 
> I am trying to construct a class based trigram LM. The function
> "ngram-class" only induces classes for a bigram model. I have my own
> class definitions with the class-format. When I use these definition
> with the "ngram" function (-classes option), the LM leads to a higher
> perplexity and word error rate than those from a word based trigram. Is
> there any other approach with which I can get a class-based LM with
> lower perplexity the a word-based?
> 
> By the way, anyone tried a 4gram model with pfsg format?
> 
> Thanks!
> 
> Hongqin Liu
> 
>