Rescoring query

[Andreas Stolcke]

In message <Pine.LNX.4.44.0501041823120.32198-100000 at mail-speech.cs.iitm.ernet.
in>you wrote:
> Sir,
> 
> I have a language model (long span LM) that gives a sequence of 
> probabilites for each word in a test  sentence. I cannot write this 
> language model in arpa format. The query I have is, can I integrate these 
> probabilites into a lattice or N-best lists for rescoring.
> 
> In detail the problem is, for a test sentence/utterance I can get a 
> lattice or N-best list  generated using HTK.  For the same sentence 
> (assuming I have the transcription) I can get the probabilities for each 
> word in the sentence using my long span LM. How can I integrate/rescore 
> the lattice or N-best list using the tools in the SRI-LM toolkit. What 
> tools and options should I use.

The following approaches use the SRILM tools at a high level to
integrate LM scores that you compute externally.

For N-best lists:

1. Generate N-best lists using HTK, then convert them into the 3rd format
   described in nbest-format(5).  

2. Generate your own LM scores by whatever means, and store them into
   a separate directory.   For example, if one of your waveform names is
   abcde.wav, then format the corresponding N-best LM scores into a single
   column of numbers and put them in a file called DIR/abcde or
   DIR/abcde.gz (compressed).

3. Use the rescore-reweight script (see nbest-scripts(1) man page to
   combine the standard scores and your own and extract the 1-best hypotheses.

4. You can use the nbest-optimize(1) tool to tune the score combination
   weights on a held-out set.

For lattices:

1. Generate lattices using HTK.

2. Generate your own LM scores by whatever means and insert them into the 
   lattices.  You can either replace the original LM scores (by modifying
   the "l=" fields), or add them as a separate set of scores.
   In SRILM you can add the "x1=", "x2=", etc. fields to add arbitrary
   additional scores to lattice nodes or links.
   Note this assumes you can somehow compute LM scores on a word-by-word
   basis.  This might not be simple, especially if your LM is "long-span",
   and might require expanding the lattice etc.

3. Use lattice-tool(1) to combine the old and new scores in a weighted
   fashion and extract the 1-best hypotheses.

The other possibility is to implement your LM in C++ as a LM class in
the SRILM framework.  This is a fair amount of work, would require some
study of the existing code, etc., but would ultimately allow you to use
your LM seamlessly in all the SRILM tools, for perplexity computation,
nbest rescoring, lattice expansion, etc.  (I'm assuming you probably do
NOT want to attempt this for now.)

> Kindly help me in this regard. Any suggestions is welcome.
> 
> Any pointers to important papers on rescoring is also requested.

The classic paper on rescoring is 

M. Ostendorf, A. Kannan, S. Austin, O. Kimball, R. Schwartz, J. R. Rohlicek,
Integration of diverse recognition methodologies through reevaluation of N-best sentence hypotheses, Proceedings DARPA workshop on Speech and natural language,
Pacific Grove, California, pp. 83-87,
Morgan Kaufmann Publishers Inc., San Francisco, CA, 1991.

The framework has been extremely popular since, and there have probably been
probably hundreds of other papers since.

--Andreas