[SRILM User List] lattice-tool related issues

[Andreas Stolcke]

Anoop Deoras wrote:
> Hello Andreas,
>
>   My message got bounced from the mailing list. I am hence sending the 
> email directly to you.
Probably due to the problems with the mailing list software that are now 
solved.
>
> Thanks and Regards
> Anoop
>
> Begin forwarded message:
>
>> *From: *Anoop Deoras <adeoras at jhu.edu <mailto:adeoras at jhu.edu>>
>> *Date: *July 10, 2010 2:51:07 PM EDT
>> *To: *srilm-user at speech.sri.com <mailto:srilm-user at speech.sri.com>
>> *Cc: *Anoop Deoras <adeoras at jhu.edu <mailto:adeoras at jhu.edu>>
>> *Subject: **lattice-tool related issues*
>>
>> Hello Andreas,
>>
>>  I need to generate N best list from an HTK format lattice and 
>> unfortunately I am not able to
>> suppress the default operation of deleting the duplicates.
>>
>> I give following command:
>>
>> lattice-tool -read-htk -in-lattice test.lat -nbest-decode 10 
>> -nbest-duplicates 10 -out-nbest-dir my_nbest_dir
>>
>> I have the following test lattice:
>> test.lat
>> **************************************************
>> VERSION=1.0
>> UTTERANCE=test.mfc
>> lmname=test.bg
>> lmscale=16.00  wdpenalty=0.00
>> acscale=1.00
>> vocab=test
>> N=439  L=1108
>> I=0    t=0.00  W=!NULL
>> I=1    t=0.02  W=b                   v=1
>> I=2    t=0.08  W=b                   v=1
>> I=3    t=0.14  W=c                   v=1
>> J=0     S=0    E=1    a=0.00      l=0.000
>> J=1     S=1    E=3    a=-382.52   l=-3.730
>> J=2     S=0    E=2    a=-669.26   l=-3.730
>> J=3     S=2    E=3    a=0.00      l=0.00
>>
>> **************************************************
>>
>> I get following nbest hypotheses:
>> $: less my_nbest_dir/test.mfc.gz
>>
>> -166.126 -1.61992 2 <s> b c </s>
>>
>> (the scores get divided by natural log of 10.)
>> ******************
>> The nbest file contain just one hypothesis instead of two. The 
>> lattice-tool has deleted
>> duplicate hypothesis. Inspite of specifying -nbest-duplicate option, 
>> I dont see the duplicates.
>>
>> To check if deletion of duplicates is the only issue, if we replace 
>> word at node I=2 by say 'd', then
>> we do get 2 hypotheses i.e <s> b c </s> AND <s> d c </s>
>>
>> Am I missing any specific flag required to get duplicates in N Best 
>> lists ?
The -nbest-duplicates option is no longer supported by the new nbest 
implementation that performs LM rescoring on the fly (as of SRILM 
1.5.7).   However, it still works if you enable use of the "old" 
decoding method. So use

lattice-tools -old-decoding -nbest-duplicates 10 ...

Andreas



>>
>> Thanks and Regards
>> Anoop
>