[SRILM User List] Lattice tool issue

[Nickolay V. Shmyrev]

Hello

I've found another corner case in lattice-tool processing from SRILM
1.6.0. If weights are not quite accurate (contain -inf or -nan),
lattice-tool -viterbi-decode can crash.

The stacktrace looks like this one

0x000000000044ad84 in cmpPath (p1=0x170bc58, p2=0x170bc68) at
LatticeDecode.cc:169
169	  float diff = (*p1)->m_Prob - (*p2)->m_Prob;
Missing separate debuginfos, use: debuginfo-install
glibc-2.12-1.47.el6_2.9.x86_64 libgcc-4.4.6-3.el6.x86_64 libstdc
++-4.4.6-3.el6.x86_64
(gdb) bt
#0  0x000000000044ad84 in cmpPath (p1=0x170bc58, p2=0x170bc68) at
LatticeDecode.cc:169
#1  0x00000000004be418 in qsort (base=0x170bc60 "H.\313\001", n=6,
size=8, 
    compar=0x44ad6d <cmpPath(LatticeDecodePath const**,
LatticeDecodePath const**)>) at qsort.c:75
#2  0x000000000044bcd4 in Lattice::decode (this=0x7fffffffd4c0,
contextLen=3, lm=0x144c530, 
    finalPosition=214, sortedNodes=0x1ca7040, beamwidth=0, lmscale=9.5,
nbest=0, maxFanIn=0, 
    logP_floor=-inf, maxPaths=0) at LatticeDecode.cc:481
#3  0x000000000044c3dd in Lattice::decode1Best (this=0x7fffffffd4c0,
winfo=0x7ffff0969018, 
    maxWords=50000, ignoreWords=..., lm=0x144c530, contextLen=3,
beamwidth=0, logP_floor=-inf, 
    maxPaths=0) at LatticeDecode.cc:612
#4  0x000000000044bfdc in Lattice::decode1Best (this=0x7fffffffd4c0,
words=0x7ffff7e33010, 
    maxWords=50000, ignoreWords=..., lm=0x144c530, contextLen=3,
beamwidth=0, logP_floor=-inf, 


The problem is in both qsort and in cmpPath function which is used to
compare path scores

This part of qsort assumes that qcmp returns 0 for hi == min, which is
not always true

                while (qcmp(hi -= qsz, min) > 0)
                        /* void */;

This function
// sort in descending order
int
cmpPath(const LatticeDecodePath ** p1, const LatticeDecodePath ** p2)
{
  float diff = (*p1)->m_Prob - (*p2)->m_Prob;
  if (diff > 0)
    return -1;
  else if (diff == 0)
    return 0;
  else
    return 1;
}

Doesn't always return 0 properly if diff is nan or inf.

So I suggest two things:

1. Add additional check in qsort in a while loop for hi >= min
2. Warn about diff being nan or compare it with 0 properly.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://www.speech.sri.com/pipermail/srilm-user/attachments/20120517/16e8df9b/attachment.bin>