<html>
  <head>
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div class="moz-cite-prefix">Eeva,<br>
      <br>
      I think this bug was fixed in 2015, but is not in the beta version
      on the download server.  I need to work with people at SRI to get
      it updated.  Sorry about that.  <br>
      <br>
      Below  is the patch to lm/src/segment.cc.<br>
      <br>
      Andreas<br>
      <br>
      ***************<br>
      *** 74,80 ****<br>
        }<br>
      <br>
        inline Boolean<br>
      ! Map_noKeyP(SegmentState &state)<br>
        {<br>
            return state == NOSTATE;<br>
        }<br>
      --- 74,80 ----<br>
        }<br>
      <br>
        inline Boolean<br>
      ! Map_noKeyP(const SegmentState &state)<br>
        {<br>
            return state == NOSTATE;<br>
        }<br>
      <br>
      <br>
      <br>
      <br>
      On 11/4/2016 7:54 AM, Eeva Nikkari wrote:<br>
    </div>
    <blockquote
cite="mid:CABHg-goxa+d5qFGVSCDJ=htXf+G2HKcfi-CaaJBTMuf_7KKWJw@mail.gmail.com"
      type="cite">
      <div dir="ltr">
        <div>Hi,</div>
        <div><br>
        </div>
        I installed the SRILM 1.7.1 version in a linux environment and I
        get the same error with the segment function.
        <div><br>
        </div>
        <div>However, I managed to get results on both mac and linux
          using the hidden-ngram function and hidden vocabulary with
          words <s> and </s></div>
        <div><br>
        </div>
        <div>
          <p
style="margin:0px;font-size:11px;line-height:normal;font-family:menlo;color:rgb(0,0,0)"><span
              style="font-variant-ligatures:no-common-ligatures">$
              hidden-ngram -lm minicorpus.lm -text minicorpus.txt
              -hidden-vocab hidden_vocab -continuous</span></p>
          <p
style="margin:0px;font-size:11px;line-height:normal;font-family:menlo;color:rgb(0,0,0)"><span
              style="font-variant-ligatures:no-common-ligatures">dogs
              chase cats </s> dogs bark </s> cats meow
              </s> dogs chase birds </s> cats chase birds
              </s> dogs chase the cats </s> the birds chirp</span></p>
          <p
style="margin:0px;font-size:11px;line-height:normal;font-family:menlo;color:rgb(0,0,0)"><span
              style="font-variant-ligatures:no-common-ligatures"><br>
            </span></p>
          <p
style="margin:0px;font-size:11px;line-height:normal;font-family:menlo;color:rgb(0,0,0)"><span
              style="font-variant-ligatures:no-common-ligatures">Thanks,</span></p>
          <p
style="margin:0px;font-size:11px;line-height:normal;font-family:menlo;color:rgb(0,0,0)"><span
              style="font-variant-ligatures:no-common-ligatures">Eeva</span></p>
        </div>
      </div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">2016-11-03 14:39 GMT+02:00 Eeva Nikkari
          <span dir="ltr"><<a moz-do-not-send="true"
              href="mailto:eevanikkari@gmail.com" target="_blank">eevanikkari@gmail.com</a>></span>:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div dir="ltr">
              <div>Thank you for the quick answer, </div>
              <div><br>
              </div>
              <div>I was using version 1.7.1, but updated now to
                1.7.2-beta. The problem persists.</div>
              <div><br>
              </div>
              <div>$ segment -version</div>
              <div>SRILM release 1.7.2-beta (with third-party
                contributions)</div>
              <div><br>
              </div>
              <div>Program version @(#)$Id: segment.cc,v 1.21 2014-08-29
                21:35:48 frandsen Exp $</div>
              <div><br>
              </div>
              <div>Support for compressed files is included.</div>
              <div>Using libLBFGS.</div>
              <div><br>
              </div>
              <div>This software is subject to the SRILM Community
                Research License Version</div>
              <div>1.0 (the "License"); you may not use this software
                except in compliance</div>
              <div>with the License.  A copy of the License is included
                in the SRILM root</div>
              <div>directory in the "License" file.  Software
                distributed under the License</div>
              <div>is distributed on an "AS IS" basis, WITHOUT WARRANTY
                OF ANY KIND, either</div>
              <div>express or implied.  See the License for the specific
                language governing</div>
              <div>rights and limitations under the License.</div>
              <div><br>
              </div>
              <div>This software is Copyright (c) 1995-2014 SRI
                International.  All rights</div>
              <div>reserved.</div>
              <div><br>
              </div>
              <div>Portions of this software are</div>
              <div>Copyright (c) 2002-2005 Jeff Bilmes</div>
              <div>Copyright (c) 2009-2013 Tanel Alumae</div>
              <div>Copyright (c) 2012-2013 Microsoft Corp.</div>
              <div><br>
              </div>
              <div>SRILM also includes open-source software as listed in
                the</div>
              <div>ACKNOWLEDGEMENTS file in the SRILM root directory.</div>
              <div><br>
              </div>
              <div>If this software was obtained under a commercial
                license agreement with</div>
              <div>SRI then the provisions therein govern the use of the
                software and the</div>
              <div>above notice does not apply.</div>
              <div><br>
              </div>
              <div><br>
              </div>
              <div><br>
              </div>
              <div>I still get the same error </div>
              <span class="">
                <div><br>
                </div>
                <div>$ segment -order 2 -lm minicorpus.lm -text
                  minicorpus.txt -continuous -debug 5</div>
                <div>reading 10 1-grams</div>
                <div>reading 18 2-grams</div>
                <div>warning: p(w1) < p(<s> w1)) </div>
                <div>0: p(NOS) = 0, P(S) = 0.148148</div>
                <div>1: p(NOS) = 0.111111, P(S) = 0</div>
                <div>2: p(NOS) = 0.0277778, P(S) = 6.10653e-10</div>
                <div>3: p(NOS) = 3.66393e-10, P(S) = 0.00793651</div>
                <div>4: p(NOS) = 0.00198413, P(S) = 0</div>
                <div>5: p(NOS) = 0, P(S) = 0.000566893</div>
                <div>6: p(NOS) = 0.000141723, P(S) = 0</div>
                <div>7: p(NOS) = 0, P(S) = 8.09848e-05</div>
                <div>8: p(NOS) = 6.07386e-05, P(S) = 0</div>
                <div>9: p(NOS) = 3.03693e-05, P(S) = 0</div>
                <div>10: p(NOS) = 0, P(S) = 5.78463e-06</div>
                <div>11: p(NOS) = 1.44616e-06, P(S) = 0</div>
                <div>12: p(NOS) = 7.23079e-07, P(S) = 0</div>
                <div>13: p(NOS) = 0, P(S) = 2.75459e-07</div>
                <div>14: p(NOS) = 2.06594e-07, P(S) = 0</div>
                <div>15: p(NOS) = 5.16485e-08, P(S) = 5.67708e-16</div>
                <div>16: p(NOS) = 2.58243e-08, P(S) = 1.70313e-16</div>
                <div>17: p(NOS) = 1.70313e-16, P(S) = 1.84459e-09</div>
                <div>18: p(NOS) = 9.22294e-10, P(S) = 0</div>
                <div>19: p(NOS) = 3.07431e-10, P(S) = 0</div>
              </span>
              <div>Assertion failed: (!Map_noKeyP(key)), function
                locate, file /Users/eevanikkari/src/srilm//<wbr>include/LHash.cc,
                line 275.</div>
              <div>Abort trap: 6</div>
              <div><br>
              </div>
              <div><br>
              </div>
              <div><br>
              </div>
              <div>Before updating the version was</div>
              <div><br>
              </div>
              <div><br>
              </div>
              <div>
                <div>$ segment -version</div>
                <div>SRILM release 1.7.1 (with third-party
                  contributions)</div>
                <div><br>
                </div>
                <div>Program version @(#)$Id: segment.cc,v 1.20
                  2012/05/17 06:46:46 stolcke Exp $</div>
                <div><br>
                </div>
                <div>Support for compressed files is included.</div>
                <div>Using libLBFGS.</div>
              </div>
              <div><br>
              </div>
              <div>...</div>
              <div><br>
              </div>
              <div><br>
              </div>
              <div><br>
              </div>
              <div>Thank you,</div>
              <div>Eeva</div>
              <div><span
                  style="font-variant-ligatures:no-common-ligatures"><br>
                </span></div>
            </div>
            <div class="HOEnZb">
              <div class="h5">
                <div class="gmail_extra"><br>
                  <div class="gmail_quote">2016-11-02 20:42 GMT+02:00
                    Andreas Stolcke <span dir="ltr"><<a
                        moz-do-not-send="true"
                        href="mailto:stolcke@icsi.berkeley.edu"
                        target="_blank">stolcke@icsi.berkeley.edu</a>></span>:<br>
                    <blockquote class="gmail_quote" style="margin:0 0 0
                      .8ex;border-left:1px #ccc solid;padding-left:1ex">
                      <div bgcolor="#FFFFFF" text="#000000"><span>
                          <div
                            class="m_-7879180930401595m_-1923685805697030223moz-cite-prefix">On
                            11/2/2016 6:22 AM, Eeva Nikkari wrote:<br>
                          </div>
                          <blockquote type="cite">
                            <div>I get the </div>
                            <div>
                              <p
style="margin:0px;font-size:11px;line-height:normal;font-family:menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">Assertion failed:
                                  (!Map_noKeyP(key)), function locate,
                                  file ../../include/LHash.cc, line 275.</span></p>
                              <p
style="margin:0px;font-size:11px;line-height:normal;font-family:menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">Abort trap: 6</span></p>
                              <p
style="margin:0px;font-size:11px;line-height:normal;font-family:menlo;color:rgb(0,0,0)"><span
style="font-variant-ligatures:no-common-ligatures">-error every time I
                                  use the segment function. I've tried
                                  with different texts and language
                                  models (different orders, smoothing
                                  and corpora). Is my model missing
                                  something? The man page says to use "</span><span
style="font-family:-webkit-standard;font-size:medium">standard backoff
                                  N-gram model in ARPA </span><a
                                  moz-do-not-send="true"
href="http://www.speech.sri.com/projects/srilm/manpages/ngram-format.5.html"
                                  style="font-family:-webkit-standard"
                                  target="_blank">ngram-format(5)</a><span
style="font-family:-webkit-standard;font-size:medium">, modeling
                                  segmentation using the boundary tags
                                  <s> and </s></span>",
                                which to my understanding minicorpus.lm
                                is. I use macOS Sierra Version 10.12.1  </p>
                            </div>
                          </blockquote>
                        </span>
                        <p>Eeva,</p>
                        <p>I cannot reproduce your error.  It could be a
                          bug that's fixed in the latest version.  
                          What's the output when you run "segment
                          -version"?</p>
                        <span class="m_-7879180930401595HOEnZb"><font
                            color="#888888">
                            <p>Andreas</p>
                            <p><br>
                            </p>
                          </font></span></div>
                    </blockquote>
                  </div>
                  <br>
                </div>
              </div>
            </div>
          </blockquote>
        </div>
      </div>
    </blockquote>
  </body>
</html>