read/write counts in FLMs
shachi at streamsage.com
Wed Jun 8 06:29:56 PDT 2005
I am trying to build a factored language model(FLM) using "fngram-count"
in SRILM toolkit.
When I run it using "-write-counts" and "-lm" options together, it
builds the FLM correctly. But when I try to break it down into two
(a) only "-write-counts" option to write the counts file
(b) "-read-counts" and "-lm" options to build the FLM using the counts
it gives errors. I checked the debug output; it seems it is getting the
count-of-counts for modified Kneser-Ney discounting wrong in the step
(b) above. The counts file generated in step (a) is exactly similar to
the one generated using both "-write-counts" and "-lm" options together.
I tried these steps using a couple of different FLM specifications and
the error is the same. Has anyone faced this problem before? I will
appreciate if you can help me out here.
More information about the SRILM-User