CORSIS

While implementing the perplexity function on Markov n-gram models as it is described on page 14 of Jurafsky & Martin's SLP to appear , I came across some floating point overflow, underflow issues and had to come up with equations to avoid them. Here is my solution in detail and its bigram implementation . It took me a lot of whiteboarding and a few hours to figure this one out but the resulting libcorsis code is %100 foreign intellectual property free ^_^. I am now looking for ways to analyze distributions graphically and testing different encapsulations of probability values and their interaction with the public API.

Here is a very short update on what I have been doing recently. 1) Thanks to the excellent quality of the introduction to computational linguistics course I am taking this term , I get to learn a great many new stuff which boost the development tempo of my library. Here is a particularly interesting case: Markov N-Gram Models . These are soon going to find some interesting application in my experimental HMM-based PoS-Tagger. 2) C# 3.0 and .NET 3.5 have been released. I'm truly astonished with the improved ease of programming this iteration delivers. Waiting eagerly to lay my hands on some quality books on both C# and .NET 3.5. 3) F#! After watching Dr. Brian Beckman's latest video on Channel 9 about monads I decided to learn F#. Parallelization libraries like Parallel FX might provide all that is needed to seamlessly parallelize execution but learning something natively functional seems to be a great asset for the very near future. 4) I had the chance yesterday to attend a ...