Posts

Showing posts with the label finite state automaton

New Segmenter Compiler: Benihime, 紅姫

Image
A quick update on API refactorings! Here is a snapshot of what the code examined in the last post would look like with refactorings and improvements I have made so far: Before: the code examined in the last post After: For segmenting streams into, say, words for example, one could also use something like SED on GNU/Linux, some regular expressions implementation of a programming language or whatever. So why am I such an otaku ? Why not just go with the given the naive and easy way? Well, I am a performance and control freak and CIL is great fun and I feel 'pleasure' writing assembly code for a VM but most importantly, Benihime, 紅姫 makes the perfect training ground for learning language and compiler design. Prior to getting into deep hack mode on Benihime, 紅姫, I had no idea about the differences between 'expressions', 'statements', 'branches' or 'stacks'. Implementing complex boolean expressions in conditional statements like if((c && pm) ...

New Segmenter Compiler: Benihime, 紅姫

Image
An interesting design concern has brought the development of my new segmenter compiler to a temporary standstill for tonight: parallelization . I was trying to refactor and improve the design of my new segmenter compiler Benihime, 紅姫. (It is named after Urahara's sword from the Japanese anime Bleach . Benihime means crimson princess , what is more suitable to call a state-of-the-art segmenter. ^o^ theheee~~) One of my major concerns with the new implementation was decoupling the flow control logic from the segmenter builder and the flow direction. This is essential for being able to reuse the same logic to compile two segmenters that run in opposite directions for example. Let me illustrate the problem with the help of a file I happened to submit as a practice for our introduction to computational linguistics course just 5 days ago: // /home/sert/Projects/hw1/hw1/Main.cs created with MonoDevelop // // project created on 10/21/2007 at 3:19 AM using System; using System.IO; using...