Frequency Lists - Statistics View



^_^ Here is the new statistics view for frequency lists from the 2007-02-05 pre-alpha release.

I'm still actively working on frequency lists at GUI and lower levels and some of the following features might be ready for the next release:
  1. An option to have the program display in the statistics view for frequency lists either
    • the count of tokens that are a certain number of characters long or
    • the count of types that are a certain number of characters long. *

    Tenka Text 2007-02-05 and WordSmith Tools 4 can only display the first but not the latter. There is only a little GUI work to deliver this option.

  2. Live updates.

    Once in place, users will be able to add or remove files from their frequency lists without the need to recalculate everything from the start.
Well, that's all the news about Tenka Text 2007-02-05. Let's move on to more personal stuff.

Logographic Failure

I found out that the "Tenka" Logo 「天花」, which was my personal attempt at creating a digital seal with the kanji of a literary word for "snow", has a huge flaw! A senpai from university was kind enough to point to the fact that the kanji combination I chose (「天花」) for representing the sound of "tenka" actually meant "smallpox" - a disease. She is now cooperating with me, trying to come up with a new logo.

Institut für Deutsche Sprache

I have been working at IDS since 2006-10-16 as an assistant student in corpus compilation and analysis and have come to accumulate some experience which I am going to consider putting into Tenka Text at some future date. By the way, I owe my current position to the Mono project team. It is their commitment to making .NET multi-platform that I am now able to write tools in C# that run on Linux, which is for some reason the preferred platform of the department of lexical analysis at IDS.

Structured Plain Text Extraction from PDFs and Other Office Files

To accomplish a task I was assigned for example, I had to put together several builds of a tool called 'pdftohtml' and a few lines of C# (2.0 with some Linq to Xml) and there I had a multi-platform, open-source and managed pdf import method. This method will definitely find its way into Tenka Text once I have developed a common API for extracting structured plain text from file formats such as PDFs or office files.

Comments

Post a Comment

Popular posts from this blog

Levenshtein Distance Algorithm: Fastest Implementation in C#

Image
Here is a cleaned-up performance test for several different implementations of levenshtein I have blogged about recently. This test was emailed to me by Ahmed Ghoneim, who has also kindly agreed to its publication on my blog. I am very grateful to him for his excellent contribution. I have slightly altered his file to do away with the unnecessary local variables in my C2C# port of the GNULevenshtein method. I would like to hear from you which methods perform best on your machine. Please drop a comment ^_^! LevenshteinAlgorithmPerformanceTest.cs code only Packages code, data and sample binary in zip and self-executable zip formats Please note that the GNULevenshtein method was found to be buggy! Here is the new replacement method .
Read more

Mono 1.2.5 binaries for Solaris 10/x86

I think I've finally managed to botch it all up. I created a JRE-style zipped package, which you can download here 1,2 at your own risk ;). Whoa, it's taken me 4 days just to get so far with mono/solaris. ^o^ I wanna sleep like the relakkuma now hehe... These binaries are compiled for 32-bit/x86 processors. They also run on 64-bit/x86-64 processors. They are not compiled for running winforms applications. To extract the archive use: "gtar -zxvf"
Read more

WordSmith Tools 5.0, Tenka Text in China

Mike Scott has finally released a public beta of WordSmith Tools 5.0 . I want to quote him on a few points: System Requirements WordSmith Tools 5 is for Windows 2000 or later. It will be happiest on a fairly modern PC (e.g bought in the last 4 years) with plenty of memory and hard disk space. Or an Intel Mac running Windows . Now that's an interesting world view we have here. What's an Intel Mac anyway? It's a PC produced by Apple. So this sentence actually reads: System Requirements: a PC running Windows or a PC (produced by Apple) running Windows. Technically deep Mr. Scott seems to be. -__-" Here is my favourite one: What's new in version 5 WordSmith is organic software ! Version 4 was a complete new re-write. Since it was launched in 2004, numerous re-compilations of WS4 were issued (about one a week on average), sometimes with a very small bug-fix but other times incorporating changes users had suggested. At the same time the Help was updated. Version 5.0 was...
Read more