Sequence Classifiers in C#: Hidden Conditional Random Fields

After a preliminary article on hidden Markov models, some months ago I had finally posted the article on Hidden Conditional Random Fields (HCRF) on CodeProject. The HCRF is a discriminative model, forming the generative-discriminative pair with the hidden Markov model classifers.

This CodeProject article is a second on a series of articles about sequence classification, the first being about Hidden Markov Models. I’ve used this opportunity to write a little about generative versus discriminative models, and also provide a brief discussion on how Vapnik’s ideas apply to these learning paradigms.

All the code available on those articles are also available within the Accord.NET Framework. Those articles provide good examples on how to use the framework and can be regarded as a practical implementation on how to use those models with the framework.

Complete framework documentation can be found live at the project’s website, as well as in the framework’s GitHub wiki. The framework has now been referred on 30+ publications over the years, and several more are already in the works, by me and users around the world.

Academical publications

Talking about publications, the framework has been used within my own research on Computer Vision. If you need help in understanding the inner workings of the HCRF, a more visual explanation on the HCRF derivation can also be found at the presentation I gave on Iberamia 2012 about Fingerspelling Recognition with Support Vector Machines and Hidden Conditional Random Fields.

An application to a more interesting problem, namely natural words drawn from Sign Languages using a Microsoft Kinect, has also been accepted for publication at the 9th International Conference on Machine Learning and Data Mining, MLDM 2013, and will be available soon. Update: it is available at

As usual, hope you find it interesting!

5 Comments

  1. Hello Cesar,
    I am trying to play with your HCRF implementation. I use multivariate HMM to initialize HCRF and when I want to run HCRF teacher object, independently from selected training method, I get exception that is basically saying that I have to use positive real numbers in my training data. Is there limitation like that in HCRF classes?
    Regards, Stanko

  2. Hi Stanko!

    Yeah, indeed, there is such limitation! But actually, it can be circunverted by adding a positive constant to your continuous data so it becomes always positive. So, if you know beforehand your values are always between a range of values, I would suggest normalizing the values to be always between 0 and 1 before feeding them to the HRCF.

    Hope it helps!

    Best regards,
    Cesar

  3. Hi cesar,

    Congrats four your great job in Accord.net. Its been unvaluable for me so far. Now I am in the need of an LVHMM, variable order HMM implementation for sequence prediction.

    In your opinion, would it be easy to do it using Accord.net? What would you change/reuse of your set of classes?

    Thanks in advance

  4. Hi there,

    I am glad you have been finding the framework useful! For the moment, Accord.NET doesn’t offers LVHMMs, but this is something that could be incorporated in the future. I also don’t have much experience with LVHMMs, so I can’t answer with certainty how much time and effort it would consume.

    Thus said, may I ask if you could please point me to the first publication about the topic that introduced the LVHMM model? Those first publications are invaluable to get a proper comprehension on how the model works and how it could be implemented!

    Regards,
    Cesar

  5. Cesar,

    Since I am not an expert on the field, I don’t know if I can help you on that. Very little code/literature is available on high-order HMMs, and what is available, concerned specifics models like second-order. The paper I plan to follow can be found in http://www.joics.com/publishedpapers/2011_8_10_1761_1777.pdf, which is not the fundational publication, though it has some references 1,2,3 that could be helpful to find it.

    I’ve also found “Efficient High-Order Hidden Markov Modelling”, 1997,http://www.ussigbase.org/downloads/jadp_phd.pdf. This paper aimed to find some generalization while bearing in mind a practical implementation.

    Hope this is useful!

Leave a Reply

Your email address will not be published. Required fields are marked *