Sequence Classifiers in C#: Hidden Markov Models

sample-5

Few days ago I published a new article on CodeProject, about classifiers based on banks of hidden Markov models to accomplish sequence classification.

While I have written about this subject in the past on this very blog, this time the article deals with Gaussian, continuous-density hidden Markov models rather than discrete ones. Plus, at this time the Accord.NET Framework has evolved much since that 2010 post, and the new article reflects most of the improvements and additions in those last two years.

In the meantime, this article is also serving as a hook to a future article, an article about Hidden Conditional Random Fields (HCRFs). The HCRF models can serve the same purpose as the HMMs but can be generalized to arbitrary graph structures and be trained discriminatively, which could be an advantage on classification tasks.

As always, I hope readers can find it a good read 🙂

15 Comments

  1. I read all posts in this blog and very impressed with it. I’m in love with AI and Machine Learning and also I like C# and .net. It is cool that we have such great libraries for .net!
    If you ever need volunteer for C# coding, contact me, I’ll be happy to participate or just help)

  2. Hi Oleksandr! I am glad you have found the posts interesting! If you would like to help, perhaps you could take a look on the project’s issue tracker. It has some things there which were just enhancement requests, and shouldn’t be much difficult to do, perhaps a bit difficult, but certainly doable. One example is the implementation of HoGs in the framework, which I am always trying to find some free time to complete 🙂

    But this was just a suggestion, of course. Another way to contribute would be simply to spread the word about the project. Perhaps posting some examples on how to use the framework on a blog or really, anything else. This alone would also be a huge help!

    Regards,
    Cesar

    1. Thanks James! I would love to write something as a guest someday 🙂

      I’ve also seen you posted a summary link for machine learning libraries in .NET and included a link to Accord. Many, many thanks!

      All the best!
      Cesar

  3. A question regarding HMMs: having a long time serie, let’s say many hundreds of points, what is best training method: training it over one BIG sequence, or splitting the input into many smaller sequences?

    1. Hi Nino! It depends on what you are trying to do. If the sequences have indeed a certain starting point, such as for example, a gesture (it has to begin somewhere and end somewhere) then it would be better to split it into smaller sequences, where each sequence has its own definite start and ending points, and train with those sequences (most likely with a Forward topology).

      If, however, you are trying to model something which doesn’t have clear starting and ending points, then using one big sequence (with an Ergodic topology) would seem more appropriate!

      Hope it helps! By the way now we have a better discussion group for the project to share more questions and answer with users!

      Cheers!
      Cesar

  4. Hi Cesar,

    Thank you for amazing articles and tutorials about Accord.net.
    I am working on 3D gesture recognition using Kinect and HMM and i want to use Accord.net for that. I am trying to learn but let me know if you can help me out with some tips about how do i start with it.

    Thanks.

  5. Hello and thank you for this incredible library! My interest is predictive sequential string data mining. I would appreciate it greatly if you could point me towards the right direction/algorithms/example applications to use. I can take it from there. I am trying to predict the next a or b customer decision based on past history.

    1. Given a database of say 1000 strings of a or b customer decisions as historical data
    2. Predict the next customer decision a or b

    Rules – Length of a or b decisions can be 1 or more with no upper limit

    Example Predict Next Customer Decisions on Wednesday:
    String1/Monday – A BB AA BBBBB A BBB AA B AAA B AAAAAAAA B
    String2/Tuesday – BBBBB AAAA BBBBBB AA B AA BBB A B
    Wednesday Start of Day – AA BB A BBB ? < What is the next customer decision

    Questions: Since I have a lot of historical data…
    1. Should I identify the strings that most closely resemble the current days customer choices first and use those for the prediction? Say 10 of my 1000 customer choice strings begin with the exact "wednesday" sequence, many of them do.
    2. I am looking for groups of customer decisions, what algorithm/method is best. For example will the next customer choose A or will the next group of customers choose AAA?

    Thank you for your guidance!

  6. Hi César

    I found your HCRF development is great and interesting. I am much familiar with c#.net. However, I would like to modify HCRF to be applicable for online in an activity recognition. Kindly, is that possible to have the source code of HCRF? The provided HCRF is in Accord dll file.

    Thanks. Your kind concern is apprecitaed.
    Ahmad

Leave a Reply

Your email address will not be published. Required fields are marked *