Accord.NET Framework: Gesture Controller Components


A new version (2.2.0) of the Accord.NET Framework has just been released. This new version introduces many new features, fixes and improvements. The most interesting additions are certainly the HeadController and FaceController .NET components.


Accord.NET Framework sample application for Gesture Controller Components

The Accord.NET Controller components can be used to generate events based on webcam motion. By using a combination of HaarCascadeClassifiers, Camshift and Template-based Tracking, those components are able to detect when a face enters scene, leaves the scene, and moves across a scene.

The video above shows only the sample application which comes together with the framework. However, the interesting part is that this is just a sample of what can be accomplished using the real controller components. The controller components are .NET components, similar to Button, Label or Timer, and can be dragged and dropped from Visual Studio’s ToolBox directly into any application.

Accord.NET Controller Components in Visual Studio

Once inside an application, it will be possible to set event actions just as in any other .NET component:

Setting events for head movements using Accord.NET Controller Components

The controls have built-in support for calibration. All values except tilting angle are passed to the hosting application in the [-1;+1] range, in which -1 indicates either a total left/down/backwards position and +1 indicates a total right/up/forward position. The tilting angle is given in radians. Please note that the face controller is still a bit experimental and still requires some tuning.


This new version also introduces HSL Color Range object trackers, more default Haar Cascades, an experimental version of linear-chain Conditional Random Fields, and the ability to generate hardcoded C# definitions of any Haar cascade available in the OpenCV XML format. There is also initial support for finger detection using new implementations for Border-Following contour extraction, K-Curvatures and Convex Hull Defects extraction. On the statistics side, there has been the inclusion of the Von-Mises distribution, Moving and Running Normal distributions and improvements in the Multivariate Gaussian implementation. The full release notes are available in the release’s download page.

Also, a special thanks to Professor Dr. Modesto Castrillón Santana to let me embed some of his Haar definitions into the framework under the LGPL license. Please be sure to include a reference to his work if you plan to use this in an academic publication.


As always, I hope those additions and improvements will be useful to everyone 🙂


  1. Hi Antonino! Well, it is nice to see you are still interested in reading this blog after all this time 🙂

    Actually, I believe I must apologize. I haven’t finished writing the post about Gaussian mixture HMMs. One of the reasons is that I am still not completely confident in the implementation. While the discrete version has been exhaustively tested, and while the continuous version has been working well, the combination of a continuous model with mixtures has not been extensively used and explored yet.

    In this recent release I have made some improvements in the Gaussian distributions which should help reduce the number of “Matrix is singular” exceptions (which occurred fairly often when using mixtures). However, today I have found an issue in a cloning operation during mixture initialization. This issue prevents the use of one of the ContinuousSequenceClassifier constructors accepting a single density to be cloned.

    If you wish to take a look, I have posted a very simple example of initializing a continuous hidden Markov model using mixture distributions on pastebin. However, I apologize for any issues it may still have.

    Best regards,

    1. Hello Cesar,

      I’m very thankful for the huge amount of work you’ve done up till now, i’ve a small question, i’m trying to use gaussian mixture HMMs, i’ve successfully used the continuous normal distribution and it works great, and i’m currently trying to use the gaussian mixture HMMs, i’m stuck at how could i initialize the HMMclassifier and HMMclassifierlearning.

      the code sample link you mentioned is broken, so if there’s a simple code that can explain that would be great

  2. Thanks Antonino, and sorry for taking this much time to get it ready. Also, the example as it is shown will not work right away. The constructor I’ve used in the example has the cloning issue I mentioned. To overcome this issue, you can create a mixture for each of the states in your model and pass it to the constructor as an array.

    And if you really want to experiment, I would strongly advise to try first with Normal density models (no mixtures) first. I couldn’t find yet an example in which mixture models worked better than normal Gaussian distributions (but this may be because I couldn’t think of a problem complicated enough).

    Best regards,

  3. Hi CC,

    I decided to retire the repository and downloads until I could get more licensing info from the Marilena and FaceIt projects, as well as the Cephes Math Library, from which portions of the project were based. The Marilena and FaceIt are distributed under different (but compatible) open-source licenses and this should be made clearer. The Cephes Math Library licensing scheme, however, is not so clear, albeit it seems to have been used in many open-source projects. The project will be restored when everything is resolved.


  4. Hi Cèsar,

    thanks for the quick response and I’m glad to hear that the project should be back soon. Looking forward to test the new 2.2.0 framework as I didn’t have the opportunity to download it yet.

    Best wishes,

  5. I am very curious to read more about Gesture Control. Information present on the blog, really looks very nice and Written in good manner. Nice work, Keep sharing. I Look forward to read more,

Leave a Reply

Your email address will not be published. Required fields are marked *