Harris Corners Detector in C#

Corner detection, or the more general terminology interest point detection, is an approach used within computer vision systems to extract certain kinds of features from an image. Corner detection is frequently used in motion detection, image matching, tracking, image mosaicing, panorama stitching, 3D modelling and object recognition.

This code has also been incorporated in Accord.NET Framework, which includes the latest version of this code plus many other statistics and machine learning tools. In order to install the Accord.NET Framework in your projects, use NuGet. Type in the package manager: Install-Package Accord.Imaging

Introduction

One of the firsts operators for interest point detection was developed by Hans P. Moravec in 1977 for his research involving the automatic navigation of a robot through a clustered environment. It was also Moravec who defined the concept of “points of interest” in a image and concluded these interest points could be used to find matching regions in different images.

The Moravec operator is considered to be a corner detector because it defines interest points as points where there are large intensity variations in all directions. This often is the case at corners. It is interesting to note, however, that Moravec was not specifically interested in finding corners, just distinct regions in an image that could be used to register consecutive image frames.

The Harris Operator

Harris Corners detected on the famous Lenna image.

This operator was developed by Chris Harris and Mike Stephens in 1988 as a processing step to build interpretations of a robot’s environment based on image sequences. Like Moravec, they needed a method to match corresponding points in consecutive image frames, but were interested in tracking both corners and edges between frames.

Harris and Stephens improved upon Moravec’s corner detector by considering the differential of the corner score with respect to direction directly. The Harris corner detector computes the locally averaged moment matrix computed from the image gradients, and then combines the Eigenvalues of the moment matrix to compute a corner measure, from which maximum values indicate corners positions.

Source Code

Below is the source code for the Harris Corners Detector algorithm. This is mostly the same code implemented in the Accord.NET Framework.

Using the code

Code usage is very simple. A Corners Marker filter from the AForge Framework can be used to directly draw the detected interest points in the original image as it would be usual within the framework.

Sample application

The accompanying sample application is pretty much self-explanatory. It performs the corners detection in the famous Lena Söderberg‘s picture, but can be adapted to work with other pictures as well. Just add another image to the project settings’ resources and change the line of code which loads the bitmap.

It is also very straightforward to adapt the application to load arbitrary images from the hard disk. I have opted to leave it this way for simplicity.

References

Disclaimer

Before using any information, applications or source code available in this article, please be sure to have read the site usage disclaimer.

31 Comments

  1. Hi,

    I have just updated the downloadable sources in this article with a more recent version. If you are still having trouble, send me an email with the images and I’ll take a look.

    By the way, the latest version of the Harris Corner Detector will always be first available inside Accord.NET Framework.

    Regards,
    César

  2. hi,

    thanks for sharing this tutorial…

    is it possible to get the 3D coordinates fo each point?
    I mean, i want to obtain the point clouds from captured object…

    please help me, how i can do it. i really need it for my final project…

    many thanks before…

  3. I had written a java program for harris corner detection, but my output is not so accurate. I have tried the above program and i’m getting the correct output. However, what does this statement means

    if (measure==HarrisCornerMeasure.Harris)

  4. Hi Pushpender,

    This statement is only comparing if the chosen corners measure is indeed the original Harris measure. There are other ways to compute the corners measure which do not use the arbitrary factor k. HarrisCornersMeasure is just an enumeration.

    I hope I have answered your question.

    Best regards,
    César

  5. Hi Pushpender,

    Well, not exactly. If you set the Measure property of an instance of the HarrisCornersDetector class to be HarrisCornerMeasure.Nobel, then the algorithm will ignore the value of k and use Nobel’s measure instead.

    i.e.:

    // create corners detector’s instance
    HarrisCornersDetector hcd = new HarrisCornersDetector();
    hcd.Measure = HarrisCornerMeasure.Nobel;

    // process image searching for corners
    List<IntPoint&gl; corners = hcd.ProcessImage( image );

    Regards,
    César

  6. Hello Cesar

    Thanx again. One more question: what is the use dfference between original harris and harris nobel method.? Is it only the k-value. I’m using java to implement the corner detection. So pls tell me how do I set which method to use and when??

  7. The only difference is that it does not requires the tuning of the parameter k. It makes the usage of the Harris operator simpler, since you won’t have to care about choosing a suitable value for k in your application. For more details, please see Alison Noble, “Descriptions of Image Surfaces”, PhD thesis, Department of Engineering Science, Oxford University 1989, p45.

    By the way, if you are basing your implementation on my code, I would kindly ask you to add a reference in your sources about this page and about the Accord.NET Framework (http://accord-net.origo.ethz.ch/).

    Best regards,
    César

  8. I want to put those inputs like k, threshold and sigma in your Automatic Image Stitching but why is it when I press the Detect button and the sigma’s value sets in its max value is which 5.000 or I set it to min value which is 0.000 there’s points being detected that supposed to be none. I find it hard to refactor in this part this on Harris Corners Detector not AIS:

    float[,] diffx = new float[height, width];
    float[,] diffy = new float[height, width];
    float[,] diffxy = new float[height, width];

    fixed (float* pdx = diffx, pdy = diffy, pdxy = diffxy)
    {
    byte* src = (byte*)grayImage.ImageData.ToPointer() + srcStride + 1;

    // Skip first row and first column
    float* dx = pdx + width + 1;
    float* dy = pdy + width + 1;
    float* dxy = pdxy + width + 1;

    // for each line
    for (int y = 1; y < height – 1; y++)
    {
    // for each pixel
    for (int x = 1; x < width – 1; x++, src++, dx++, dy++, dxy++)
    {
    // Convolution with horizontal differentiation kernel mask
    float h = ((src[-srcStride + 1] + src[+1] + src[srcStride + 1]) –
    (src[-srcStride – 1] + src[-1] + src[srcStride – 1])) * 0.166666667f;

    // Convolution vertical differentiation kernel mask
    float v = ((src[+srcStride – 1] + src[+srcStride] + src[+srcStride + 1]) –
    (src[-srcStride – 1] + src[-srcStride] + src[-srcStride + 1])) * 0.166666667f;

    // Store squared differences directly
    *dx = h * h;
    *dy = v * v;
    *dxy = h * v;
    }

    // Skip last column
    dx++; dy++; dxy++;
    src += srcOffset + 1;
    }

    // Free some resources which wont be needed anymore
    if (image.PixelFormat != PixelFormat.Format8bppIndexed)
    grayImage.Dispose();
    }

    // 2. Smooth the diff images
    if (sigma > 0.0)
    {
    float[,] temp = new float[height, width];

    // Convolve with Gaussian kernel
    convolve(diffx, temp, kernel);
    convolve(diffy, temp, kernel);
    convolve(diffxy, temp, kernel);
    }

    Where exactly the part I should alter so that sigma will work what I want to. Should I also include the convolve method

  9. Hi César,
    Sorry for the typos (*there are points). Wanna ask again do I have to include the convolve method in HarrisCornersDetector in Automatic Image Stitching. Convert the float [,] types to UnmanagedImage and change those fixed values to unsafe. Just really got problem with sigma alone. The k and threshold works well.

  10. hello;
    I use the java version of this implementation and I want to recover the points detect a new vector to use it later if anyone has any idea how to recover the point values ​​in a new vector

  11. hi Cesar
    i make download to the source code of harris in c# ,and when i start to excute the code i found some error in the program and i cannnot solve it can u help me plzz to solve these error .. if u can tell me, then i will send u the error
    thanx

  12. hi..
    how can i solve this erro ?
    {accord-imaging-harris-srcbinx86debugHarris.exe is missing .please build the project and retry , or set the outputPath and assembly name properties appropriately to point at the correct location for the target assembly.} plz help me … thanx

  13. hi cesar
    how can i change the color of point in harris corner detector to anther color like red or yellow in visual studio (c# , accord.net)

  14. Hi,

    You can just change the CornersMarker’s color property, or pass a different constructor argument. But please note this will only work if the image the filter is applied to indeed supports color… If the image is grayscale, it won’t work. You will have to either create a new color image, or transform it to a color image. I forgot how to do this in AForge, but you can do this with plain .NET, as depicted here.

    Regards,
    Cesar

  15. hello
    I can use this program on my C#. it’s very useful for me.
    but, I want to know the select corner’s image index.

    How can I get that?
    Have some commands or sample to get the corner’s information?
    thanks.

  16. Hi
    i want to apply your code on image contain buildings so i can extract them the buildings how to draw a rectangle or borders on every buildings , do you have idea how to do that ?
    Thanks 🙂

  17. I am not sure, but :
    If you compute A=dx = h*h and B=dy = v*v and C=dxdy = hv Then
    A*B-C*C is always 0 … What you compute as “harris” is only the trace of the matrix. You do not get the hessian part like this in the sum.

  18. I suppose this is why the Gaussian averaging part is needed. This way, the A*B-C*C in the last part becomes different than 0, as it no longer represents the original A, B and C values that were computed as hh, vv and hv. But I am also not completely sure either, as it has been a while since the code was written. Do you think it makes sense?

    Best regards,
    Cesar

  19. Hi, Cesar. Truly thanks for the awesome function.
    I want to apply the function in stitching image of documents to get a higher resolution pictures to extract the character later on. For the image stitching part, do u think it can be applied on document’s processing?

Leave a Reply

Your email address will not be published. Required fields are marked *