Harris Corners Detector in C#

May 12, 2010 / cesarsouza / 31 Comments

Corner detection, or the more general terminology interest point detection, is an approach used within computer vision systems to extract certain kinds of features from an image. Corner detection is frequently used in motion detection, image matching, tracking, image mosaicing, panorama stitching, 3D modelling and object recognition.

Download sample application (with source code)
Browse the sample application source code
Browse the Harris Corners Detector classes

This code has also been incorporated in Accord.NET Framework, which includes the latest version of this code plus many other statistics and machine learning tools. In order to install the Accord.NET Framework in your projects, use NuGet. Type in the package manager: Install-Package Accord.Imaging

Introduction

One of the firsts operators for interest point detection was developed by Hans P. Moravec in 1977 for his research involving the automatic navigation of a robot through a clustered environment. It was also Moravec who defined the concept of “points of interest” in a image and concluded these interest points could be used to find matching regions in different images.

The Moravec operator is considered to be a corner detector because it defines interest points as points where there are large intensity variations in all directions. This often is the case at corners. It is interesting to note, however, that Moravec was not specifically interested in finding corners, just distinct regions in an image that could be used to register consecutive image frames.

The Harris Operator

This operator was developed by Chris Harris and Mike Stephens in 1988 as a processing step to build interpretations of a robot’s environment based on image sequences. Like Moravec, they needed a method to match corresponding points in consecutive image frames, but were interested in tracking both corners and edges between frames.

Harris and Stephens improved upon Moravec’s corner detector by considering the differential of the corner score with respect to direction directly. The Harris corner detector computes the locally averaged moment matrix computed from the image gradients, and then combines the Eigenvalues of the moment matrix to compute a corner measure, from which maximum values indicate corners positions.

Source Code

Below is the source code for the Harris Corners Detector algorithm. This is mostly the same code implemented in the Accord.NET Framework.

/// Process image looking for corners.
/// </summary>
/// 
/// <param name="image">Source image data to process.</param>
/// 
/// <returns>Returns list of found corners (X-Y coordinates).</returns>
/// 
/// <exception cref="UnsupportedImageFormatException">
///   The source image has incorrect pixel format.
/// </exception>
/// 
public unsafe List<IntPoint> ProcessImage(UnmanagedImage image)
{
    // check image format
    if (
        (image.PixelFormat != PixelFormat.Format8bppIndexed) &amp;&amp;
        (image.PixelFormat != PixelFormat.Format24bppRgb) &amp;&amp;
        (image.PixelFormat != PixelFormat.Format32bppRgb) &amp;&amp;
        (image.PixelFormat != PixelFormat.Format32bppArgb)
        )
    {
        throw new UnsupportedImageFormatException("Unsupported pixel format of the source image.");
    }

    // make sure we have grayscale image
    UnmanagedImage grayImage = null;

    if (image.PixelFormat == PixelFormat.Format8bppIndexed)
    {
        grayImage = image;
    }
    else
    {
        // create temporary grayscale image
        grayImage = Grayscale.CommonAlgorithms.BT709.Apply(image);
    }


    // get source image size
    int width = grayImage.Width;
    int height = grayImage.Height;
    int srcStride = grayImage.Stride;
    int srcOffset = srcStride - width;


    // 1. Calculate partial differences
    float[,] diffx = new float[height, width];
    float[,] diffy = new float[height, width];
    float[,] diffxy = new float[height, width];


    fixed (float* pdx = diffx, pdy = diffy, pdxy = diffxy)
    {
        byte* src = (byte*)grayImage.ImageData.ToPointer() + srcStride + 1;

        // Skip first row and first column
        float* dx = pdx + width + 1;
        float* dy = pdy + width + 1;
        float* dxy = pdxy + width + 1;

        // for each line
        for (int y = 1; y < height - 1; y++)
        {
            // for each pixel
            for (int x = 1; x < width - 1; x++, src++, dx++, dy++, dxy++)
            {
                // Convolution with horizontal differentiation kernel mask
                float h = ((src[-srcStride + 1] + src[+1] + src[srcStride + 1]) -
                           (src[-srcStride - 1] + src[-1] + src[srcStride - 1])) * 0.166666667f;

                // Convolution vertical differentiation kernel mask
                float v = ((src[+srcStride - 1] + src[+srcStride] + src[+srcStride + 1]) -
                           (src[-srcStride - 1] + src[-srcStride] + src[-srcStride + 1])) * 0.166666667f;

                // Store squared differences directly
                *dx = h * h;
                *dy = v * v;
                *dxy = h * v;
            }

            // Skip last column
            dx++; dy++; dxy++;
            src += srcOffset + 1;
        }

        // Free some resources which wont be needed anymore
        if (image.PixelFormat != PixelFormat.Format8bppIndexed)
            grayImage.Dispose();
    }


    // 2. Smooth the diff images
    if (sigma > 0.0)
    {
        float[,] temp = new float[height, width];

        // Convolve with Gaussian kernel
        convolve(diffx, temp, kernel);
        convolve(diffy, temp, kernel);
        convolve(diffxy, temp, kernel);
    }


    // 3. Compute Harris Corner Response Map
    float[,] map = new float[height, width];

    fixed (float* pdx = diffx, pdy = diffy, pdxy = diffxy, pmap = map)
    {
        float* dx = pdx;
        float* dy = pdy;
        float* dxy = pdxy;
        float* H = pmap;
        float M, A, B, C;

        for (int y = 0; y < height; y++)
        {
            for (int x = 0; x < width; x++, dx++, dy++, dxy++, H++)
            {
                A = *dx;
                B = *dy;
                C = *dxy;

                if (measure == HarrisCornerMeasure.Harris)
                {
                    // Original Harris corner measure
                    M = (A * B - C * C) - (k * ((A + B) * (A + B)));
                }
                else
                {
                    // Harris-Noble corner measure
                    M = (A * B - C * C) / (A + B + Accord.Math.Special.SingleEpsilon);
                }

                if (M > threshold)
                {
                    *H = M; // insert value in the map
                }
            }
        }
    }


    // 4. Suppress non-maximum points
    List<IntPoint> cornersList = new List<IntPoint>();

    // for each row
    for (int y = r, maxY = height - r; y < maxY; y++)
    {
        // for each pixel
        for (int x = r, maxX = width - r; x < maxX; x++)
        {
            float currentValue = map[y, x];

            // for each windows' row
            for (int i = -r; (currentValue != 0) &amp;&amp; (i <= r); i++)
            {
                // for each windows' pixel
                for (int j = -r; j <= r; j++)
                {
                    if (map[y + i, x + j] > currentValue)
                    {
                        currentValue = 0;
                        break;
                    }
                }
            }

            // check if this point is really interesting
            if (currentValue != 0)
            {
                cornersList.Add(new IntPoint(x, y));
            }
        }
    }


    return cornersList;
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

/// Process image looking for corners.

/// </summary>

///

/// <param name="image">Source image data to process.</param>

///

/// <returns>Returns list of found corners (X-Y coordinates).</returns>

///

/// <exception cref="UnsupportedImageFormatException">

/// The source image has incorrect pixel format.

/// </exception>

///

public unsafe List<IntPoint> ProcessImage(UnmanagedImage image)

{

// check image format

if (

(image.PixelFormat != PixelFormat.Format8bppIndexed) &&

(image.PixelFormat != PixelFormat.Format24bppRgb) &&

(image.PixelFormat != PixelFormat.Format32bppRgb) &&

(image.PixelFormat != PixelFormat.Format32bppArgb)

)

{

throw new UnsupportedImageFormatException("Unsupported pixel format of the source image.");

}

// make sure we have grayscale image

UnmanagedImage grayImage = null;

if (image.PixelFormat == PixelFormat.Format8bppIndexed)

{

grayImage = image;

}

else

{

// create temporary grayscale image

grayImage = Grayscale.CommonAlgorithms.BT709.Apply(image);

}

// get source image size

int width = grayImage.Width;

int height = grayImage.Height;

int srcStride = grayImage.Stride;

int srcOffset = srcStride - width;

// 1. Calculate partial differences

float[,] diffx = new float[height, width];

float[,] diffy = new float[height, width];

float[,] diffxy = new float[height, width];

fixed (float* pdx = diffx, pdy = diffy, pdxy = diffxy)

{

byte* src = (byte*)grayImage.ImageData.ToPointer() + srcStride + 1;

// Skip first row and first column

float* dx = pdx + width + 1;

float* dy = pdy + width + 1;

float* dxy = pdxy + width + 1;

// for each line

for (int y = 1; y < height - 1; y++)

{

// for each pixel

for (int x = 1; x < width - 1; x++, src++, dx++, dy++, dxy++)

{

// Convolution with horizontal differentiation kernel mask

float h = ((src[-srcStride + 1] + src[+1] + src[srcStride + 1]) -

(src[-srcStride - 1] + src[-1] + src[srcStride - 1])) * 0.166666667f;

// Convolution vertical differentiation kernel mask

float v = ((src[+srcStride - 1] + src[+srcStride] + src[+srcStride + 1]) -

(src[-srcStride - 1] + src[-srcStride] + src[-srcStride + 1])) * 0.166666667f;

// Store squared differences directly

*dx = h * h;

*dy = v * v;

*dxy = h * v;

}

// Skip last column

dx++; dy++; dxy++;

src += srcOffset + 1;

}

// Free some resources which wont be needed anymore

if (image.PixelFormat != PixelFormat.Format8bppIndexed)

grayImage.Dispose();

}

// 2. Smooth the diff images

if (sigma > 0.0)

{

float[,] temp = new float[height, width];

// Convolve with Gaussian kernel

convolve(diffx, temp, kernel);

convolve(diffy, temp, kernel);

convolve(diffxy, temp, kernel);

}

// 3. Compute Harris Corner Response Map

float[,] map = new float[height, width];

fixed (float* pdx = diffx, pdy = diffy, pdxy = diffxy, pmap = map)

{

float* dx = pdx;

float* dy = pdy;

float* dxy = pdxy;

float* H = pmap;

float M, A, B, C;

for (int y = 0; y < height; y++)

{

for (int x = 0; x < width; x++, dx++, dy++, dxy++, H++)

{

A = *dx;

B = *dy;

C = *dxy;

if (measure == HarrisCornerMeasure.Harris)

{

// Original Harris corner measure

M = (A * B - C * C) - (k * ((A + B) * (A + B)));

}

else

{

// Harris-Noble corner measure

M = (A * B - C * C) / (A + B + Accord.Math.Special.SingleEpsilon);

}

if (M > threshold)

{

*H = M; // insert value in the map

}

// 4. Suppress non-maximum points

List<IntPoint> cornersList = new List<IntPoint>();

// for each row

for (int y = r, maxY = height - r; y < maxY; y++)

{

// for each pixel

for (int x = r, maxX = width - r; x < maxX; x++)

{

float currentValue = map[y, x];

// for each windows' row

for (int i = -r; (currentValue != 0) && (i <= r); i++)

{

// for each windows' pixel

for (int j = -r; j <= r; j++)

{

if (map[y + i, x + j] > currentValue)

{

currentValue = 0;

break;

}

// check if this point is really interesting

if (currentValue != 0)

{

cornersList.Add(new IntPoint(x, y));

}

return cornersList;

}

Using the code

Code usage is very simple. A Corners Marker filter from the AForge Framework can be used to directly draw the detected interest points in the original image as it would be usual within the framework.

// Open a image
Bitmap image = ...

// Create a new Harris Corners Detector using the given parameters
HarrisCornersDetector harris = new HarrisCornersDetector(k)
{
    Threshold = threshold,
    Sigma = sigma
};

// Create a new AForge's Corner Marker Filter
CornersMarker corners = new CornersMarker(harris, Color.White);

// Apply the filter and display it on a picturebox
pictureBox1.Image = corners.Apply(image);

// Open a image

Bitmap image = ...

// Create a new Harris Corners Detector using the given parameters

HarrisCornersDetector harris = new HarrisCornersDetector(k)

{

Threshold = threshold,

Sigma = sigma

};

// Create a new AForge's Corner Marker Filter

CornersMarker corners = new CornersMarker(harris, Color.White);

// Apply the filter and display it on a picturebox

pictureBox1.Image = corners.Apply(image);

Sample application

The accompanying sample application is pretty much self-explanatory. It performs the corners detection in the famous Lena Söderberg‘s picture, but can be adapted to work with other pictures as well. Just add another image to the project settings’ resources and change the line of code which loads the bitmap.

It is also very straightforward to adapt the application to load arbitrary images from the hard disk. I have opted to leave it this way for simplicity.

References

P. D. Kovesi. MATLAB and Octave Functions for Computer Vision and Image Processing. School of Computer Science & Software Engineering, The University of Western Australia.
D. Parks and J. P. Gravel. Corner Detectors: The Harris/Plessey Operator. Web. 11 May. 2010.
J. Hutton and B. Dowling. Computer Vision Demonstration Website. Electronics and Computer Science, University of Southampton
H. P. Moravec. Towards Automatic Visual Obstacle Avoidance. Proc. 5th International Joint Conference on Artificial Intelligence, pp. 584, 1977.
H. P. Moravec. Visual Mapping by a Robot Rover. International Joint Conference on Artificial Intelligence, pp. 598-600, 1979.
C. Harris and M. Stephens. A Combined Corner and Edge Detector. Proc. Alvey Vision Conf., Univ. Manchester, pp. 147-151, 1988.
Wikipedia contributors. “Corner detection.” Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia, 5 May. 2010. Web. 11 May. 2010.

Disclaimer

Before using any information, applications or source code available in this article, please be sure to have read the site usage disclaimer.

31 Comments

asd.and.Rizzo says:

May 17, 2010 at 11:59 am

Hi.
Cool thing Accord.Math.
If it could be searchable(like project some where), I would not experiment doing this matrixextensions.codeplex.com.

Reply
César Souza says:

May 21, 2010 at 7:26 pm

Hi,

Accord.NET is a LGPL-licensed project which is available here. The SVN is not open yet, but will be available soon.

Regards,
César

Reply
Txema says:

June 1, 2010 at 2:57 pm

I have tried your code with other images and it doesn’t find corners, it’s necesary to use a specific format file?

Reply
César Souza says:

June 1, 2010 at 4:31 pm

Hi,

I have just updated the downloadable sources in this article with a more recent version. If you are still having trouble, send me an email with the images and I’ll take a look.

By the way, the latest version of the Harris Corner Detector will always be first available inside Accord.NET Framework.

Regards,
César

Reply
Sany Aji M. says:

July 28, 2010 at 5:07 am

hi,

thanks for sharing this tutorial…

is it possible to get the 3D coordinates fo each point?
I mean, i want to obtain the point clouds from captured object…

please help me, how i can do it. i really need it for my final project…

many thanks before…

Reply
Pushpender says:

March 19, 2011 at 7:54 am

I had written a java program for harris corner detection, but my output is not so accurate. I have tried the above program and i’m getting the correct output. However, what does this statement means

if (measure==HarrisCornerMeasure.Harris)

Reply
César Souza says:

March 19, 2011 at 11:28 am

Hi Pushpender,

This statement is only comparing if the chosen corners measure is indeed the original Harris measure. There are other ways to compute the corners measure which do not use the arbitrary factor k. HarrisCornersMeasure is just an enumeration.

I hope I have answered your question.

Best regards,
César

Reply
Pushpender says:

March 20, 2011 at 4:42 am

Hello Cesar,

Thanks.
Do you mean that if we set k=0 then I can use the harris nobel corner measure method i.e.

else
{
M = (A*B-C*C) / (A+B+eps);
}

Reply
César Souza says:

March 20, 2011 at 10:39 pm

Hi Pushpender,

Well, not exactly. If you set the Measure property of an instance of the HarrisCornersDetector class to be HarrisCornerMeasure.Nobel, then the algorithm will ignore the value of k and use Nobel’s measure instead.

i.e.:

// create corners detector’s instance
HarrisCornersDetector hcd = new HarrisCornersDetector();
hcd.Measure = HarrisCornerMeasure.Nobel;

// process image searching for corners
List<IntPoint&gl; corners = hcd.ProcessImage( image );

Regards,
César

Reply
Pushpender says:

March 21, 2011 at 3:00 pm

Hello Cesar

Thanx again. One more question: what is the use dfference between original harris and harris nobel method.? Is it only the k-value. I’m using java to implement the corner detection. So pls tell me how do I set which method to use and when??

Reply
César Souza says:

March 21, 2011 at 11:45 pm

The only difference is that it does not requires the tuning of the parameter k. It makes the usage of the Harris operator simpler, since you won’t have to care about choosing a suitable value for k in your application. For more details, please see Alison Noble, “Descriptions of Image Surfaces”, PhD thesis, Department of Engineering Science, Oxford University 1989, p45.

By the way, if you are basing your implementation on my code, I would kindly ask you to add a reference in your sources about this page and about the Accord.NET Framework (http://accord-net.origo.ethz.ch/).

Best regards,
César

Reply
Pushpender says:

March 22, 2011 at 3:01 pm

Thanks a lot. I’ll add the respective references.

Reply
Anonymous says:

February 26, 2012 at 7:18 pm

I want to put those inputs like k, threshold and sigma in your Automatic Image Stitching but why is it when I press the Detect button and the sigma’s value sets in its max value is which 5.000 or I set it to min value which is 0.000 there’s points being detected that supposed to be none. I find it hard to refactor in this part this on Harris Corners Detector not AIS:

float[,] diffx = new float[height, width];
float[,] diffy = new float[height, width];
float[,] diffxy = new float[height, width];

fixed (float* pdx = diffx, pdy = diffy, pdxy = diffxy)
{
byte* src = (byte*)grayImage.ImageData.ToPointer() + srcStride + 1;

// Skip first row and first column
float* dx = pdx + width + 1;
float* dy = pdy + width + 1;
float* dxy = pdxy + width + 1;

// for each line
for (int y = 1; y < height – 1; y++)
{
// for each pixel
for (int x = 1; x < width – 1; x++, src++, dx++, dy++, dxy++)
{
// Convolution with horizontal differentiation kernel mask
float h = ((src[-srcStride + 1] + src[+1] + src[srcStride + 1]) –
(src[-srcStride – 1] + src[-1] + src[srcStride – 1])) * 0.166666667f;

// Convolution vertical differentiation kernel mask
float v = ((src[+srcStride – 1] + src[+srcStride] + src[+srcStride + 1]) –
(src[-srcStride – 1] + src[-srcStride] + src[-srcStride + 1])) * 0.166666667f;

// Store squared differences directly
*dx = h * h;
*dy = v * v;
*dxy = h * v;
}

// Skip last column
dx++; dy++; dxy++;
src += srcOffset + 1;
}

// Free some resources which wont be needed anymore
if (image.PixelFormat != PixelFormat.Format8bppIndexed)
grayImage.Dispose();
}

// 2. Smooth the diff images
if (sigma > 0.0)
{
float[,] temp = new float[height, width];

// Convolve with Gaussian kernel
convolve(diffx, temp, kernel);
convolve(diffy, temp, kernel);
convolve(diffxy, temp, kernel);
}

Where exactly the part I should alter so that sigma will work what I want to. Should I also include the convolve method

Reply
Anonymous says:

February 26, 2012 at 8:32 pm

Hi César,
Sorry for the typos (*there are points). Wanna ask again do I have to include the convolve method in HarrisCornersDetector in Automatic Image Stitching. Convert the float [,] types to UnmanagedImage and change those fixed values to unsafe. Just really got problem with sigma alone. The k and threshold works well.

Reply
DANNI31 says:

April 13, 2012 at 12:27 am

hello;
I use the java version of this implementation and I want to recover the points detect a new vector to use it later if anyone has any idea how to recover the point values in a new vector

Reply
Anonymous says:

October 21, 2012 at 4:45 pm

hi Cesar
i make download to the source code of harris in c# ,and when i start to excute the code i found some error in the program and i cannnot solve it can u help me plzz to solve these error .. if u can tell me, then i will send u the error
thanx

Reply
Anonymous says:

December 17, 2012 at 1:05 pm

hi..
how can i solve this erro ?
{accord-imaging-harris-srcbinx86debugHarris.exe is missing .please build the project and retry , or set the outputPath and assembly name properties appropriately to point at the correct location for the target assembly.} plz help me … thanx

Reply
César Souza says:

December 17, 2012 at 2:13 pm

Hi,

I am not sure about this particular error, but I would advise you to download the full Accord.NET Framework. Perhaps the installer may resolve any issues you are having. Please see the starting guide to get started with the framework if you wish!

Best regards,
Cesar

Reply
Anonymous says:

December 17, 2012 at 7:32 pm

ok thank u very much i will try

Reply
Anonymous says:

January 14, 2013 at 3:40 pm

hi cesar
how can i change the color of point in harris corner detector to anther color like red or yellow in visual studio (c# , accord.net)

Reply
César Souza says:

January 14, 2013 at 6:04 pm

Hi,

You can just change the CornersMarker’s color property, or pass a different constructor argument. But please note this will only work if the image the filter is applied to indeed supports color… If the image is grayscale, it won’t work. You will have to either create a new color image, or transform it to a color image. I forgot how to do this in AForge, but you can do this with plain .NET, as depicted here.

Regards,
Cesar

Reply
Anonymous says:

April 1, 2013 at 9:45 am

Hi

The “Download sample application” is missing.(404 Not Found)
Can you upload again? plz.
thx

Reply
César Souza says:

April 2, 2013 at 12:21 am

Thanks, should be fixed now.

Reply
Anonymous says:

April 2, 2013 at 9:20 am

hello
I can use this program on my C#. it’s very useful for me.
but, I want to know the select corner’s image index.

How can I get that?
Have some commands or sample to get the corner’s information?
thanks.

Reply
Anonymous says:

May 8, 2013 at 2:56 am

Hi
i want to apply your code on image contain buildings so i can extract them the buildings how to draw a rectangle or borders on every buildings , do you have idea how to do that ?
Thanks 🙂

Reply
Anonymous says:

December 2, 2013 at 8:58 am

I am not sure, but :
If you compute A=dx = h*h and B=dy = v*v and C=dxdy = hv Then
A*B-C*C is always 0 … What you compute as “harris” is only the trace of the matrix. You do not get the hessian part like this in the sum.

Reply
César Souza says:

February 13, 2014 at 12:33 pm

I suppose this is why the Gaussian averaging part is needed. This way, the A*B-C*C in the last part becomes different than 0, as it no longer represents the original A, B and C values that were computed as hh, vv and hv. But I am also not completely sure either, as it has been a while since the code was written. Do you think it makes sense?

Best regards,
Cesar

Reply
Anonymous says:

March 9, 2014 at 2:10 pm

Hi, Cesar. Truly thanks for the awesome function.
I want to apply the function in stitching image of documents to get a higher resolution pictures to extract the character later on. For the image stitching part, do u think it can be applied on document’s processing?

Reply
isuru lakmal Perera says:

May 26, 2014 at 7:05 am

Hi Cesar,
Nice post.Can you please let me know that if I can change this code to get only the most top edge(point) of the image.

Reply
Илья Запруцкий says:

October 16, 2014 at 9:04 am

Hi

The “Download sample application” is missing.(404 Not Found)
Can you upload again?
thx

Reply
Pingback: Image Stitching | Jan Kristanto