detect-simple-shapes-feat-img

When I came to the tutorial on how to detect simple shapes with AForge.NET I wonder how we can do it in OpenCV. Luckily in the OpenCV samples directory there is a program named squares.cpp which is a rectangle detector program. Let’s see how we can extend it to detect other shapes as well.

The program squares.cpp actually detect rectangles rather than squares. But its result is very good. First we’ll take a look at how the program works and then adapt the techniques to detect other simple shapes: triangle, square, pentagon, hexagon, and circle.

How squares.cpp detect rectangles

After some pre-processing steps and retrieving all contours, it loop all of the contours and check if the contour shape is rectangle. Take a look at the code snippet below.

vector<Point> approx;

for (size_t i = 0; i < contours.size(); i++)
{
    approxPolyDP(Mat(contours[i]), approx, 
                 arcLength(Mat(contours[i]), true)*0.02, true);

    if (approx.size() == 4 &&
        fabs(contourArea(Mat(approx))) > 1000 &&
        isContourConvex(Mat(approx)))
    {
        double maxCosine = 0;

        for( int j = 2; j < 5; j++ )
        {
            double cosine = fabs(angle(approx[j%4], approx[j-2], approx[j-1]));
            maxCosine = MAX(maxCosine, cosine);
        }

        if( maxCosine < 0.3 )
            squares.push_back(approx);
    }
}

Given a contours, the program approximate a polygonal curve for that contours. This polygonal curve is the key for detecting the contour’s shape. In the snippet above, the shape is determined to be a rectangle if the polygonal curve meets the following conditions:

  1. It is convex.
  2. It has 4 vertices.
  3. All angles are ~90 degree.

Algorithm to detect other simple shapes

Given the approximation curve, we can quickly determine a contour is a triangle if the curve has 3 vertices. Currently I skip the detection of the angles (right triangle, equilateral triangle, etc.) and simply label them as “triangle”.

To detect pentagon, hexagon, and circle, we will use their properties as shown below.

Pentagon:

  1. Has 5 vertices.
  2. All angles are ~108 degree.

Hexagon:

  1. Has 6 vertices.
  2. All angles are ~120 degree.

Circle:

  1. Has more than 6 vertices.
  2. Has diameter of the same size in each direction.
  3. The area of the contour is ~πr2

In addition, I also want to detect square objects. A square is defined by a rectangle with four equal sides.

Write the code

Let’s get started to write the code. I will use this image as the source image.

detect-simple-shapes-src-img.png

First load the source image,

cv::Mat src = cv::imread("basic-shapes.png");
if (src.empty())
    return -1;

We need to convert the source image to binary image. We do this by converting the image to grayscale first and threshold the grayscaled image. But as suggested in squares.cpp, we’ll use Canny operator rather than thresholding since Canny is able to catch squares with gradient shading.

// Convert to grayscale
cv::Mat gray;
cv::cvtColor(src, gray, CV_BGR2GRAY);

// Convert to binary image using Canny
cv::Mat bw;
cv::Canny(gray, bw, 0, 50, 5);

Perhaps you need to play with the parameters a bit.

The array bw is now a binary image with edges from the shapes. Obtain the contours,

// Find contours
std::vector<std::vector<cv::Point> > contours;
cv::findContours(bw.clone(), contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);

Loop through all the contours and get the approximate polygonal curves for each contour,

// The array for storing the approximation curve
std::vector<cv::Point> approx;

// We'll put the labels in this destination image
cv::Mat dst = src.clone();

for (int i = 0; i < contours.size(); i++)
{
    // Approximate contour with accuracy proportional
    // to the contour perimeter
    cv::approxPolyDP(
        cv::Mat(contours[i]), 
        approx, 
        cv::arcLength(cv::Mat(contours[i]), true) * 0.02, 
        true
    );

    // Skip small or non-convex objects 
    if (std::fabs(cv::contourArea(contours[i])) < 100 || !cv::isContourConvex(approx))
        continue;

The vector approx now contains the vertices of the polygonal approximation for the contour. We will use its size to determine whether the contour is a triangle, polygon, or hexagon.

Getting the triangle is easy, just look where the number of vertices = 3.

    if (approx.size() == 3)
        setLabel(dst, "TRI", contours[i]);    // Triangles

The function setLabel() above is a helper for displaying text in the center of a contour. See the full source code on Github to see its implementation.

To detect square, rectangle, pentagon, and hexagon, we need to obtain the degree of the corners. We will use these values to determine the contour’s shape by using the shape properties above.

    else if (approx.size() >= 4 && approx.size() <= 6)
    {
        // Number of vertices of polygonal curve
        int vtc = approx.size();

        // Get the degree (in cosines) of all corners
        std::vector<double> cos;
        for (int j = 2; j < vtc+1; j++)
            cos.push_back(angle(approx[j%vtc], approx[j-2], approx[j-1]));

        // Sort ascending the corner degree values
        std::sort(cos.begin(), cos.end());

        // Get the lowest and the highest degree
        double mincos = cos.front();
        double maxcos = cos.back();

        // Use the degrees obtained above and the number of vertices
        // to determine the shape of the contour
        if (vtc == 4 && mincos >= -0.1 && maxcos <= 0.3)
        {
            // Detect rectangle or square
            cv::Rect r = cv::boundingRect(contours[i]);
            double ratio = std::abs(1 - (double)r.width / r.height);

            setLabel(dst, ratio <= 0.02 ? "SQU" : "RECT", contours[i]);
        }
        else if (vtc == 5 && mincos >= -0.34 && maxcos <= -0.27)
            setLabel(dst, "PENTA", contours[i]);
        else if (vtc == 6 && mincos >= -0.55 && maxcos <= -0.45)
            setLabel(dst, "HEXA", contours[i]);
    }

If number of the vertices more than 6 then it should fall to the circle detection.

    else
    {
        // Detect and label circles
        double area = cv::contourArea(contours[i]);
        cv::Rect r = cv::boundingRect(contours[i]);
        int radius = r.width / 2;

        if (std::abs(1 - ((double)r.width / r.height)) <= 0.2 &&
            std::abs(1 - (area / (CV_PI * std::pow(radius, 2)))) <= 0.2)
        {
            setLabel(dst, "CIR", contours[i]);
        }
    }
} // end of for() loop

Display both src and dst images,

cv::imshow("src", src);
cv::imshow("dst", dst);
cv::waitKey(0);
return 0;

And here is the final result.

detect-simple-shapes-dst-img.png

The full source code is available on Github.