Course notes: Real-time object measurement

Preamble

Course notes from REAL TIME OBJECT MEASUREMENT | OpenCV Python (2020)

Github

Course notes

Add opencv2-python, File>New Projects Settings…>Preferences for New Projects. numpy is added automatically with opencv2-python

Added libraries are shown under Project/External Libraries/site-packages

Notes

cap[3] = 1920 not allowed on OSX

A4 paper is a known size, so used as background (297 height x 210 width)

 

Utilities.py

default parameter to function

Note: at around 21:38 (actually 21:4021:42), brackets, [] , are suddenly added around the parameters passed to append(), because append() can only take one object. It is not mentioned in the video, and it is easily missed.

Explanation of warping: 26:51

Points aren’t always received in the correct order:

Find the correct order of points based on summation and subtraction:

  • point 1 is always the lower number when both co-ordinates of each point are summed: x+y. (x,y) or (0,0)
  • point 4 is always the highest number when summed: x+w + y+h. (x+w, y+h)
  • based on subtraction we can find (width, 0) and (0, height)

The shape of biggest is (4,1,2) which is: 4 points, 2 for x and y,. The 1 is redundant (what does 1 represent???) , so we reshape to (4,2).

However, as we need to send back (i.e return) a matrix of the same shape as the one that was passed to the function (even though one was passed as an argument, and the other is a return value), we have to make a copy of the former shape of the matrix  biggest (before it was reshaped, so including the redundant 1), so we make pointsNew and fill with zeros.

Time: 42:28

Find Objects in the A4 paper

Even though the objects are skewed, we don’t need to warp them.

We can use Pythagorus.

Why: pointsNew = Utilities.reorder(2)????? Why (2)???? Time 49:00. Error on my side, which caused error below. Should be reorder(obj[2])

Issue

I had a strange issue where

 File "/Users/macbook/PycharmProjects/ObjectSizeEstimator/ObjectMeasurement.py", line 56, in <module>
pointsNew = Utilities.reorder(2)
  File "/Users/macbook/PycharmProjects/ObjectSizeEstimator/Utilities.py", line 44, in reorder
    print(f'points.shape={points.shape}')
AttributeError: 'int' object has no attribute 'shape'

Upon the second iteration. Note that reorder() is called by passing 2, not an actual point. Typo: should be reorder(obj[2]), see above.

Notes on quotes and printing

Links

Use single-quotes for string literals, e.g. ‘my-identifier’, but use double-quotes for strings that are likely to contain single-quote characters as part of the string itself (such as error messages, or any strings containing natural language), e.g. “You’ve got an error!”.

 

 

 

Code

Utilities.py

# Utilities.py

import cv2
import numpy as np


def getContours(image, cThr=[100, 100], showCanny=False, minArea=1000, filter=0, draw=False):
    imageGrey = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    imageBlur = cv2.GaussianBlur(imageGrey, (5, 5), 1)
    # Edge detector (Canny)
    imageCanny = cv2.Canny(imageBlur, cThr[0], cThr[1])
    kernel = np.ones((5, 5))
    imageDilation = cv2.dilate(imageCanny, kernel, iterations=3)
    imageEroded = cv2.erode(imageDilation, kernel, iterations=2)
    if showCanny:
        cv2.imshow("Canny", imageCanny)
        cv2.imshow("Canny", imageEroded)

    contours, hierarchy = cv2.findContours(imageEroded, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    finalContours = []
    for i in contours:
        area = cv2.contourArea(i)
        if area > minArea:
            perimeter = cv2.arcLength(i, True)
            approx = cv2.approxPolyDP(i, 0.02 * perimeter, True)
            boundrybox = cv2.boundingRect(approx)
            if filter > 0:
                if len(approx) == filter:
                    finalContours.append([len(approx), area, approx, boundrybox, i])
            else:
                finalContours.append([len(approx), area, approx, boundrybox, i])

    finalContours = sorted(finalContours, key=lambda x: x[1], reverse=True)
    if draw:
        for contour in contours:
            cv2.drawContours(image, contour[4], -1, (0, 0, 255), 3)

    return image, finalContours


def reorder(points):
#    print("points.shape=")
#    print(points.shape)
    print(f'points with error={points}')
    print(f'points.shape={points.shape}')
    print()
    pointsNew = np.zeros_like(points)  # make a copy of the shape of points, before reshaping
    points = points.reshape((4, 2))
    add = points.sum(1)  # sum axis 1
    pointsNew[0] = points[np.argmin(add)]  # x,y
    pointsNew[3] = points[np.argmax(add)]  # x+w,y+h
    diff = np.diff(points, axis=1)
    pointsNew[1] = points[np.argmin(diff)]  # x+w,0
    pointsNew[2] = points[np.argmax(diff)]  # 0,y+h
    return pointsNew


def warpImage(image, points, width, height):
    print(f'points={points}')
    print()
    pointsNew = reorder(points)
    print(f'pointsNew={pointsNew}')
    print()
    points1 = np.float32(pointsNew)
    points2 = np.float32([[0, 0], [width, 0], [0, height], [width, height]])
    matrix = cv2.getPerspectiveTransform(points1, points2)
    imageWarp = cv2.warpPerspective(image, matrix, (width, height))

    return imageWarp


def cropImage(image, crop=20):
    image = image[crop:image.shape[0] - crop, crop:image.shape[1] - crop]
    return image


def findDistance(points1, points2):
    return ((points2[0] - points1[0]) ** 2 + (points2[1] - points1[1]) ** 2) ** 0.5

Main: ObjectMeasurement.py

# ObjectMeasurement.py

import cv2

import numpy as np

import Utilities

#############################

webcam = False
path = "Resources/1.jpg"
cap = cv2.VideoCapture(0)
cap.set(3, 1920)
cap.set(4, 1080)
cap.set(10, 160)

# Size of A4 paper (pixels = mm)
widthA4 = 210
heightA4 = 297
# However this makes for a small image, so we can scale it up
scale = 3
widthA4Window = widthA4 * scale
heightA4Window = heightA4 * scale

while True:
    if webcam:
        success, image = cap.read()
    else:
        image = cv2.imread(path)
    # For initial test
    #   image, finalContours = Utilities.getContours(image, showCanny=True, draw=True)
    # For second test
    #   image, finalContours = Utilities.getContours(image, showCanny=True, minArea=50000, filter=4)
    # For an actual run we don't need to see the Canny image
    imageContours, contours = Utilities.getContours(image, minArea=50000, filter=4)

    if len(contours) != 0:
        biggest = contours[0][2]  # 2nd element is approx variable
        # For testing
        print(f'biggest={biggest}')
        print()
        imageWarp = Utilities.warpImage(image, biggest, widthA4, heightA4)
        imageWarp = Utilities.cropImage(imageWarp)
#        imageWarp = Utilities.warpImage(image, biggest, widthA4Window, heightA4Window)
        imageWarp = cv2.resize(imageWarp, (widthA4Window, heightA4Window))
        # The A4 sheet is now complete
        cv2.imshow("A4", imageWarp)
        # Now look for objects on the A4
        imageContours2, contours2 = Utilities.getContours(imageWarp, minArea=2000, filter=4, cThr=[50, 50],
                                                          draw=False)  # draw = False because we use polylines below
#        cv2.imshow("Objects", imageContours2)
        if len(contours2) != 0:
            for obj in contours2:
                cv2.polylines(imageContours2, [obj[2]], True, (0, 255, 0), 2)
                pointsNew = Utilities.reorder(obj[2])
                widthObj = round(Utilities.findDistance(pointsNew[0][0] // scale, pointsNew[1][0] // scale), 1)
                heightObj = round(Utilities.findDistance(pointsNew[0][0] // scale, pointsNew[2][0] // scale), 1)
                cv2.arrowedLine(imageContours2, (pointsNew[0][0][0], pointsNew[0][0][1]),
                                (pointsNew[1][0][0], pointsNew[1][0][1]),
                                (255, 0, 255), 3, 8, 0, 0.05)
                cv2.arrowedLine(imageContours2, (pointsNew[0][0][0], pointsNew[0][0][1]),
                                (pointsNew[2][0][0], pointsNew[2][0][1]),
                                (255, 0, 255), 3, 8, 0, 0.05)
                x, y, w, h = obj[3]
                cv2.putText(imageContours2, '{}cm'.format(widthObj), (x + 30, y - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5,
                            (255, 0, 255), 2)
                cv2.putText(imageContours2, '{}cm'.format(heightObj), (x - 70, y + h // 2), cv2.FONT_HERSHEY_SIMPLEX,
                            0.5,
                            (255, 0, 255), 2)
        cv2.imshow("Objects", imageContours2)

    # Only resize for display purposes, not for calculations
    cv2.resize(image, (0, 0), None, 0.5, 0.5)
    cv2.imshow("Original", image)
    cv2.waitKey(1)

 

Done

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s