Course notes: OpenCV/Python

Preamble

Notes from LEARN OPENCV in 3 HOURS with Python | Including 3x Example Projects

Github and OpenCV-Python-Tutorials-and-Projects, Document Scanner

See also

Related courses – videos

Notes from OpenCV/Python video course

Intro

  • Pixels,
  • Resolution
      • VGA 640×480,
      • HD 1280×720,
      • FHD 1920×1080,
      • 4k 3840 x 2160
  • binary image (B&W),
  • greyscale (8 bit 256 level, 254 shades of grey),
  • RGB (three greyscale images, one each for red, green and blue 640×480 x 3)

Software

  • PyCharm
  • Python 3.7.6 as this works well with OpenCV
  • Added Python 3.7 to the Path (on Windows)
  • New Project : OpenCVPython
  • If using brew then path is /usr/local/Cellar/python@3.7/bin/python3.7

Chapter 1 – Read images, video, webcam

File>Settings… (New Projects Settings…>Preferences for New Projects…), interpreter, add opencv-python.

Package

import cv2

print("Package imported")

Image

# Chapter1_image.py

import cv2
image = cv2.imread("Resources/Air_Pakistan.png")
cv2.imshow("output", image)
cv2.waitKey(0)

Video

# Chapter1_video.py

import cv2

cap = cv2.VideoCapture("Resources/sample.mkv")

while True:
    success, image = cap.read()
    cv2.imshow("Video", image)
    if cv2.waitKey(1) & 0xFF == ord('q') :
        break

Webcam

# Chapter1_webcam.py

import cv2

cap = cv2.VideoCapture(0)

cap.set(3, 640)  # Width
cap.set(4, 480)  # Height
cap.set(10, 100) # Brightness

while True:
    success, image = cap.read()
    cv2.imshow("Video", image)
    if cv2.waitKey(1) & 0xFF == ord('q') :
        break

Chapter 2 – Basic functions

 

# Chapter2_imageManipulation.py

import cv2
import numpy as np

kernel = np.ones((5,5),np.uint8)

image = cv2.imread("Resources/Air_Pakistan.png")
imageGrey = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
imageBlur = cv2.GaussianBlur(imageGrey, (7,7), 0)
# Edge detector (Canny)
imageCanny1 = cv2.Canny(image,100, 100)
imageCanny2 = cv2.Canny(image,150, 200)
# Dilation
imageDilation1 = cv2.dilate(imageCanny2, kernel,iterations = 1)
imageDilation2 = cv2.dilate(imageCanny2, kernel,iterations = 5)
imageEroded1 = cv2.erode(imageDilation1, kernel, iterations = 1)
imageEroded2 = cv2.erode(imageDilation2, kernel, iterations = 5)


cv2.imshow("Grey image", imageGrey)
cv2.imshow("Blur image", imageBlur)
cv2.imshow("Canny image1", imageCanny1)
cv2.imshow("Canny image2", imageCanny2)
cv2.imshow("Dilated image1", imageDilation1)
cv2.imshow("Dilated image2", imageDilation2)
cv2.imshow("Eroded image1", imageEroded1)
cv2.imshow("Eroded image2", imageEroded2)
cv2.waitKey(0)

Notes

  • Canny works on image, i.e. the original, should it work on imageBlur?

Chapter 3 – Resizing and cropping

OpenCV convention ; +Y axis is south

Code

# Chapter3_imageResizeCrop.py

import cv2
import numpy as np

image = cv2.imread("Resources/Air_Pakistan.png")
print(image.shape)
imageResize1 = cv2.resize(image,(200,300))
print(imageResize1.shape)
imageResize2 = cv2.resize(image,(1000,1400))
print(imageResize2.shape)
#Cropping: height and width
imageCropped = image[0:200,200:500]

cv2.imshow("image", image)
cv2.imshow("image resized1", imageResize1)
cv2.imshow("image resized2", imageResize2)
cv2.imshow("image cropped", imageCropped)
cv2.waitKey(0)

Note: image.shape shows [height, width, colour depth]. Yet, cv2.resize(width, height). There is a transposition of height and width. Ridiculous!!!

Chapter 4 – shapes and texts

Array of zeros (zero is black)

 

# Chapter4_ShapesText.py

import cv2
import numpy as np

imageBW = np.zeros((512,512))
print(imageBW.shape)
imageColour = np.zeros((512,512,3),np.uint8)
print(imageColour.shape)
imageColour[:] = 255,0,0
imageColourPartial = np.zeros((512,512,3),np.uint8)
print(imageColour.shape)
imageColourPartial[200:300, 100:300] = 255,255,0

cv2.line(imageColour,(0,0), (300,300), (0,255,0), 3)
cv2.line(imageColour,(10,0), (imageBW.shape[1],imageBW.shape[0]), (0,255,255), 3)
cv2.rectangle(imageColour,(0,0),(250,350),(128,0,255),2)
cv2.rectangle(imageColour,(10,100),(450,350),(128,128,255),cv2.FILLED)
cv2.circle(imageColour, (128,128), 30, (130,255,34),5)
cv2.putText(imageColour, "OpenCV", (300,100),cv2.FONT_HERSHEY_SIMPLEX,1,(0,150,0),1)
cv2.putText(imageColour, "OpenCV", (300,100),cv2.FONT_HERSHEY_SIMPLEX,2,(0,150,0),1)
cv2.putText(imageColour, "OpenCV", (300,100),cv2.FONT_HERSHEY_SIMPLEX,0.5,(0,150,0),1)
cv2.putText(imageColour, "OpenCV", (400,100),cv2.FONT_HERSHEY_SIMPLEX,1,(0,150,0),3)

cv2.imshow("imageBW", imageBW)
cv2.imshow("imageColour", imageColour)
cv2.imshow("imageColourPartial", imageColourPartial)
cv2.waitKey(0)

Chapter 5 – Warp perspective

# Chapter5_WarpPerspective.py

import cv2
import numpy as np

image = cv2.imread("Resources/Air_Pakistan.png")
# 2,5" x 3.5"
width, height = 250,350
# These values are determined manually in a drawing application
points1 = np.arryfloat32([111,219],[287,188],[154,482],[352,440])
points2 = np.float32([0,0],[width,0],[0,height],[width,height])

matrix = cv2.getPerspectiveTransform(points1, points2)
imageOutput = cv2.warpPerspective(image, matrix, (width,height))

cv2.imshow("Image input", image)
cv2.imshow("Image Output", imageOutput)
cv2.waitKey(0)

Chapter 6 – Joining Images

Issues:

  • Both images must have same number of channels
  • Can’t resize image
# Chapter6_JoiningImages.py

import cv2
import numpy as np

image = cv2.imread("Resources/Air_Pakistan.png")

imageHorizontal = np.hstack((image, image))
imageVertical = np.vstack((image, image))

cv2.imshow("Image input", image)
cv2.imshow("Horizontal", imageHorizontal)
cv2.imshow("Vertical", imageVertical)
cv2.waitKey(0)

Function: stackImages (full function can be found on Github)

Time: 53:35

# Chapter6_StackImages.py

import cv2
import numpy as np


def stackImages(scale, imageArray):
    rows = len(imageArray)
    cols = len(imageArray[0])
    rowsAvailable = isinstance(imageArray[0], list)

    width = imageArray[0][0].shape[1]
    height = imageArray[0][0].shape[0]

    if rowsAvailable:
        for x in range(0, rows):
            for y in range(0, cols):

                if imageArray[x][y].shape[:2] == imageArray[0][0].shape[:2]:
                    imageArray[x][y] = cv2.resize(imageArray[x][y], (0, 0), None, scale, scale)
                else:
                    imageArray[x][y] = cv2.resize(imageArray[x][y], (imageArray[0][0].shape[1],imageArray[0][0].shape[0]), None, scale, scale)
                if len(imageArray[x][y].shape) == 2:
                    imageArray[x][y] = cv2.cvtColor(imageArray[x][y],cv2.COLOR_GRAY2BGR)

        imageBlank = np.zeros((height, width, 3), np.uint8)
        hor = [imageBlank] * rows
        hor_com = [imageBlank] * rows

        for x in range(0, rows):
            hor[x] = np.hstack(imageArray[x])
        ver = np.vstack(hor)

    else:
        for x in range(0, rows):
            if imageArray[x].shape[:2] == imageArray[0].shape[:2]:
                imageArray[x] = cv2.resize(imageArray[x], (0, 0), None, scale, scale)
            else:
                imageArray[x] = cv2.resize(imageArray[x], (imageArray[0].shape[1], imageArray[0].shape[0]), None, scale,
                                           scale)
            if len(imageArray[x].shape) == 2:
                imageArray[x] = cv2.cvtColor(imageArray[x],cv2.COLOR_GRAY2BGR)  
        hor = np.hstack(imageArray)
        ver = hor
    return ver


image = cv2.imread("Resources/Air_Pakistan.png")
imageGrey = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

imageStack1 = stackImages(0.5, ([image, image, image]))
imageStack2 = stackImages(0.5, ([image, image, image], [image, image, image]))
imageStack3 = stackImages(0.5, ([image, imageGrey, image], [image, image, image]))

cv2.imshow("Image Stack1", imageStack1)
cv2.imshow("Image Stack2", imageStack2)
cv2.imshow("Image Stack3", imageStack3)
cv2.waitKey(0)

Chapter 7 – Colour detection

# Chapter7_ColourDetection.py

import cv2
import numpy as np


def empty(a):
    pass

def stackImages(scale, imageArray):
    rows = len(imageArray)
    cols = len(imageArray[0])
    rowsAvailable = isinstance(imageArray[0], list)

    width = imageArray[0][0].shape[1]
    height = imageArray[0][0].shape[0]

    if rowsAvailable:
        for x in range(0, rows):
            for y in range(0, cols):

                if imageArray[x][y].shape[:2] == imageArray[0][0].shape[:2]:
                    imageArray[x][y] = cv2.resize(imageArray[x][y], (0, 0), None, scale, scale)
                else:
                    imageArray[x][y] = cv2.resize(imageArray[x][y], (imageArray[0][0].shape[1],imageArray[0][0].shape[0]), None, scale, scale)
                if len(imageArray[x][y].shape) == 2:
                    imageArray[x][y] = cv2.cvtColor(imageArray[x][y],cv2.COLOR_GRAY2BGR)

        imageBlank = np.zeros((height, width, 3), np.uint8)
        hor = [imageBlank] * rows
        hor_com = [imageBlank] * rows

        for x in range(0, rows):
            hor[x] = np.hstack(imageArray[x])
        ver = np.vstack(hor)

    else:
        for x in range(0, rows):
            if imageArray[x].shape[:2] == imageArray[0].shape[:2]:
                imageArray[x] = cv2.resize(imageArray[x], (0, 0), None, scale, scale)
            else:
                imageArray[x] = cv2.resize(imageArray[x], (imageArray[0].shape[1], imageArray[0].shape[0]), None, scale,
                                           scale)
            if len(imageArray[x].shape) == 2:
                imageArray[x] = cv2.cvtColor(imageArray[x],cv2.COLOR_GRAY2BGR)
        hor = np.hstack(imageArray)
        ver = hor
    return ver


path = "Resources/Air_Pakistan.png"
cv2.namedWindow("TrackBars")
cv2.resizeWindow("TrackBars", 640, 240)
cv2.createTrackbar("HUE Min", "TrackBars", 0, 179, empty)
cv2.createTrackbar("HUE Max", "TrackBars", 179, 179, empty)
cv2.createTrackbar("SAT Min", "TrackBars", 0, 255, empty)
cv2.createTrackbar("SAT Max", "TrackBars", 255, 255, empty)
cv2.createTrackbar("VAL Min", "TrackBars", 0, 255, empty)
cv2.createTrackbar("VAL Max", "TrackBars", 255, 255, empty)

image = cv2.imread(path)
imageHSV = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)

while True:


    hueMinimum = cv2.getTrackbarPos("HUE Min","TrackBars")
    hueMaximum = cv2.getTrackbarPos("HUE Max","TrackBars")
    satMinimum = cv2.getTrackbarPos("SAT Min", "TrackBars")
    satMaximum = cv2.getTrackbarPos("SAT Max", "TrackBars")
    valueMinimum = cv2.getTrackbarPos("VAL Min", "TrackBars")
    valueMaximum = cv2.getTrackbarPos("VAL Max", "TrackBars")
#    print(hMinimum)
#    print(hMaximum)
    print(hueMinimum, hueMaximum, satMinimum, satMaximum, valueMinimum, valueMaximum)
    lower = np.array([hueMinimum, satMinimum,valueMinimum])
    upper = np.array([hueMaximum, satMaximum,valueMaximum])
    mask = cv2.inRange(imageHSV, lower, upper)
    imageResult = cv2.bitwise_and(image, image, mask=mask)


    image = cv2.imread(path)

    imageHSV = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)

 #   cv2.imshow("Image input", image)
 #   cv2.imshow("Image HSV", imageHSV)
 #   cv2.imshow("Mask", mask)
 #   cv2.imshow("Result", imageResult)

    imageStack = stackImages(0.6, ([image, imageHSV],[mask,imageResult]))
    cv2.imshow("Stack", imageStack)

    cv2.waitKey(1)

Notes:

  • Hue goes up to 360, but opencv only up to 179 (half of 360) – this is because Hue is -180° to 180°
  • Use empty function for trackbar as we get values in another way, rather than onEvent
  • Not reatime move of trackbars, it is “move and release”
  • No values shown on the trackbars themselves. Known Issue: 5056
  • Window not resized
  • Trackbars are added in reverse order. Known Issue: 5056
  • See also OpenCV trackbars and OS X

Chapter 8 – Contours/Shape Detection

 

  1. Preprocess – to gray
  2. Find edges
  3. find corner points
# Chapter8_Contours.py

import cv2
import numpy as np


def stackImages(scale, imageArray):
    rows = len(imageArray)
    cols = len(imageArray[0])
    rowsAvailable = isinstance(imageArray[0], list)

    width = imageArray[0][0].shape[1]
    height = imageArray[0][0].shape[0]

    if rowsAvailable:
        for x in range(0, rows):
            for y in range(0, cols):

                if imageArray[x][y].shape[:2] == imageArray[0][0].shape[:2]:
                    imageArray[x][y] = cv2.resize(imageArray[x][y], (0, 0), None, scale, scale)
                else:
                    imageArray[x][y] = cv2.resize(imageArray[x][y],
                                                  (imageArray[0][0].shape[1], imageArray[0][0].shape[0]), None, scale,
                                                  scale)
                if len(imageArray[x][y].shape) == 2:
                    imageArray[x][y] = cv2.cvtColor(imageArray[x][y], cv2.COLOR_GRAY2BGR)

        imageBlank = np.zeros((height, width, 3), np.uint8)
        hor = [imageBlank] * rows
        hor_com = [imageBlank] * rows

        for x in range(0, rows):
            hor[x] = np.hstack(imageArray[x])
        ver = np.vstack(hor)

    else:
        for x in range(0, rows):
            if imageArray[x].shape[:2] == imageArray[0].shape[:2]:
                imageArray[x] = cv2.resize(imageArray[x], (0, 0), None, scale, scale)
            else:
                imageArray[x] = cv2.resize(imageArray[x], (imageArray[0].shape[1], imageArray[0].shape[0]), None, scale,
                                           scale)
            if len(imageArray[x].shape) == 2:
                imageArray[x] = cv2.cvtColor(imageArray[x], cv2.COLOR_GRAY2BGR)
        hor = np.hstack(imageArray)
        ver = hor
    return ver


def getContours(image):
    contours, hierarchy = cv2.findContours(image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
    for cnt in contours:
        area = cv2.contourArea(cnt)
        print(area)
        if area > 500:
            cv2.drawContours(imageContour, cnt, -1, (255, 0, 0), 3)
            perimeter = cv2.arcLength(cnt, True)
            print(perimeter)
            approxCorner = cv2.approxPolyDP(cnt, 0.02 * perimeter, True)
            print(approxCorner, len(approxCorner))
            objectCorners = len(approxCorner)
            x, y, width, height = cv2.boundingRect(approxCorner)
            if objectCorners == 3:
                objectType="Triangle"
            elif objectCorners == 4:
                aspectRatio = width/float(height)
                if aspectRatio >0.95 and aspectRatio < 1.05: objectType = "Square" else: objectType = "Rectangle" elif objectCorners >4:
                objectType = "Circle"
            else:
                objectType="null"


            cv2.rectangle(imageContour,(x,y),(x+width,y+height),(0,255,0),3)
            cv2.putText(imageContour,objectType, (x+(width//2) -10, y+(height//2)-10),cv2.FONT_HERSHEY_SIMPLEX, 0.5,(0,0,0),2)

# path = "Resources/Air_Pakistan.png"
path = "Resources/shapes.png"
image = cv2.imread(path)
imageContour = image.copy()

imageGrey = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
imageBlur = cv2.GaussianBlur(imageGrey, (7, 7), 0)
# Edge detector (Canny)
imageCanny = cv2.Canny(image, 50, 50)
getContours(imageCanny)

imageBlank = np.zeros_like(image)
# cv2.imshow("Original image", image)
# cv2.imshow("Grey image", imageGrey)
# cv2.imshow("Blur image", imageBlur)
# cv2.imshow("Canny image1", imageCanny)

imageStack = stackImages(0.6, ([image, imageGrey, imageBlur], [imageCanny, imageContour, imageBlank]))

cv2.imshow("Stack", imageStack)

cv2.waitKey(0)

Chapter 9 – Face detection

  • Viola and Jones
  • Collect a lot of Positives (faces) and Negative (non-faces) > train>XML file
  • OpenCV has default cascades (XML files).
  • Creating a custom XML file -> LINK?????
# Chapter9_FaceDetection.py

import cv2
import numpy as np

faceCascade = cv2.CascadeClassifier("Resources/haarcascade_frontalface_default.xml")
path = "Resources/lena.png"
image = cv2.imread(path)
imageGrey = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

faces = faceCascade.detectMultiScale(imageGrey, 1.1, 4)

for (x, y, width, height) in faces:
    cv2.rectangle(image, (x, y), (x + width, y + height), (255, 0, 0), 2)

cv2.imshow("Face Detection", image)

cv2.waitKey(0)

Chapter 10 – Virtual Paint

Uses Chapter1_webcam.py

Notes

  • webcam.py has changed to use frameWidth, frameHeight
  • Makes use of ColourPicker – however, the locations of the TrackBars has been changed, so that the min and the max trackbars are groups together, rather than grouping in pairs for hue, sat, and value.
  • [0:3] and [3:6] should be [0:2] and [3:5] (???)
  • At 2:07:07 there is a random dot top left corner, which is green with no, and orange and purple pens and then changes colour (to purple) when green pen is shown in camera image

Process

  • Create mask
  • For each colour in mask, locate colour in image
  • Find contours
  • Draw bounding rect
  • However, we want to tip of the pen and not the center
  • Tip is only correct if pen held vertically (with tip at the top)
  • Draw tip in the correct colour
  • Create a list of points
  • Draw all points in list
# Chapter10_VirtualPaint.py

import cv2
import numpy as np

cap = cv2.VideoCapture(0)

cap.set(3, 640)
cap.set(4, 480)
cap.set(10, 150)

# Values for orange, purple and green pens
myColours = [[5, 107, 0, 19, 255, 255],
             [133, 56, 0, 156, 255],
             [57, 76, 0, 100, 255, 255]
             [90, 58, 0, 118, 255, 255]]

myPaints = [[51, 153, 255], [255, 0, 255], [0, 255, 0], [255, 0, 0]]  # BGR

myPaintsName = ["Orange", "Purple", "Green", "Blue"]

myPoints = []  # [x,y,ColorID]


def findColour(image, myColours, myPaints):
    imageHSV = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
    count = 0
    newPoints = []
    for colour in myColours:
        #       lower = np.array([myColours[0][0:3]])
        #       upper = np.array([myColours[0][3:6]])
        lower = np.array(colour[0:3])
        upper = np.array(colour[3:6])
        mask = cv2.inRange(imageHSV, lower, upper)
        x, y = getContours(mask)
        #       cv2.circle((imageResult, (x, y), 10, (255,0,0), cv2.FILLED))  # Paint in Blue
        cv2.circle((imageResult, (x, y), 10, myPaints[count], cv2.FILLED))

        #        cv2.imshow(str(colour[0]), mask)  # Print first value in HSV array (Hue minimum) as window title
        cv2.imshow(myPaintsName[count], mask)  # Use count to point to paint name
        if x != 0 and y != 0:
            newPoints.append[x, y, count]
        count += 1
    return newPoints


def getContours(image):
    x, y, wisth, height = 0, 0, 0, 0
    contours, hierarchy = cv2.findContours(image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
    for cnt in contours:
        area = cv2.contourArea(cnt)
        #       print(area)
        if area > 500:
            # Draw bounding box
            cv2.drawContours(imageResult, cnt, -1, (255, 0, 0), 3)  # Only needed for testing
            perimeter = cv2.arcLength(cnt, True)
            #           print(perimeter)
            approxCorner = cv2.approxPolyDP(cnt, 0.02 * perimeter, True)
            #           print(approxCorner, len(approxCorner))
            #           objectCorners = len(approxCorner)
            x, y, width, height = cv2.boundingRect(approxCorner)
            return x + width // 2, y


def drawOnCravas(myPoints, myPaints):
    for point in myPoints:
        cv2.circle((imageResult, (point[0], point[1]), 10, myPaints[point[2]], cv2.FILLED))


while True:
    success, image = cap.read()
    imageResult = image.copy()  # image that will have all final information on it
    newPoints = findColour(image, myColours, myPaints)
    if len(newPoints) != 0:
        for newP in newPoints:
            myPoints.append(newP)
    if len(myPoints) != 0:
        drawOnCravas(myPoints, myPaints)
    cv2.imshow("Video", image)
    cv2.imshow("Video", imageResult)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

Chapter 11 – Document Scanner

Uses Chapter1_webcam.py but can check using hi-res still (provided paper.jpg)

Notes

  • webcam.py has changed to use frameWidth, frameHeight

Process

  • Preprocess to detect edge
  • Find biggest contour (and store in biggest)
  • Warp (but fix (i.e. reorder) the jumbled points of biggest)
      • add x and y values of each point to find the smallest and largest, i.e. (0,0) and (x+w, y+h)
      • difference (of axis=1) min (width, 0) and max (0, height)
      • Confused? What is axis=1? axis=1 is the axis of the matrix which holds the (x,y) co-ordinates. We reshaped biggest from (4,1,2) to (4,2) => 4 co-ordinates in the first axis of the matrix
      • For full explanation see REAL TIME OBJECT MEASUREMENT | OpenCV Python (2020) @ 26:51
  • then the warp is too wide, so… change aspect ratio image width and height are transposed
  • Clean up image (remove edges caught by the dilation), by a crop (remember order: height, width)
  • Join images to see workflow
  • Fix error if no document

 

 

Chapter 12 – Number plate detection

 

Uses Chapter1_webcam.py but can check using hi-res still images (provided p1.jpg, p2.jpg and p3.jpg)

Process

  • Filter to take objects bigger than a certain area
  • Save image when number plate detected

 

 

 

ddd

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s