Course notes: Personal AI trainer

Preamble

Notes from Build a Personal AI Trainer | OpenCV Python 2021 | Computer Vision

Related video courses

See also

Courses

Required files

  • AITrainer/test.jpg – I used a screenshot, hence .png
  • AITrainer/AITrainerSummary.jpg – I used a screenshot, hence .png
  • AITrainer/curls.mp4
  • All from pexels.com

Required Packages

  • opencv-python
  • mediapipe
  • numpy

Course notes

  • Create a method to find the angle between any three points – just specify the landmarks (i.e 11,, 13, 15, or 12, 14, 16)
  • Based on the angle we can see how many “curls” the person has done

First we will use the test image and then later the video

import cv2
import numpy as np
import time

cap = cv2.VideoCapture('AITrainer/curls.mp4')
image = cv2.imread('AITrainer/test.png')

while True:
    # success, image = cap.read()
    # image = cv2.resize(image, (1280, 720))


    cv2.imshow('Image', image)
    cv2.waitKey(1)

Now find the pose, using the module. Add the detector

detector = pm.poseDetector()

We need to read the image inside the loop, else the points and lines get drawn multiple times.

image = cv2.imread('AITrainer/test.png')
image = detector.findPose(image)

Now get the points/landmarks

lmList = detector.findPosition(image, False)
print(lmList)

Add conditional, to do something if there are landmarks

if len(lmList) != 0:

Add a new method to find the angle

Leaving the script for a while and returning to the module – we want to add a method, findAngle(), to the class/module to get an angle for any three points

def findAngle(self, image, p1, p2, p3, draw+True):

However, first we need lmList. We could send it back to the new method from the script that calls it, but it is stupid as it is already part of the class. Make lmList an instance variable/property of the class. Declare in __init()__

self.lmList = []

and change findPosition()

def findPosition(self, image, draw=True):
    iW, iH, iC = image.shape
    if self.results.pose_landmarks:
        for lmId, lm in enumerate(self.results.pose_landmarks.landmark):
            # print(id, lm)
            cx, cy = int(lm.x*iW), int(lm.y*iH)
            self.lmList.append([lmId, cx, cy])
            if draw:
                cv2.circle(image, (cx, cy), 5, (255, 0, 0), cv2.FILLED)
    return self.lmList, image

Returning to findAngle() we can obtain the x1, and y1 coords in one of two ways:

# Slicing
x1, y2 = self.lmList[p1][1:]
# Ignoring the first element
_, x1, y2 = self.lmList[p1]

Highlighting

Add highlighting of the three points to the image

if draw:
    cv2.circle(image, (x1, y1), 5, (255, 0, 0), cv2.FILLED)
    cv2.circle(image, (x2, y3), 5, (255, 0, 0), cv2.FILLED)
    cv2.circle(image, (x3, y3), 5, (255, 0, 0), cv2.FILLED)

As we will eventually remove the mediapipe landmark drawing, we should add additional highlighting ourselves, i.e. draw bullseyes on the three landmarks of interest. Add lines before the points as the points are drawn on top of the lines. So the drawing becomes:

if draw:
    cv2.line(image, (x1, y1), (x2, y2), (255, 255, 255), 3)
    cv2.line(image, (x3, y3), (x2, y2), (255, 255, 255), 3)
    cv2.circle(image, (x1, y1), 10, (0, 0, 255), cv2.FILLED)
    cv2.circle(image, (x1, y1), 15, (0, 0, 255), 2)
    cv2.circle(image, (x2, y3), 5, (0, 0, 255), cv2.FILLED)
    cv2.circle(image, (x1, y1), 15, (0, 0, 255), 2)
    cv2.circle(image, (x3, y3), 5, (0, 0, 255), cv2.FILLED)
    cv2.circle(image, (x1, y1), 15, (0, 0, 255), 2)

Looking at the diagram

We need points/landmarks:

  • left arm – 11, 13, 15
  • right arm – 12, 14, 16

So back in the calling script, after the len(lmList) conditional, call the new method:

if len(lmList) != 0:
    detector.findAngle(image, 12, 14, 16)

Remove the mediapipe landmark highlighting

image = detector.findPose(image, False)

Find the angle

21:03

Back in the new method in the module. First import math and then

# Calculate the angle
angle = math.degrees(math.atan2(y3-y2, x3-x2) - math.atan2(y1-y2, x1-x2))
print(angle)

Display the angle

cv2.putText(image, str(int(angle)), (x2-20, y2 + 50), cv2.FONT_HERSHEY_PLAIN, 2, (255, 0, 255), 2)

Add a check for negative angles (however, this gives weird results (angles greater than 180°) when the test subject is face to the left):

if angle < 0:
    angle += 360

Also, return the angle

return angle

Note: the image is drawn on, without the image being returned to the calling script – strange?

Adding a new arm

if len(lmList) != 0:
    # Right arm
    angle = detector.findAngle(image, 12, 14, 16)
    # Left arm
    angle = detector.findAngle(image, 11, 13, 15)

Running the script on the video

27:16

In the script, we need to map min angle (~320°) to 100 and max angle (~200°) to 0. Use numpy to map.

percentage = np.interp(angle, (210, 310), (0, 100))

By coincidence the range of angle is already 100, but that is just by chance.

print(percentage)

As we have used “safe” values (210, 310), the swing is comfortably in the range of 0 – 100 %, and the 0% and 100% are both reached.

 

Counting the curls

32:15

Add two variables

count = 0
direction = 0

direction = 0 when going up, 1 when going down. A full curl is both of these.

# check for the dumbell curls
if percentage == 100:
    if direction == 0:
        count += 0.5
        direction = 1
if percentage == 0:
    if direction == 1:
        count += 0.5
        direction = 0

print(count)

Why not use and? For feedback, see below

Add FPS

38:00

This is not really needed

Put the count in a box

39:30

# For 720 images
cv2.rectangle(image, (0, 450), (250, 720), (0, 255, 255), cv2.FILLED)
cv2.putText(image, f'{str(int(count))}', (45, 670), cv2.FONT_HERSHEY_PLAIN, 15, (255, 0, 0), 25)

Note that the angle printing is commented out in the module – this should really have a parameter option. However, there is a positioning issue whether the angle is printing inside or outside of the angle.

Putting in the bar

41:12

This is the same as the volume control bar, see Course notes: Finger volume control.

bar = np.interp(angle, (210, 310), (650, 100))

Add

# Make the bar
cv2.rectangle(image, (1100, 100), (1175, 650), (0, 255, 255), 3)
cv2.rectangle(image, (1100, int(bar)), (1175, 650), (0, 255, 255), cv2.FILLED)
cv2.putText(image, f'{int(percentage)} %', (1100, 75), cv2.FONT_HERSHEY_PLAIN, 4, (255, 0, 0), 4)

but…

One more thing – feedback

44:50

(IMHO this is ugly)

When 0 % or 100 % is reached, give visual feedback

Colour of bar is by default purple.

colour = (255, 0, 255)

When min or max reached, turn it green

colour = (0, 255, 0)

Full code

# check for the dumbell curls
colour = (255, 0, 255)
if percentage == 100:
    colour = (0, 255, 0)
    if direction == 0:
        count += 0.5
        direction = 1
if percentage == 0:
    colour = (0, 255, 0)
    if direction == 1:
        count += 0.5
        direction = 0

# Make the bar
cv2.rectangle(image, (1100, 100), (1175, 650), colour, 3)
cv2.rectangle(image, (1100, int(bar)), (1175, 650), colour, cv2.FILLED)
cv2.putText(image, f'{int(percentage)} %', (1100, 75), cv2.FONT_HERSHEY_PLAIN, 4, colour, 4)

Final result

Issue with results

As results is only determined in findPose() it is not declared in __init__(), which causes this warning

Instance attribute results defined outside __init__

As results ends up being assigned a class from  mediapipe, how can we declare it in __init()__. To fix, from this answer, declare in __init__(), as None.

self.results = None

 

 

 

This is the end, my friend

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s