Course notes: Face Landmarks

Preamble

Notes from Detect 468 Face Landmarks in Real-time | OpenCV Python | Computer Vision

See also

  • blah

Courses

Required files

Required packages

  • opencv-python
  • mediapipe

Course notes

Create a new file FaceMeshBasics.py

First test

 

import cv2
import mediapipe as mp
import time

cap = cv2.VideoCapture("Videos/2.mp4")


while True:
    success, image = cap.read()
    cv2.imshow("Image", image)
    cv2.waitKey(1)

Add frame rate

cTime = time.time()
fps = 1 / (cTime - pTime)
pTime = cTime

cv2.putText(image, f'FPS: {int(fps)}', (20,70), cv2.FONT_HERSHEY_PLAIN, 3, (0,255,0), 3)

FPS is 170

Use mediapipe

6:15

Use mediapipe to find the points on the face

We could draw the lines ourselves, but the mp has complex drawing routines built in. However, we could draw the points ourselves.

 

Arguments  to Facemesh():

  • static_image_mode
  • max_num_faces
  • min_detection_confidence
  • min_tracking_confidence

static only for detection, if false, it detection and then track. Detection is heavier than Tracking. Detect first, when con>50%,  then detect, and keep tracking the face if track con>50%. Don’t change the con, but set max faces to 2

mpDraw = mp.solutions.drawing_utils
mpFaceMesh = mp.solutions.face_mesh
faceMesh = mpFaceMesh.Facemesh(max_num_faces=2

Only RGB is accepted. So convert

imageRGB = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
results = faceMesh.process(imageRGB)

now FPS is 60

Display results

10:55

Need to loop through the faces before drawing

if results.multi_face_landmarks:
    for faceLms in results.multi_face_landmarks:
        mpDraw.draw_landmarks(img, faceLms, mpFaceMesh.FACE_CONNECTIONS)

Changing the size of the circles

13:00

Can change from the default thickness=1, circle_radius=1

drawSpec = mpDraw.DrawingSpec(thickness=1, circle_radius=2)

and change

mpDraw.draw_landmarks(img, faceLms, mpFaceMesh.FACE_CONNECTIONS, drawSpec, drawSpec)

For HD (720) 1,1 is good, but for FullHD 1080 then 1,1 is too subtle, and 2,2 is more visible.

Doing something with the points

15:30

To use the points, you knew to know their position – total 468 points

The first loop was for the face number

Now add new loop

for lm in faceLms.landmark:
    print(lm)

You get the x, y, z. Normalised from 0 to 1. Need to convert to pixels coords.

imageWidth, imageHeight, imageChannels = image.shape
x, y = int(lm.x*imageWidth), int(lm.y*imageHeight)
 print(x,y)

We have ignored z

TO get id

for id, lm in enumerate(faceLms.landmark):

...

 print(id, x, y)

Values of x and y for landmarks 0-467 are printed

For multiple faces, to enumerate the face ID, change

for faceId, faceLms in enumerate(results.multi_face_landmarks):

We can make a list of the id landmarks, we will do this in the module

Module creation

20:37

Create new file FaceMeshModule.py

Copy and paste all of the code.

Add

if __name__ == "__main__":
    main()

Make a main()

cut and paste into main()

cTime = time.time()
fps = 1 / (cTime - pTime)
pTime = cTime

cv2.putText(image, f'FPS: {int(fps)}', (20, 70), cv2.FONT_HERSHEY_PLAIN, 3, (0, 255, 0), 3)

cv2.imshow("Image", image)
cv2.waitKey(1)

and cut and paste the loop

while True:
    success, image = cap.read()

 

and cut and paste the globals

cap = cv2.VideoCapture("Videos/2.mp4")

pTime = 0

Indent appropriately.

Quick test: Comment out the rest of the code in the module file, and run. It should run without an error

Convert into class

22:30

Make the class, note no () are needed:

class FaceMeshDetector:

Add initialiser

def __init__(self):

Pass parameters (these parameters are the same as those used by FaceMesh(), but we can rename them:

def __init__(self, staticMode=False, maxFaces=2, minDetectionCon=0.5, minTrackCon=0.5):

Add the members

self.staticMode = staticMode
self.maxFaces = maxFaces
self.minDetectionCon = minDetectionCon
self.minTrackCon = minTrackCon

Comment the variables in the code and make members too

self.mpDraw = mp.solutions.drawing_utils
self.mpFaceMesh = mp.solutions.face_mesh
self.faceMesh = self.mpFaceMesh.Facemesh(max_num_faces=2)
self.drawSpec = self.mpDraw.DrawingSpec(thickness=1, circle_radius=2)

Call the FaceMesh using the parameters:

self.mpDraw = mp.solutions.drawing_utils
self.mpFaceMesh = mp.solutions.face_mesh
self.faceMesh = self.mpFaceMesh.Facemesh(self.staticMode, self.maxFaces, self.minDetectionCon, self.minTrackCon)
self.drawSpec = self.mpDraw.DrawingSpec(thickness=1, circle_radius=2)

The DrawingSpec parameters can be made as class parameters as well.

def __init__(self, staticMode=False, maxFaces=2, minDetectionCon=0.5, minTrackCon=0.5, thickness=1, circle_radius=2):

and change

self.drawSpec = self.mpDraw.DrawingSpec(thickness, circle_radius)

Add a method:

def findFaceMesh(self, image, draw=True):

Now uncomment the rest of the code, and put in the method. Fix any indentation issues. Add self.everywhere

Remember no print statements allowed, in the class.

Make the drawing of the facemesh optional

if draw:
    self.mpDraw.draw_landmarks(image, faceLms, self.mpFaceMesh.FACE_CONNECTIONS, self.drawSpec, self.drawSpec)

Add a return, to return the image

return image

To finish the main, to actually use the class, add

detector = FaceMeshDetector()

and

image = detector.findFaceMesh(image)

or

image = detector.findFaceMesh(image, draw=True)

Adding more return values

We still need to add the values of the points in the return statement.

28:50

So, first, make a face list to store the points (by appending). Then, because we have multiple faces, create a faces list and store the face lists, again by appending. This way we don’t ned to enumerate the faces id, in the first face loop – we only enumerate in the landmarks loop (TBH, we don’t even need to enumerate that, as the IDs aren’t returned as we rely on the index of the list instead).

faces = []
if self.results.multi_face_landmarks:
    # for faceId, faceLms in enumerate(results.multi_face_landmarks):
    for faceLms in self.results.multi_face_landmarks:
        if draw:
            # self.mpDraw.draw_landmarks(image, faceLms, self.mpFaceMesh.FACE_CONNECTIONS)
            self.mpDraw.draw_landmarks(image, faceLms, self.mpFaceMesh.FACE_CONNECTIONS, self.drawSpec, self.drawSpec)
        face = []
        # for id, lm in enumerate(faceLms.landmark):  # we don't need to enumerate, as we have list index
        for lm in faceLms.landmark:
            print(lm)
            imageWidth, imageHeight, imageChannels = image.shape
            x, y = int(lm.x*imageWidth), int(lm.y*imageHeight)
            print(x, y)
            # print(id, x, y)
            face.append([x, y])
        faces.append(face)

return image, faces

In main(), change

image, faces = detector.findFaceMesh(image, draw=True)

To test, add in main()

# For test only
if len(faces):
    print(f'Faces: {faces}')

Printing the ids

33:10

In the method, assuming that you are still enumerating the ids,  add

# To print the id next to each point - but this is messy
# cv2.putText(image, str(id), (x, y), cv2.FONT_HERSHEY_PLAIN, 1, (0, 255, 0), 1)

If you use a video with the face fulling the screen it is better. Also, when calling findFaceMesh(), set draw to False to avoid drawn points and lines filling the image.

You could do this using the index of the list, in main()

 

 

 

This is the end, my friend

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s