Detect facial landmarks inside a detected face image using opencv dnn face detector

问题

I trying to detect the 68 facial landmarks of human face. I detected the face using OpenCV dnn face detector as in https://www.pyimagesearch.com/2018/02/26/face-detection-with-opencv-and-deep-learning/ The face detection process done successfully, this is my code:

# import the necessary packages
import numpy as np
import argparse
import cv2
import dlib

ap = argparse.ArgumentParser()
ap.add_argument("-c", "--confidence", type=float, default=0.5,
                help="minimum probability to filter weak detections")
args = vars(ap.parse_args())

# load our serialized model from disk
print("[INFO] loading model...")
net = cv2.dnn.readNetFromCaffe("D:\deep-learning-face-detection\deploy.prototxt.txt",
                               r"D:\deep-learning-face-detection\res10_300x300_ssd_iter_140000.caffemodel")

image = cv2.imread("image\path\jpg")
(h, w) = image.shape[:2]
blob = cv2.dnn.blobFromImage(cv2.resize(image, (300, 300)), 1.0,
                             (300, 300), (104.0, 177.0, 123.0))

print("[INFO] computing object detections...")
net.setInput(blob)
detections = net.forward()

# loop over the detections
for i in range(0, detections.shape[2]):
    # extract the confidence (i.e., probability) associated with the
    # prediction
    confidence = detections[0, 0, i, 2]

    # filter out weak detections by ensuring the `confidence` is
    # greater than the minimum confidence
    if confidence > args["confidence"]:
        # compute the (x, y)-coordinates of the bounding box for the
        # object
        box = detections[0, 0, i, 3:7] * np.array([w, h, w, h])
        (startX, startY, endX, endY) = box.astype("int")

        # draw the bounding box of the face along with the associated
        # probability
        text = "Face#{}".format(i)
        y = startY - 10 if startY - 10 > 10 else startY + 10
        cv2.rectangle(image, (startX, startY), (endX, endY),
                      (0, 0, 255), 2)
        cv2.putText(image, text, (startX, y),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.45, (0, 0, 255), 2)

    # show the output image
cv2.imshow("Output", image)
cv2.waitKey(0)

but when I trying to detect the facial landmarks inside the face, as the following:

predictor = dlib.shape_predictor("D:\shape_predictor_68_face_landmarks.dat")
shape = predictor(image, detections)
vec = []
for i in range(68):
    v = shape.part(i)
    vec.append(v)
print(vec)

I get the following error message

shape = predictor(image, detections) TypeError: call(): incompatible function arguments. The following argument types are supported: 1. (self: dlib.shape_predictor, image: array, box: dlib.rectangle) -> dlib.full_object_detection

Invoked with: , array([[[ 0, 0, 0], [ 0, 0, 0], [ 0, 0, 0], ...,

The error message appeared when I'm using OpenCV dnn face detector, and MTCNN fro dlib face detector, but it cannot be appeared with Haar cascade face detector and the facial landmarks detected successfully. I want to detect the facial landmarks within the OpenCV dnn face detector as the above code due to its accurate where the Haar cascade face detector don't benefit with my face image due to high occlusions consistence. Can anyone please help me.

回答1:

This will solve the problem:

shape = predictor(image,dlib.rectangle(startX, startY, endX, endY))

回答2:

If you want something like this: example You can use face_recognition.face_landmarks(image) function from face recognition library, is much easier to use than the dnn detector. I hope I've helped.