How can I move mouse by detected face and Eye using OpenCV and Python

问题

I have managed to detect face and eyes by drawing cycles around them and it works fine with the help of Python tutorials Python tutorial & Learn Opencv. Now I would like to make the mouse (Cursor) moves when Face moves and Eyes close/open to do mouse clicking. (PYTHON & OPENCV). Please help to give me more ideas on how I can make it works. Thanks.

Check the codes below:

import cv2
import sys
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml')
video_capture = cv2.VideoCapture(0)
while True:
    ret, img = video_capture.read()
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    Face = face_cascade.detectMultiScale(gray, 1.3, 5)
    for (x,y,w,h) in Face:
         cv2.rectangle(img,(x,y),(x+w,y+h),(0,0,255),2)
         roi_gray = gray[y:y+h, x:x+w]
         roi_color = img[y:y+h, x:x+w]
         eyes = eye_cascade.detectMultiScale(roi_gray)
         for (ex,ey,ew,eh) in eyes:
             cv2.rectangle(roi_color,(ex,ey), (ex+ew,ey+eh), (0,255,0), 2)
   # Display the resulting frame
   cv2.imshow('Face and Eye Detected', img)
   if cv2.waitKey(1) & 0xFF == ord('q'):
        break
#cv2.waitKey(0)
video_capture.release()
cv2.destroyAllWindows()    

回答1:

If you're working in Windows environment, what you're looking for is the SetCursorPos method in the python win32api. Use:

import win32api
import cv2
import sys
...
for (ex,ey,ew,eh) in eyes:
    win32api.SetCursorPos((ex,ey))
    cv2.rectangle(roi_color,(ex,ey), (ex+ew,ey+eh), (0,255,0), 2)

This will set the cursor to the top-left vertex of the rectangle. If you wish to move the cursor to the center of the rect, use:

win32api.SetCursorPos((ex+ew/2,ey+eh/2))

回答2:

Use pyautogui module for accessing the mouse and keyboard controls . The pyautogui is very simple to learn and the documentation is the best source no need to see any type of videos on that. Next you need to detect the white area of the eyes(corenia may be) using the contoursArea method available in open cv. When u see towards left the white area of right side of eye increases, and when the white area increases then the mouse must move left by function available in pyautogui that is pyautogui.moveRel(None,10)

来源：https://stackoverflow.com/questions/40354074/how-can-i-move-mouse-by-detected-face-and-eye-using-opencv-and-python