How can I move mouse by detected face and Eye using OpenCV and Python

Question

I have managed to detect face and eyes by drawing cycles around them and it works fine with the help of Python tutorials Python tutorial & Learn Opencv. Now I would like to make the mouse (Cursor) moves when Face moves and Eyes close/open to do mouse clicking. (PYTHON & OPENCV). Please help to give me more ideas on how I can make it works. Thanks.

Check the codes below:

import cv2
import sys
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml')
video_capture = cv2.VideoCapture(0)
while True:
    ret, img = video_capture.read()
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    Face = face_cascade.detectMultiScale(gray, 1.3, 5)
    for (x,y,w,h) in Face:
         cv2.rectangle(img,(x,y),(x+w,y+h),(0,0,255),2)
         roi_gray = gray[y:y+h, x:x+w]
         roi_color = img[y:y+h, x:x+w]
         eyes = eye_cascade.detectMultiScale(roi_gray)
         for (ex,ey,ew,eh) in eyes:
             cv2.rectangle(roi_color,(ex,ey), (ex+ew,ey+eh), (0,255,0), 2)
   # Display the resulting frame
   cv2.imshow('Face and Eye Detected', img)
   if cv2.waitKey(1) & 0xFF == ord('q'):
        break
#cv2.waitKey(0)
video_capture.release()
cv2.destroyAllWindows()    

Answer 1

If you're working in Windows environment, what you're looking for is the SetCursorPos method in the python win32api. Use:

import win32api
import cv2
import sys
...
for (ex,ey,ew,eh) in eyes:
    win32api.SetCursorPos((ex,ey))
    cv2.rectangle(roi_color,(ex,ey), (ex+ew,ey+eh), (0,255,0), 2)

This will set the cursor to the top-left vertex of the rectangle. If you wish to move the cursor to the center of the rect, use:

win32api.SetCursorPos((ex+ew/2,ey+eh/2))