How to align face images c++ opencv

Question

I am developing a C++ application for face authentication. First, I have to detect the face and pre-process the image.

1. For face detection I have used the HaarC

Answer 1

I tried the following face alignment code from the Labelled Faces in the Wild project page. It works really well and does not require detecting facial feature points. The C++ code can be downloaded from: http://vis-www.cs.umass.edu/faceAlignment/

If you still wish to find face key points, I find that the Viola-Jones detector is not very robust and accurate. I personally recommend using the Flandmark face keypoint detector: http://cmp.felk.cvut.cz/~uricamic/flandmark/ which is much much more robust and accurate. C code can be downloaded from the above site.

Answer 2

Detecting misaligned faces make face recognition difficult. Sometimes you want to fix the alignment, sometimes it is sufficient to exclude the ones that aren't aligned correctly (for instance if you're detecting faces in a video stream). I took the latter approach and trained a special Haar Cascade to only detect correctly aligned, well-lit faces. Details here: http://rwoodley.org/?p=417.

If you use my cascade let me know how it works for you. I'm curious what results others would obtain. It met my needs.

Answer 3

Finding the accurate position of the eyes in a given image is far from trivial. The Haar-cascades for finding the eyes in OpenCV produce too many false positive to be useful, moreover this approach won't be robust to image rotation (it may compensate slight rotation though, I don't know the training images). If I were you I'd start a breadth-first search on http://scholar.google.com for relevant papers of this research area.

You'll need a robust head pose estimation for aligning face images. I did some research myself and I think sharing algorithms and code is useful here. The most interesting approaches I have seen are:

• Gary B. Huang, Vidit Jain, and Erik Learned-Miller. Unsupervised joint alignment of complex images. International Conference on Computer Vision (ICCV), 2007. (Project page), (PDF Online available), (Source code)

• X. Zhu, D. Ramanan. Face Detection, Pose Estimation and Landmark Localization in the Wild Computer Vision and Pattern Recognition (CVPR) Providence, Rhode Island, June 2012. (Project page), (PDF Online available), (Source code)

Answer 4

I have implemented it here using OpenCV & DLib: https://github.com/ManuBN786/Face-Alignment-using-Dlib-OpenCV

Any tilted faces can be aligned using my code.

Answer 5

After searching all day for an algorithm to accomplish this with, I found "Face Detection By Finding The Facial Features And The Angle of Inclination of Tilted Face " by Hemlata et al. after switching from Google to DuckDuckGo. It supports faces that are tilted at an angle larger than 45 degrees.

As for how to implement in code, that's another problem that I'm currently working on, but at least this is a starting point.

Answer 6

For face authentication, you can use dlib or face_recognition to do this which is very convenient and more accurate than opencv now.

As to dlib, face alignment can be found here (C++ code) http://dlib.net/face_alignment.py.html

or here(python code)https://www.pyimagesearch.com/2017/05/22/face-alignment-with-opencv-and-python/.

The algorithm paper named Face Alignment at 3000 FPS via Regressing Local Binary Features is realized by dlib.