Day 12: OpenCV--Face Detection for Java Developers

Today for my 30 day challenge, I decided to learn how to do face detection using the Java programming language. Face detection helps in recognizing human faces in arbitrary (digital) images. After doing some research, I discovered that the OpenCV library can help me detect faces in an image. However, I wasn't able to find a complete beginner tutorial for using OpenCV library with Java, so this blog might help others looking for an introductory text on this subject.

OpenCV logo

What is OpenCV?

OpenCV(Open Source Computer Vision) is an open source computer vision algorithms library. It is written in C/C++, and is designed to take advantage of multiple cores. It provides interfaces for the C++, C, Python and Java programming languages and supports all the major operating system platforms including Windows, Linux, Mac OS, iOS, and Android.

Github Repository

The code for today's demo application is available on github: day12-face-detection.

Getting Started with OpenCV

In order to get started with OpenCV, the first step is to download the latest OpenCV package for your operating system from the official website. In this blog, I will be using the 2.4.7 version.

After downloading the package, extract the package using tar command.

$ tar xvf opencv-2.4.7.tar.gz

Change to the directory to opencv-2.4.7

$ cd opencv-2.4.7

Build OpenCV jar

It took me lot of time to understand how to get the OpenCV jar file. The Java tutorial in the documentation assumes the OpenCV jar file is in the build folder. The tutorial works for windows users as the OpenCV package for windows includes the jar file. This is not true for Linux and Mac OS users. To build the OpenCV jar, executes the command shown below.

$ cd opencv-2.4.7
 
$ mkdir build
 
$ cd build/
 
$ cmake -G "Unix Makefiles" -D CMAKE_CXX_COMPILER=/usr/bin/g++ -D CMAKE_C_COMPILER=/usr/bin/gcc -D WITH_CUDA=ON .. 
 
$make -j4 
 
$ make install

The above commands will create the opencv-247.jar file in the opencv-2.4.7/build/bin directory. This is the Java binding to the native OpenCV installation.

Download Eclipse

If you do not have Eclipse installed on your machine, download the latest Eclipse package for your operating system from the official Eclipse website. At the time of writing this blog, the latest Eclipse package is called Kepler.
It is very easy to install eclipse, just extract the downloaded package and you are done. On linux or mac machines, open a new command line terminal and type command shown below.

$ tar -xzvf eclipse-jee-kepler-R-*.tar.gz 

On windows you can extract the zip file using winzip or 7-zip or any other software. After you have extracted Eclipse, there will be a folder named eclipse in the directory where you extracted the zip file. You can optionally create a shortcut of the executable file.

Add the User Library

Open the Eclipse IDE and then navigate to the project workspace. Go to Windows > Preferences > Java > Build Path > User Libraries, and select to add a new library.

OpenCV Add New User Libraries

Give the user library a name like OpenCV-2.4.7 and then press OK.

OpenCV User Libraries

Click on Add External Jars, and add the opencv-247.jar file.

Add External Jar

Select the Native library location and then press Edit.

Edit Native Library

Click on External Folder

Add External Library

Give the location of lib directory under the opencv-2.4.7/build/lib folder.

Add Lib directory

Finally, press OK. Now we have added OpenCV as the user library.

Create New Java Project

Create a new Java project by going to File > New > Other > Java Project. After the project is created, right click on the project and configure the build path.

Configure Project Build Path

Go to the Libraries tab, and click on Add Library.

Add Lib directory

Select the User Library

Select User library

Choose the OpenCV-2.4.7 user library we added in last step and click Finish.

Select OpenCV User Library

Finally, the Java project will include the OpenCV-2.4.7 user library.

Project with OpenCV user library

Write FaceDetector

Create a new class in the java project we created in the last step and add the following code.

package com.shekhar.facedetection;
 
import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.core.MatOfRect;
import org.opencv.core.Point;
import org.opencv.core.Rect;
import org.opencv.core.Scalar;
import org.opencv.highgui.Highgui;
import org.opencv.objdetect.CascadeClassifier;
 
public class FaceDetector {
 
    public static void main(String[] args) {
 
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
        System.out.println("\nRunning FaceDetector");
 
        CascadeClassifier faceDetector = new CascadeClassifier(FaceDetector.class.getResource("haarcascade_frontalface_alt.xml").getPath());
        Mat image = Highgui
                .imread(FaceDetector.class.getResource("shekhar.JPG").getPath());
 
        MatOfRect faceDetections = new MatOfRect();
        faceDetector.detectMultiScale(image, faceDetections);
 
        System.out.println(String.format("Detected %s faces", faceDetections.toArray().length));
 
        for (Rect rect : faceDetections.toArray()) {
            Core.rectangle(image, new Point(rect.x, rect.y), new Point(rect.x + rect.width, rect.y + rect.height),
                    new Scalar(0, 255, 0));
        }
 
        String filename = "ouput.png";
        System.out.println(String.format("Writing %s", filename));
        Highgui.imwrite(filename, image);
    }
}

The code shown above does the following :

  1. It loads the native OpenCV library so that we can use it using Java API.

  2. We create an instance of CascadeClassifier passing it the name of the file from which the classifier is loaded.

  3. Next we convert the image to a format which the Java API will accept using the Highui class. Mat is the OpenCV C++ n-dimensional dense array class.

  4. Then we call the detectMultiScale method on the classifier passing it the image and MatOfRect object. After processing, the MatOfRect will have face detections.

  5. We iterate over all the face detections and mark the image with rectangles.

  6. Finally, we write the image to the output.png file.

The output of the program is shown below. This is my pic before and after face detection.

OpenCV Output

That's it for today. Keep giving feedback.

What's Next

Hello! Thank you for your given lecture, but i have some troubles with NullPointerException. In the line where we wrote:

CascadeClassifier faceDetector = new CascadeClassifier(FaceDetector.class.getResource("haarcascade_frontalface_alt.xml").getPath());

How can i fix it? Should move haarcascade_frontalface_alt.xml from initial openCV's directory to my project's folder? What is the poblem?

Hope for you soonest respond

Medet

The reason is that haarcascade_frontalface_alt.xml does not exist at the required location. It should be place inside the package where you have your face detection class. Look at the source code here https://github.com/shekhargulati/day12-face-detection

Thanks a lot) BTW, i got another question. This program creates png picture but gives warning that widht, height is zero. And output image is 0 sized and empty. What may be the problem?

Thank you for your response!

Where is the answer, Mr. Gulati? The program creates png picture but gives warning that width, height is zero. And output image is 0 sized and empty. What may be the problem?

Can you share your code i.e. github repository or email me at sgulati@redhat.com? It looks like it is not reading the input image.

I have mailed you, Mr. Gulati! Please check it out. Hope for you fast answer.

Sir, I am also facing the same oroblem that the width and height of the output image is zero. I have already mailed you the code.

Hoping for soon replies.

Thanks in advance.

output: Running FaceDetector Detected 0 faces Writing ouput.png libpng warning: Image width is zero in IHDR libpng warning: Image height is zero in IHDR libpng error: Invalid IHDR data

same problem arised for me too!

Because your operating system is windows, relative file path is wrong, try using an absolute path

For me, at least,

FaceDetector.class.getResource("haarcascade_frontalface_alt.xml").getPath()

returns something like

/C:/src/open/haarcascade_frontalface_alt.xml

Note the slash at the start. As a non-portable workaround, you can modify the call to be

FaceDetector.class.getResource("haarcascade_frontalface_alt.xml").getPath().substring(1)

Hello Guys,

I can not help if you just post the message. I need code to debug the problem. The way you should work is :

  1. Fork the repository. Install git on your operating system. Create a github account and fork the https://github.com/shekhargulati/day12-face-detection.

  2. Then make your changes and send me your repository url.

Please do not send me your application code. Just send me the code(github repository url) related to FaceDetection. This code works and I have tested it 100 times. The issue might be related to user library configuration.

Thanks

Shekhar

helloo friends ı have a problem and ı do not know why this happens everytime ı need your help about face detection. ı am triyng to execute opencv face detection sample but ı have Android NDK: WARNING:jni/Android.mk:detection_based_tracker: non-system libraries in linker flags: -lopencv_java
Android NDK: This is likely to result in incorrect builds. Try using LOCAL_STATIC_LIBRARIES
Android NDK: or LOCAL_SHARED_LIBRARIES instead to list the library dependencies of the
Android NDK: current module
[armeabi-v7a] Install : libdetection_based_tracker.so => libs/armeabi-v7a/libdetection_based_tracker.so this problem and apperantly all paths are correct ı need help to be able to improve code ı will appreciate if it will be solved thanks in advance :)

$make -j4

There should be a space between $ and make.

wow fantastic any suggestion or lecturing for face tracing

For windows users who get the following error:

libpng warning: Image width is zero in IHDR  
libpng warning: Image height is zero in IHDR  
libpng error: Invalid IHDR data 

Trevie is right, you need to remove the slash from the beginning of the path but for ALL resources i.e. "haarcascade_frontalface_alt.xml" AND "shakhar.JPG"

Something like this:

CascadeClassifier faceDetector = new 
                   CascadeClassifier(FaceDetector.class.getResource("haarcascade_frontalface_alt.xml").getPath().substring(1));

and

Mat image = Highgui
                .imread(FaceDetector.class.getResource("shekhar.JPG").getPath().substring(1));

Hope this helps

I tried this but nothing will happen so please tell suggestion to overcome from this problem.

Exception in thread "main" java.lang.UnsatisfiedLinkError: no opencv_java248 in java.library.path at java.lang.ClassLoader.loadLibrary(Unknown Source) at java.lang.Runtime.loadLibrary0(Unknown Source) at java.lang.System.loadLibrary(Unknown Source) at FaceDetection.main(FaceDetection.java:12)

i am doing this in windows 8 and same error in netbeans and eclips both help me please.

hello I am getting a error : libpng warning: Image width is zero in IHDR
libpng warning: Image height is zero in IHDR
libpng error: Invalid IHDR data

i am not using any slash before file name:

CascadeClassifier faceDetector = new CascadeClassifier(FaceDetector.class.getResource("haarcascade_frontalface_alt.xml").getPath()); Mat image = Highgui .imread(FaceDetector.class.getResource("shekhar.JPG").getPath()); So help me how to overcome this problem