Open CV and Face Detection

After the problems I had with making my previous camera tests work, I decided to look into other methods of camera interaction that I could use in my sketches. One of the easiest ways to do this would be to look into a computer vision library.

The library has more than 2500 optimized algorithms, which includes a comprehensive set of both classic and state-of-the-art computer vision and machine learning algorithms. These algorithms can be used to detect and recognize faces, identify objects, classify human actions in videos, track camera movements, track moving objects, extract 3D models of objects, produce 3D point clouds from stereo cameras, stitch images together to produce a high resolution image of an entire scene, find similar images from an image database, remove red eyes from images taken using flash, follow eye movements, recognize scenery and establish markers to overlay it with augmented reality, etc.

Itseez (2014)

One of these is Open CV, which is supported by Processing as well as many other different programming languages and environments. It has a range of different computer-vision applications, some of which deal with detecting humans, such as face-detection and skeleton detection algorithms. This would be very useful for my project as many of my concepts deal with interacting with human subjects. I chose to use it because it is well-used and well supported, which means I might be able to find any solutions to any problems I have in developing my sketches on online forums.

Image from LiveCam Test, Open CV examples for Processing

After downloading the library I looked through the examples to see the applications for this library in Processing. One of these was the LiveCam Test which detected people’s faces and drew a green rectangle around them. The code it uses to do this is below:

  // detects faces and adds them to a rectangle array
  Rectangle[] faces = opencv.detect();
//goes through all the faces and draws a rectangle at the borders
  for (int i = 0; i < faces.length; i++) {
    rect(faces[i].x, faces[i].y, faces[i].width, faces[i].height);

Through editing this code I hoped to to create an effect similar to what I hoped for in my previous sketches, by substituting part of the example sketch with a Attractor class from my Gravity sketches. In this instance, I was hoping to create an instance of an Attractor class whenever a face is detected, instead of a rectangle.


Thankfully, after learning more about error testing in my previous tests, it was fairly easy to combine the two sketches. The sketch was far easier to control than my previous tests, as it did not rely on an average location of motion. It would also work better in a busy environment as more than one face can be detected at once, whereas previous sketches only had one object that could be manipulated.


I continued to experiment more with the sketch in order to improve it. I removed the rectangle from in front of the faces, as as I learned in my museum observations, people reacted better to video interactions when they could clearly see images of themselves. I also used the Processing commnand rectMode(CENTER); in order to make the Mover objects be attracted to the center of the faces. This was not a problem in previous sketches as the area of the Attractor objects was quite small, however in this sketch, as the faces can be quite near the camera they can be big. Therefore the Mover objects being attracted to the corner of the faces was quite noticeable and this necessitated this change.

Again there were problems with the sketch being quite slow, and I will have to try and optimize the algorithms of the sketch further so it will run faster. However I feel that the using this sketch as a base for my project will be far more effective than previous tests. I will need to test this in a variety of environments to make sure the facial detection algorithm works well in different spaces. I would also like to try experimenting with different types of Mover objects as I am finding the circles quite boring visually, so I am going to experiment with code and do some more theoretical research to try and find a more complex visual solution.

Itseez, 2014. Open CV: About Open CV. Available from: [Accessed 12.12.14]