First crack at image perception/synthesis

Posted: February 23, 2010 at 5:38 pm

The perception/synthesis project is a component of the next “Dreaming Machine” installation. In DM1 and 2 entire images were stored, not components of images. The purpose of this project is to determine a method for extracting components from an image, but without a complex shape recognition system.

The proposal is to extract components using background subtraction and to crop images using the bounding boxes of OpenCV contours. This prototype system will use a static camera. The background reference image will be an accumulation of images over time, where the static background would have emphasis over moving objects. The benefit of this installation context is that many images can be used in the construction of the reference image. The following is the reference image as used in this initial test case:

background.jpg

The image was constructed from 50 images, some with a static background and some without.

From this set eight images were selected that contain objects that differ from the background. These two sets are as follows:

balls.jpg

coasters.jpg

Contours were extracted using an absolute difference between foreground images and reference images. These difference images were processed in OpenCV to generate bounding boxes. The bounding boxes were summed to determine the size and centre of the collection of contours. This was done because the contours were too complex for my use. This introduces a problem if extracting multiple objects from one image.

The input images were cropped according to the resulting bounding boxes and accumulated into single images. Just as the background image is the result of the reinforcing effect of the static elements, this cropped accumulation reinforces the foreground objects through their central location and scale. The balls worked quite well, but the complex pattern of the coasters resulted is a less cohesive accumulation.

ball.jpg

coaster.jpg

The next step will be to collect more images to see if accumulation leads to cohesive representations of foreground objects.