The disappointing static camera

Posted: October 31, 2008 at 11:06 am

I merged the fixes for the histogram method back into the temporal patches. After running the patch overnight it was clear that the static camera is probably not going to work out well. There is just not enough variation in the environment to keep things interesting. The additional problem of the extra overhead (of the high number of sensors) means that an image can only be captured each 2s or so. An approach to revisit would be to use reference frame subtraction to feed a presence image into the SOM. I think part of the lack of interest (and organization) in the raw pixel method is that, with a static camera, the vast majority of the pixels are the same.

I’ll move the histogram patches into trunk and start working with a moving camera. The issue with a moving camera and the dream aesthetic is the relation between one image and another. A way of approaching the smoothness of a dream could be using a slow drunk-walk random movement. That way there would be a portion of the last image in the next image and give some consistency (or at least slow change) of both the images and the histograms. I should store the sequence of the images captured so that a free association could happen both through similarity (the SOM) and through time.

The results of the training using the raw pixel method overnight. There appear to be only two clusters of images. The presence of cars and such do not cluster. (white blocks are units not associated with inputs)

static_camera_raw_pixels.jpg


Histograms

Posted: October 30, 2008 at 4:04 pm

It has been some time since my last post. I was trapped by a number of technical issues that meant I was not even sure my SOM was working properly. Through the debugging process I created two variations of the current system. Both use temporal timing. That is the SOM is not triggered by the motion analysis, but runs at a fixed interval. This was done to simplify the patch as much as possible. The second variation used a histogram in place of the usual pix_dump method (of feeding the raw pixel data to the SOM).

I have learned two things. The first is that the likely cause of the  lack of organization in previous work was due to timing issues. That is the first thing I will go back to fix in other code branches. The second is that the idea of using a fixed camera may be problematic. In my experiments I was able to making a working system using the pix_histo object and passing the histogram to the SOM, rather than raw pixels. This works surprisingly well for a dynamic and moving context (that is it works best if the camera is moving, not a static camera). Of course when using the histogram the system is no longer using the raw pixel data, and therefore not making a pixel by pixel comparison. In orther words the pixel-by-pixel method is most appropriate for the static camera, where the histogram method may be more appropriate for the moving camera. It is clear that the histogram method does not work very well with a static camera.

Histogram method with static camera:

result00000.jpg

Histogram method using a moving camera:

result000001.jpg


Canada Council for the Arts – Production Grants to Media Artists – 2008

Application

Amount Awarded: $36,000