Live SOM installation

Posted: June 30, 2009 at 10:53 pm

After exhibiting the high-resolution SOMs I’ve become very interested in their aesthetic. I’ve come up with an idea for system that trains a map using the robot camera. The solution to the problem of the finite number of images is solved by having a temporal aspect to the installation. The camera would capture images, and the SOM would be trained. The exhibition would start with a blank slate before the SOM is trained, and over time the SOM would be refined more and more. Once the SOM is trained the exhibition would end, or perhaps remain for a time in a static state.

Something like the dual SOM would likely be needed as the training would work best if the SOM is presented with the same images and in random order. The first SOM would store the frames which the second SOM would randomly iterate over in training. Another idea would be to simply have the SOM trained slowly with the live images from the camera. Even if the camera took an image every 3s, it would take at least 70 days to train a 70×70 SOM. Then act of capturing images would be more closely connected to the training process. As each image is captured it would immediately be presented on the display.

The display could be an array of smaller screens, perhaps 4-9 inches diagonal.  Each screen would show one single image at full frame. One problem with this approach is that the frames would take too much visual emphasis and distract from the structural relationships between the images. Controlling as many as 40×40 or 70×70 units would be a technical difficulty.

Another idea would be to have multiple projectors each showing a portion of the whole image, 4×4 projectors (each showing 10×10 or 15×15 units), or 8×8 projectors (each showing 5×5 or 10×10 units ), depending on resolution. All the projectors would be projected on the back of a single rear-projection surface. The strength of this method is that the Gaussian masks could still be used, multiple projectors would blend together very well, and would be more technically simple. The resulting image would be extremely high resolution and still be using live content from the camera. Even the brightness of the space would be less of an issue as the projectors would likely be close to the projection surface and therefore brighter.

Even with cheaper (non-HD) projectors an 8×8 grid could create a resolution as high as 10,240×8,192 pixels which would allow the viewer to really see the details in each individual image as well as the overall structure.

SOM Video

Posted: June 30, 2009 at 8:40 pm

The idea is to take a sequence, video or film. Take the first 100 (n) frames and train a SOM on those frames. That trained SOM is the first frame of the output sequence. Then take the 100 (n) frames starting at frame #2, and so on. The resulting sequence would have 100 (n) frames less than the input sequence. The result would show the structure over the last 100 (n) frames. A new scene would start a new cluster and grow in the field until it shrinks as the scene ends and is displaced by the next scene.

The number of frames used would define the number of clusters in the feild and the number of units in the frame. HD+ resolution would likely be needed as each frame would be composed of many frames from the original footage.

“Dreaming Machine #2” 2009

Video Documentation (Hong Kong) (10MB)
Video Documentation (São Paulo) (183MB)
Dream Sequence (Montréal) (83MB)
Selection of Dreams from Hong Kong, São Paulo and Montréal (340MB)


Can a machine dream? (more…)

Post Hong Kong

Posted: June 11, 2009 at 6:49 pm

Even though I had to remove the code to save images from the installation, somehow I was able to save the memory field when striking the installation. The sawtooth method for training seems to work quite well, there is certainly a cluster of darker and lighter images. Many local errors though, clear in the number of colours floating around. This may be due to there simply being not enough iterations, this was only a two week exhibition. Here is the whole 40×40 memory field:

Dreaming Machine #2 montage after Hong Kong exhibition

A detail:

Dreaming Machine #2 Memory Field Detail Hong Kong exhibition