Training with Small Neighbourhood and Whitespace

Posted: April 16, 2016 at 11:13 am

I think I have code working where the initial neighbourhood size (the number of neurons that are updated for each training step) starts off being very small (in this case default/20). The idea is to use the neighbourhood size such that the image becomes increasingly self-organized from the bottom to the top. In the first image below, only 1000 iterations of training are done. There is an interesting deconstruction of the image from the initial conditions (seeded from the original panorama).

still-proxy-pano-edit-montageCV-5_500-SOM-1000-SOMScale-36-sigma20

The image below shows a mock-up using gradient masks to blend the original panorama (emphasized at the bottom), the fully trained SOM and the partially trained SOM with small neighbourhood (both emphasized at the top).

still-proxy-pano-edit-montageCV-5_500-SOM-1000-SOMScale-36-sigma20-mockup

This is a pretty good representation of where I’m trying to get, but I’m still unhappy with the sparsity of the segments approaching the top. This sparsity is due to the number of cells in the SOM itself, where there is a single segment for each 36px block. Due to the different sizes and aspect ratios of the segments, there is significant white space. I would like the top/sky to look more like the first SOM arranged compositions, which are dense because the resolution is quite low and the segments are overlapping.

I can’t simply reduce the resolution, because the final image is large scale, and I’d rather keep the region pixels 1 to 1 to maintain quality. One approach to fill in some of that white-space is to create a layer where the SOM structure itself is visualized. Right now each SOM cell is represented by its nearest segment, but I could use the structure of each cell (neuron weight) to generate a non-photographic image. This would mean taking the features (width, height, and HSV colour) and drawing a rectangle for each neuron. As there would be no photographic component these could be rendered at any size and thus easily fill in the white space between segments.

The concept for the work is that the structure of the image at the bottom resembles reality;  as we move up the image, the structure becomes increasingly abstract and determined by the structure of the image as seen by the SOM. The SOM arranged composition represents our projection of structure onto reality and how we infer and predict structure in our perceptions. Visualizing the SOM itself (and not the photographic segments associated with its structure) seems to be a natural extension of this. By visualizing the SOM itself, the viewer would be able to see what the SOM has actually learned and how it ‘sees’ individual segments.