Colour Fields and Neighbourhood Size

Posted: April 17, 2016 at 1:08 pm

The following images show visualizations of the SOM’s structure. The visualization is composed of rectangles where their colour, width and height correspond to the segments association with that location. The segments themselves are shown underneath the visualizations.

still-proxy-pano-edit-genSOMVizCV-5_500-SOM-10000-SOMScale-36-sigma1 still-proxy-pano-edit-montageCV-5_500-SOM-10000-SOMScale-36-sigma1 (more…)


Training with Small Neighbourhood and Whitespace

Posted: April 16, 2016 at 11:13 am

I think I have code working where the initial neighbourhood size (the number of neurons that are updated for each training step) starts off being very small (in this case default/20). The idea is to use the neighbourhood size such that the image becomes increasingly self-organized from the bottom to the top. In the first image below, only 1000 iterations of training are done. There is an interesting deconstruction of the image from the initial conditions (seeded from the original panorama).

still-proxy-pano-edit-montageCV-5_500-SOM-1000-SOMScale-36-sigma20 (more…)


No training with 36px blocks

Posted: April 14, 2016 at 11:17 am

Following from my previous post the following image and details show the level of fragmentation using blocks of 36px. This corresponds to the SOM being 36 times smaller than the original pano. I’m currently doing a 20px run (1421×306 lattice), but its proving to be very slow. Even though I’m not doing any training, getting the BMUs for each segment is extremely slow, ~4 hours for the 36px blocks. In the image below I use a gradient alpha mask to fade between the original panorama at the bottom and the lattice arranged segments.

still-proxy-pano-edit-montageCV-5_500-SOM-noTraining-SOMScale-36 (more…)


Initial Arrangement of Segments without Training

Posted: April 13, 2016 at 6:50 pm

I found a bug in the code such that the initial conditions of the network were not properly corresponding with the original pano. The following image shows a untrained map where the features of segments in each 100 px square block are averaged and then the closest region is presented in that location. The top image shows these segments in each block’s position. The image below is the same, but with the original pano underneath it and a detail. As this is the least fragmented the Cartesian SOM can represent, I’m now running a pass with smaller blocks and we’ll see how that looks.

still-proxy-pano-edit-montageCV-5_500-SOM-noTraining-noPano still-proxy-pano-edit-montageCV-5_500-SOM-noTraining (more…)


First full resolution collage

Posted: April 3, 2016 at 2:25 pm

The following images are the result of a 789×170 unit SOM where the initial weights are determined by the original panorama source. The training was done without modification, thus large initial neighbourhoods obliterate much of that initial structure. The number of units in the SOM make the segments too broadly distributed with large gaps between segments. The idea is to control the learning rate and neighbourhood size in the modified training routine such that the segments are located near where they are in the pano at the bottom and become increasingly self-organized at the top. I’ll try a smaller SOM next so there is hopefully more overlap between neighbouring segments.

testMontage-noPano

testMontage-pano


Visualization of initial weight vectors

Posted: April 2, 2016 at 11:42 am

The following image shows the initial weights I will use to train the SOM next. It just shows colour information, but the feature vectors also hold width and height of the segments. In grid locations that contain more than one segment, the feature vectors of those segments are averaged. For now, feature vectors for cells without any segments are set to zero (and appear black below).

debug


Centre points of all segments

Posted: April 1, 2016 at 2:10 pm

The following image is a plot of the centre positions of all ~45,000 segments from the original panorama. I’ll use this to set the initial SOM weights.

pano-segment-points


SOM Arranged Compositions!

Posted: March 26, 2016 at 3:26 pm

I’ve finally got things working such that the arrangement of segments reflects their structure as learned by the SOM; the lack of such structure in previous results were due to a bug in ANNetGPGPU. The following images are all the result of a 100,000 iteration training period with random initial conditions. The last three images are details of the second image.

still-proxy-pano-edit-montage-5_500-1_0.5-SOM-100000 (more…)


Using a Self-Organized Map to arrange composition.

Posted: March 17, 2016 at 9:12 pm

The following images show some very early work using a SOM (implemented in ANNetGPGPU) to arrange the montages of segmented regions. Rather than sorting according to single parameters, e.g. colour, the composition should reflect the similarity of regions across all parameters. In the first image, I used the wrong segment directory where I did not tweak the overexposed sky. The result is a number of very large white regions at the bottom. While all regions are clearly organized according to their colour parameters, the clusters don’t look right. I think I messed up some of the code mapping the 1D BMU IDs with 2D positions in the SOM lattice.

still-proxy-pano-edit-montage-5_500-1_1-SOM-100 (more…)


Collage Explorations of Panorama Segments

Posted: March 3, 2016 at 11:46 am

The following collage explorations use the same code used to generate collages during my Banff Centre Residency, where the segments are extracted from the pano previously posted. These images are composed of over 230,000 individual segments, but the very large regions from the monochromatic sky are filtered out. The images without mattes are details of the collages immediately above them. Note that these are quite low resolution as the original segments are so small. I’ve also extracted segments from a more coarse segmentation and I’ll generate some collages from those this afternoon.

still-proxy-pano-edit-montage_1_0.125-filter_10000

(more…)


Early Segmentation Results

Posted: February 29, 2016 at 8:47 am

The following image is a crop from the segmented panorama previously posted. The computer is still extracting pixels from the original image using these segmented regions, where the current count exceeds 50,000.

segmented-scale

(more…)


Public Art Commission

Posted: February 27, 2016 at 4:57 pm

I’m working on a public art commission for the City of Vancouver and I thought it would be a good opportunity to revisit the Self-Organized Landscapes series. For this project, the idea is to combine both a Self-Organized Landscape and a panoramic photograph such that a photo-realistic and readable image dissolves into abstraction as informed by a Self-Organized Map (SOM). The first step was to shoot a ‘straight’ photographic proxy image. The following is a pano constructed using hugin from ~45 exposures where the full resolution is ~30,000 pixels across.

still-proxy-pano-edit-scaled (more…)