Video Version

As previously discussed, I found there was too much variation in the process that is not manifest in the final still image. I’ve thus made a video loop that shows the gradual deconstruction of the photographic image.

Title and Project Description

As our gaze peers off into the distance, imagination takes over reality

The relation between the world as we see it and the world as we understand it to be is all wrapped up in abstraction, the act of emphasizing some details in order to dispense with others. My artistic interest in abstraction and computation is an interest in the world as it is versus the world as we read into and conceptualize it. As our gaze peers off into the distance, imagination takes over reality… is site-specific public artwork that uses a photographic image of the surrounding site as raw material. The bottom of the image shows this imperfect photographic illusion, a proxy of objective reality, that becomes increasingly abstracted towards the top. The abstraction is a proxy for the constructive powers of imagination and is the result of a machine learning process that rearranges colour values according to their similarity. The imaginary structure presented in this image is both an emergent composition and an exposition of the implicit structure of reality. The image is a representation of place (as a physical location at the site of installation) and non-place (as an ambiguous and emerging structure under constant renewal). The viewer’s experience of the tension between ambiguity and reality mirrors the mind’s constant attempt to bridge experience and truth.

Final Design

good_result_24mm-final-small

Above is the final design for the QE Plaza. I’ve removed most of the white padding to preserve the aspect ratio of the image to match that required by the installation location. I’ve included details of the full resolution version below. I’m quite happy with artifacts due to sub-sampling used to progressively smooth the transition between the full resolution pano and the 20px per cell SOM. This leads to a quite nice tilt-shift effect that contributes to the tension between reality and imagination.

good_result_24mm-final-detail-1

Read more

Going with the 20px cell size.

The 10px cell test with a small number of iterations (1,000,000) is shown below. While the spires are reduced they are still present, especially at full resolution. In the scaled image below, the most noticeable one is to the left of the leftmost set of red benches.

good_result_24mm-SOMScale10_h3775_n30e_i1000000

I also tired using the 7px cell test (below) with very few iterations (top: 500,000 and bottom: 250,000) and larger max neighbourhood (30²) to reduce spires; unfortunately, the lack of training shows a lack of organization compared to the results documented in this post. So I’m going to stick with the previous 20px 5,000,000 training iteration version and spend some additional time working on the transition.

good_result_24mm-SOMScale7_h3775_n30e_i500000 good_result_24mm-SOMScale7_h3775_n30e_i250000

Focusing in…

After some time and consideration, I’ve decided although the spires fit with the theme of imagination and emergence they are just too visually dominant. I’ve thus been exploring using the 20px cell size (that avoids spires) with a lower horizon:

good_result_24mm-SOMScale20_h3775_n20e_i5000000

Read more

Exponential Increase of Neighbourhood Size

I’m still tinkering with the code and the central issue is that I’d like to get better coverage of the area near the horizon, which seems to mean more than 5,000,000 iterations of training, and thus an increase of cell size. The following images show some of these explorations. I’ve also used an even more pointed Gaussian to soften the boundaries.

good_result_24mm-SOMScale10_h4615_n15e_i5000000 good_result_24mm-SOMScale20_h4615_n15e_i5000000 good_result_24mm-SOMScale10_h4615_n150_i5000000

Read more

5 Million Iterations

In order to investigate those spires I tried running 5 million iterations of training with a smaller network (10px per cell rather than 7px previously used). The result is quite interesting, but the neighbourhood size is too large, causing the sky to dominate significantly. There is still a tendency for clusters to grow in the upper left direction (causing spires) that I cannot explain. The light-coloured spires in the previous post originate at pixels with high degrees of brightness surrounded by darkness, but in the image below we can see they are also present in lower contrast areas (e.g. the sky).

good_result_24mm-SOMScale10_h4615_n100_i5000000

Spires

The difference between the two following images is a small change in the neighbourhood function (larger on top). I can’t explain the emergence of these beige spires; they emanate from very small areas of the same colour in the original image, but I’m not sure what drives their expansion into the sky. They seem to expand increasingly according to the number of training iterations. I’m doing another run with a larger neighbourhood and see what happens.

good_result_24mm-SOMScale7_h4615_n140_i2000000

good_result_24mm-SOMScale7_h4615_n120_i2000000

Home Stretch!

good_result_24mm-SOMScale7_h4615_n140_i200000

I found a bug in ANNetGPGPU that resulted in the neighbourhood function having a hard fall-off rather than a gradual Gaussian decay. The result really effected the final aesthetic even after millions of iterations. I also changed from HSV to RGB colour model, which was causing some artifacts. The image above shows a short training session to make sure things are going in the right direction. Extra thanks to Daniel Frenzel for the very fast turnaround fixing the bug!

Almost there…

The following images show the latest iteration of the work. In this version the SOM is blended with the original pano, and I’ve moved the ‘horizon’ higher up. The transition between the pano and the lower resolution SOM (7px per cell) is quite nice and leads to a bit of a tilt-shift effect at the ‘horizon’. See the detail image below. I’m not entirely keen on the degree to which the blue sky over-powers the buildings, so I’m now rendering a version with a smaller max neighbourhood and we’ll see what that looks like tomorrow.

Bogart-Platforms-Coastal-City-In-Progress Bogart-Platforms-Coastal-City-In-Progress-Detail

First results using new panorama!

The following images show initial results using the new panorama. The SOM scales are 10 and 20 (meaning 10 and 20px per SOM cell). These differing scales mean the maximum neighbourhood size (degree of abstraction at top) is double in the top image.

good_result_24mm_SOMScale20_h4263 good_result_24mm_SOMScale10_h4263

“Final” Panorama!

I’ve been very busy with other projects, including a big exhibition of Watching and Dreaming (2001: A Space Odyssey) at the Digital Art Biennial in Montreal this month. I had already shot two attempts at a “final” panorama, but have been having significant issues stitching the images together. I finally have a suitable result, but it uses the wider images, which limits the resolution to ~60,000 pixels, which is short of the 80,000px target:

good_result_24mm

Read more

Results without Segmentation

The following images show the results of training a SOM using the colour values of the original pano pixels and bypassing the segmentation processes. The result is that performance is improved (a 5px cell is possible), and the images are quite interesting, although the white sky dominates the image, where it was previously ignored as it was seen as a single giant segment and ignored.

SOMResults_noSegmentation_SOMScale5_h1352_ns400_i1000000-scaler8SOMResults_noSegmentation_SOMScale5_h1352_ns200_i1000000-scaler8

Read more

Initial Designs

Based on the explorations up to this point, I wanted to post some initial designs using the Gaussian renderer and blending with the original panorama at a more carefully specified horizon. The following images show the results with varying multipliers applied to the Gaussian functions used to draw SOM cells. If you look really closely, you may be able to see that one of the red benches dissolves into a plume of red. A person sitting on one of the red benches on the left turns into thick black smoke.

SOMResults-5_500-tweaked-SOMScale-10_1000000-sigma400-AVG-h1352-scaler0.25 SOMResults-5_500-tweaked-SOMScale-10_1000000-sigma400-AVG-h1352SOMResults-5_500-tweaked-SOMScale-10_1000000-sigma400-AVG-h1352-scaler2

Read more

Exponential vs Linear

The following images show exponential and linear (respectively) increases of neighbourhood size where the maximum neighbourhood size at the top is the same (1600). They don’t seem all that aesthetically different, nor does there appear to be any smoother a transition at the horizon. The linear version is slightly more interesting (and cosmological).

SOMResults-5_500-tweaked-SOMScale-20_1000000-expSigma40-AVG SOMResults-5_500-tweaked-SOMScale-20_1000000-sigma1600-AVG

Read more

Max Neighbourhood Size and Averaged Top Row

After making the decision to seed the top row with averaged values I executed a few runs using a larger network (20px for each SOM cell) and a range of maximum neighbourhood sizes (10, 50, 100, 500, and 1000). The last is so large that it nearly covers the entire width of the map (~1400 cells). Following are a few rectangle renderings of the result. Note, I used too large a scaler, so each cell is drawn quite large. I think this explains the large rectangles at the bottom of the image, but that needs more investigation.

SOMResults-5_500-tweaked-SOMScale-20_1000000-sigma10-AVG SOMResults-5_500-tweaked-SOMScale-20_1000000-sigma50-AVG

Read more

Random vs Averaged Seed Structure on Top Row

Seed data on top row is random:

Seed data on top row is calculated from the average of the segments in each column:

I wanted to follow up on some of the conceptual aspects I touched on in my previous post. When I started my Masters degree I was really interested in how I could create a computational process that was not random nor a perfect reflection of my intention nor the outside world. In general, I try to use randomness as sparingly as possible. In my Masters project I, as best as I can recall, did not use any randomness at all, making the system deterministic. Since it used images from its environment (that was constantly changing) it was unpredictable, and yet deterministic.

Read more

Mock-up with Offset

In this version, there is an offset applied such that the neighbourhood size only starts increasing at the horizon, rather than the bottom of the image. A mock-up blended with the original image is also provided below.

genSOMVizCVBatch_sigma1-40+30

genSOMVizCVBatch_sigma1-40+30-mockup

Neighbourhood Sequence Ranging from 1-30

In the following images (raw on top, original overlap on bottom) I used a smaller range of neighbhourhood sizes compared to the previous post (1-30). I was hoping the buildings would be less obliterated, and I actually prefer some of the horizontality of the previous 1-40 version. The reason why I’m unhappy with these results is that the visually interesting structure only begins on the horizon, and thus the abstraction should only start at that point.

genSOMVizCVBatch_sigma1-30 genSOMVizCVBatch_sigma1-30-mockup

Collage and Video from New Colour Field Sequence

From the previous posts, I’ve restricted the range of neighborhood sizes from 1-170 to 1-40 in the following image and video. The more gradual increase of neighborhood size increases the cohesion of the sequence and I’m quite content with these results, although I may want it even more gradual.

genSOMVizCVBatch_sigma1-40

Collage of multiple colour fields.

The following image is constructed by taking a single row from each of the trained networks such that the lowest (bottom) row has the smallest neighbourhood size (1) and the highest (top) row has the largest (170). Note that very little of the image is readable because the image is abstracted very quickly as the neighbourhood size increases in steps of 1. Note some stability in structure (position of colours) near the bottom of the image. Near the top the increasing horizontality indicates large differences in structure in subsequent neighbourhood sizes.

genSOMVizCVBatch-sigma1-170

Read more

Linear Sequence of Colour Fields

The following images show a sequence where the neighbourhood size increases linearly from 1 to 33 (skipping even neighbourhood sizes). In networks with neighbourhoods larger than 33, the results look about the same structurally and occasionally show large changes in subsequent neighbourhood sizes. The larger the neighbourhood the more instability there seems to be.

SOMWeights-0000001

Read more

Sequence of Colour Fields (Second Try)

I realized that part of why the SOMs in the previous sequences are inconsistent over time is because a time seeded random number is used to rearrange of order of the segments (inputs) for each SOM, which adds significant random variation. I first tried to use serial training in ANNetGPGPU, but found that it is significantly slower than random training (serial training time: 745.579s; random training time: 12.1s). I also rewrote the code so that the next network actually uses the previous network as a starting point, rather than starting with the original training data for each neighbourhood size. The results, a selection of which follows, certainly have more cohesion, but the use of the previous network reduces some of the colour variation.

SOMWeights-B-0000066

Read more

Sequence of Colour Fields

I managed to batch generate 170 different SOMs with different neighbourhood sizes (default/170 to default). Unfortunately, they are not stable over time; even though they have the same initial conditions, each result has a different structure. I’m not sure if this is due to the change of neighbourhood size, or some indeterminism in the way the algorithm proceeds on the GPU. Following is a selection of the sequence with increasing neighbourhood sizes. These SOMs are rendered using the new code that renders with Gaussianoids rather than rectangles.

SOMWeights-0000066

Read more