On site installation!

Posted: October 13, 2016 at 12:06 pm

installation-a installation-b

Video Version

Posted: September 1, 2016 at 5:22 pm

As previously discussed, I found there was too much variation in the process that is not manifest in the final still image. I’ve thus made a video loop that shows the gradual deconstruction of the photographic image.

Title and Project Description

Posted: August 15, 2016 at 9:17 am

As our gaze peers off into the distance, imagination takes over reality

The relation between the world as we see it and the world as we understand it to be is all wrapped up in abstraction, the act of emphasizing some details in order to dispense with others. My artistic interest in abstraction and computation is an interest in the world as it is versus the world as we read into and conceptualize it. As our gaze peers off into the distance, imagination takes over reality… is site-specific public artwork that uses a photographic image of the surrounding site as raw material. The bottom of the image shows this imperfect photographic illusion, a proxy of objective reality, that becomes increasingly abstracted towards the top. The abstraction is a proxy for the constructive powers of imagination and is the result of a machine learning process that rearranges colour values according to their similarity. The imaginary structure presented in this image is both an emergent composition and an exposition of the implicit structure of reality. The image is a representation of place (as a physical location at the site of installation) and non-place (as an ambiguous and emerging structure under constant renewal). The viewer’s experience of the tension between ambiguity and reality mirrors the mind’s constant attempt to bridge experience and truth.

Final Design

Posted: July 22, 2016 at 3:26 pm


Above is the final design for the QE Plaza. I’ve removed most of the white padding to preserve the aspect ratio of the image to match that required by the installation location. I’ve included details of the full resolution version below. I’m quite happy with artifacts due to sub-sampling used to progressively smooth the transition between the full resolution pano and the 20px per cell SOM. This leads to a quite nice tilt-shift effect that contributes to the tension between reality and imagination.

good_result_24mm-final-detail-1 (more…)

Going with the 20px cell size.

Posted: July 21, 2016 at 11:02 am

The 10px cell test with a small number of iterations (1,000,000) is shown below. While the spires are reduced they are still present, especially at full resolution. In the scaled image below, the most noticeable one is to the left of the leftmost set of red benches.


I also tired using the 7px cell test (below) with very few iterations (top: 500,000 and bottom: 250,000) and larger max neighbourhood (30²) to reduce spires; unfortunately, the lack of training shows a lack of organization compared to the results documented in this post. So I’m going to stick with the previous 20px 5,000,000 training iteration version and spend some additional time working on the transition.

good_result_24mm-SOMScale7_h3775_n30e_i500000 good_result_24mm-SOMScale7_h3775_n30e_i250000

Spires have returned…

Posted: July 20, 2016 at 2:34 pm


Clearly, the spires stick around despite the large neighbourhood size. I’m now trying a 10px cell size with only 1,000,000 iterations.

Focusing in…

Posted: July 20, 2016 at 9:49 am

After some time and consideration, I’ve decided although the spires fit with the theme of imagination and emergence they are just too visually dominant. I’ve thus been exploring using the 20px cell size (that avoids spires) with a lower horizon:



Exponential Increase of Neighbourhood Size

Posted: July 16, 2016 at 9:55 am

I’m still tinkering with the code and the central issue is that I’d like to get better coverage of the area near the horizon, which seems to mean more than 5,000,000 iterations of training, and thus an increase of cell size. The following images show some of these explorations. I’ve also used an even more pointed Gaussian to soften the boundaries.

good_result_24mm-SOMScale10_h4615_n15e_i5000000 good_result_24mm-SOMScale20_h4615_n15e_i5000000 good_result_24mm-SOMScale10_h4615_n150_i5000000


5 Million Iterations

Posted: July 13, 2016 at 9:03 am

In order to investigate those spires I tried running 5 million iterations of training with a smaller network (10px per cell rather than 7px previously used). The result is quite interesting, but the neighbourhood size is too large, causing the sky to dominate significantly. There is still a tendency for clusters to grow in the upper left direction (causing spires) that I cannot explain. The light-coloured spires in the previous post originate at pixels with high degrees of brightness surrounded by darkness, but in the image below we can see they are also present in lower contrast areas (e.g. the sky).



Posted: July 9, 2016 at 10:25 am

The difference between the two following images is a small change in the neighbourhood function (larger on top). I can’t explain the emergence of these beige spires; they emanate from very small areas of the same colour in the original image, but I’m not sure what drives their expansion into the sky. They seem to expand increasingly according to the number of training iterations. I’m doing another run with a larger neighbourhood and see what happens.



Home Stretch!

Posted: July 8, 2016 at 6:50 pm


I found a bug in ANNetGPGPU that resulted in the neighbourhood function having a hard fall-off rather than a gradual Gaussian decay. The result really effected the final aesthetic even after millions of iterations. I also changed from HSV to RGB colour model, which was causing some artifacts. The image above shows a short training session to make sure things are going in the right direction. Extra thanks to Daniel Frenzel for the very fast turnaround fixing the bug!

Almost there…

Posted: June 28, 2016 at 5:25 pm

The following images show the latest iteration of the work. In this version the SOM is blended with the original pano, and I’ve moved the ‘horizon’ higher up. The transition between the pano and the lower resolution SOM (7px per cell) is quite nice and leads to a bit of a tilt-shift effect at the ‘horizon’. See the detail image below. I’m not entirely keen on the degree to which the blue sky over-powers the buildings, so I’m now rendering a version with a smaller max neighbourhood and we’ll see what that looks like tomorrow.

Bogart-Platforms-Coastal-City-In-Progress Bogart-Platforms-Coastal-City-In-Progress-Detail

First results using new panorama!

Posted: June 17, 2016 at 3:20 pm

The following images show initial results using the new panorama. The SOM scales are 10 and 20 (meaning 10 and 20px per SOM cell). These differing scales mean the maximum neighbourhood size (degree of abstraction at top) is double in the top image.

good_result_24mm_SOMScale20_h4263 good_result_24mm_SOMScale10_h4263

“Final” Panorama!

Posted: June 16, 2016 at 9:07 am

I’ve been very busy with other projects, including a big exhibition of Watching and Dreaming (2001: A Space Odyssey) at the Digital Art Biennial in Montreal this month. I had already shot two attempts at a “final” panorama, but have been having significant issues stitching the images together. I finally have a suitable result, but it uses the wider images, which limits the resolution to ~60,000 pixels, which is short of the 80,000px target:



Results without Segmentation

Posted: May 2, 2016 at 6:27 pm

The following images show the results of training a SOM using the colour values of the original pano pixels and bypassing the segmentation processes. The result is that performance is improved (a 5px cell is possible), and the images are quite interesting, although the white sky dominates the image, where it was previously ignored as it was seen as a single giant segment and ignored.

SOMResults_noSegmentation_SOMScale5_h1352_ns400_i1000000-scaler8SOMResults_noSegmentation_SOMScale5_h1352_ns200_i1000000-scaler8 (more…)

Initial Designs

Posted: April 30, 2016 at 4:53 pm

Based on the explorations up to this point, I wanted to post some initial designs using the Gaussian renderer and blending with the original panorama at a more carefully specified horizon. The following images show the results with varying multipliers applied to the Gaussian functions used to draw SOM cells. If you look really closely, you may be able to see that one of the red benches dissolves into a plume of red. A person sitting on one of the red benches on the left turns into thick black smoke.

SOMResults-5_500-tweaked-SOMScale-10_1000000-sigma400-AVG-h1352-scaler0.25 SOMResults-5_500-tweaked-SOMScale-10_1000000-sigma400-AVG-h1352SOMResults-5_500-tweaked-SOMScale-10_1000000-sigma400-AVG-h1352-scaler2 (more…)

Exponential vs Linear

Posted: April 29, 2016 at 3:13 pm

The following images show exponential and linear (respectively) increases of neighbourhood size where the maximum neighbourhood size at the top is the same (1600). They don’t seem all that aesthetically different, nor does there appear to be any smoother a transition at the horizon. The linear version is slightly more interesting (and cosmological).

SOMResults-5_500-tweaked-SOMScale-20_1000000-expSigma40-AVG SOMResults-5_500-tweaked-SOMScale-20_1000000-sigma1600-AVG (more…)

Max Neighbourhood Size and Averaged Top Row

Posted: April 29, 2016 at 10:07 am

After making the decision to seed the top row with averaged values I executed a few runs using a larger network (20px for each SOM cell) and a range of maximum neighbourhood sizes (10, 50, 100, 500, and 1000). The last is so large that it nearly covers the entire width of the map (~1400 cells). Following are a few rectangle renderings of the result. Note, I used too large a scaler, so each cell is drawn quite large. I think this explains the large rectangles at the bottom of the image, but that needs more investigation.

SOMResults-5_500-tweaked-SOMScale-20_1000000-sigma10-AVG SOMResults-5_500-tweaked-SOMScale-20_1000000-sigma50-AVG (more…)

Random vs Averaged Seed Structure on Top Row

Posted: April 29, 2016 at 8:50 am

Seed data on top row is random:

Seed data on top row is calculated from the average of the segments in each column:

I wanted to follow up on some of the conceptual aspects I touched on in my previous post. When I started my Masters degree I was really interested in how I could create a computational process that was not random nor a perfect reflection of my intention nor the outside world. In general, I try to use randomness as sparingly as possible. In my Masters project I, as best as I can recall, did not use any randomness at all, making the system deterministic. Since it used images from its environment (that was constantly changing) it was unpredictable, and yet deterministic. (more…)

Multiple Neighbourhood Sizes in a Single Map

Posted: April 28, 2016 at 4:13 pm

Thanks to Daniel Frenzel, ANNetGPGPU now supports setting different neighbourhood sizes in a single network. This means I will no longer have to generate a different source and data file for each neighbourhood size. Following is an image visualizing the weights (using the old rectangle renderer for performance reasons).

genSOMVizCVTEST (more…)

Mock-up with Offset

Posted: April 25, 2016 at 5:33 pm

In this version, there is an offset applied such that the neighbourhood size only starts increasing at the horizon, rather than the bottom of the image. A mock-up blended with the original image is also provided below.



Neighbourhood Sequence Ranging from 1-30

Posted: April 25, 2016 at 11:46 am

In the following images (raw on top, original overlap on bottom) I used a smaller range of neighbhourhood sizes compared to the previous post (1-30). I was hoping the buildings would be less obliterated, and I actually prefer some of the horizontality of the previous 1-40 version. The reason why I’m unhappy with these results is that the visually interesting structure only begins on the horizon, and thus the abstraction should only start at that point.

genSOMVizCVBatch_sigma1-30 genSOMVizCVBatch_sigma1-30-mockup

Mock-up Using New Colour Field Sequence

Posted: April 24, 2016 at 11:05 am

Following is a mock-up of my current intention for the final image using the most recently posted collage of the new colour field sequence blended with the original pano.


Collage and Video from New Colour Field Sequence

Posted: April 24, 2016 at 11:03 am

From the previous posts, I’ve restricted the range of neighborhood sizes from 1-170 to 1-40 in the following image and video. The more gradual increase of neighborhood size increases the cohesion of the sequence and I’m quite content with these results, although I may want it even more gradual.


Video of Linear Colour Field Sequence

Posted: April 23, 2016 at 11:34 am

The following is a 4k video of a portion of the colour field sequence previously posted.

Collage of multiple colour fields.

Posted: April 22, 2016 at 5:05 pm

The following image is constructed by taking a single row from each of the trained networks such that the lowest (bottom) row has the smallest neighbourhood size (1) and the highest (top) row has the largest (170). Note that very little of the image is readable because the image is abstracted very quickly as the neighbourhood size increases in steps of 1. Note some stability in structure (position of colours) near the bottom of the image. Near the top the increasing horizontality indicates large differences in structure in subsequent neighbourhood sizes.


Linear Sequence of Colour Fields

Posted: April 22, 2016 at 4:45 pm

The following images show a sequence where the neighbourhood size increases linearly from 1 to 33 (skipping even neighbourhood sizes). In networks with neighbourhoods larger than 33, the results look about the same structurally and occasionally show large changes in subsequent neighbourhood sizes. The larger the neighbourhood the more instability there seems to be.

SOMWeights-0000001 (more…)

Sequence of Colour Fields (Second Try)

Posted: April 21, 2016 at 9:38 am

I realized that part of why the SOMs in the previous sequences are inconsistent over time is because a time seeded random number is used to rearrange of order of the segments (inputs) for each SOM, which adds significant random variation. I first tried to use serial training in ANNetGPGPU, but found that it is significantly slower than random training (serial training time: 745.579s; random training time: 12.1s). I also rewrote the code so that the next network actually uses the previous network as a starting point, rather than starting with the original training data for each neighbourhood size. The results, a selection of which follows, certainly have more cohesion, but the use of the previous network reduces some of the colour variation.

SOMWeights-B-0000066 (more…)

Sequence of Colour Fields

Posted: April 20, 2016 at 8:47 am

I managed to batch generate 170 different SOMs with different neighbourhood sizes (default/170 to default). Unfortunately, they are not stable over time; even though they have the same initial conditions, each result has a different structure. I’m not sure if this is due to the change of neighbourhood size, or some indeterminism in the way the algorithm proceeds on the GPU. Following is a selection of the sequence with increasing neighbourhood sizes. These SOMs are rendered using the new code that renders with Gaussianoids rather than rectangles.

SOMWeights-0000066 (more…)

More Colour Fields resulting from Different Neighbourhood Sizes

Posted: April 18, 2016 at 4:29 pm

The following images are renderings of the SOM structure trained and visualized using the same methods as previously posted. The only difference here is that much smaller neighbourhood sizes are used (top: default/150 and bottom: default/50)

still-proxy-pano-edit-genSOMVizCV-5_500-SOM-10000-SOMScale-36-sigma150 still-proxy-pano-edit-genSOMVizCV-5_500-SOM-10000-SOMScale-36-sigma50 (more…)