As filtering by area lead to such interesting results, I went ahead and split up the percepts into three groups according to percept areas. The triptych below shows all 200,000 percepts, but separated into three separately trained and differently sized SOMs. I’ve also included details of the latter two SOMs. I thought this approach would lead to more cohesion within each map, but the redundancy between the second and third images leads me to believe that 200,000 is too many clusters. Since I need to reduce the number of clusters for the Dreaming part of Watching and Dreaming, I’ll put the collage project aside until I’ve determined a reasonable max number of clusters for LSTM prediction and then come back to it.
After looking at the previous results I think the issue is that there is simply too much diversity in all 200,000 components to make an image with any degree of consistency. I’ve managed to implement code to filter image components based on pixel area. The following images and details are composed of the top 5,000 and 10,000 largest components. Due to the large size of these components, these are full size (no scaling) and suitable to large scale printing. I think the first image with 5,000 components is the most compelling. I will now look at making collages from the remaining smaller components, or a subset thereof.
Following shows the results of training over the weekend. It seems with this many inputs (200,000) and the requirement for over-fitting (the number of neurons ~= the number of inputs) we need a lot of iterations. I think this is the most interesting so far, but I also had the idea to break the percepts into sets and make a different SOM for each set. This would make each one more unified (in terms of scale) and give them very different character.
The following image is the result of a 5,000,000 iteration training run. Note the comparative lack of holes where no percepts are present. The more I look at these images the more I think they would need to be shown not as a print, but as a light-box. I wonder what the maximum contrast of a light-box would be… On the plus side, the collages seem to work best at a lower resolution (4096px square below) due to the small size of the percepts (extracted from a 1080p HD source); this would mean much smaller (27″ @ 150ppi, 14″ @ 300ppi) and affordable light-boxes. I wonder how the collages using the 30,000,000 segments will compare since they will not have soft edges and higher brightness and saturation. It will be a while before I get to those since the code I’m using is quite slow to return segment positions (17hours for 200,000 percepts) and is not currently scalable to the 30,000,000 segments.
I have been working on getting large percepts to stick in the middle so they don’t push the outer edges too much. I attempted this by explicitly setting particular neurons in the middle of the SOM with features corresponding to the largest percepts. While this worked for a smaller number of training iterations (1000) it did not seem to make any difference over a large number of training iterations. The following images show the results where large percepts are scaled down to reduce the size variance. The lack of training leads to quite a few dead spots where no percepts are located. While quite dark, the black background works better for this content. I’ve included a visualization of the raw weights and a few details.