Reconstructions from Soft Percepts

Posted: January 23, 2016 at 9:38 am

In an effort to decrease processing time I highly simplified the code that generates percepts and the lookup table used to render reconstructions of the original frames. This which has lead to significant differences in the aesthetic. The images below show the same frames as the last post of reconstructions and how they now appear.

store-0001043 store-0001043

In the previous version of the code I needlessly recomputed a lot of information that was used in both the generation of percepts (groups of multiple image segments) and in the reconstruction of the original frames. One of the issues with that code is that the appearance of the percepts was significantly different than how the system actually saw them. Some pixel regions where represented with much more visual emphasis than others, making percepts appear much more concrete and specific than they actually are (as far as the system is concerned). I removed all the code that recomputed the relations between the constituent segments in a percept. Now each segment is weighted equally, and the resulting percept image is the mean of the constituent parts.

While this is much more consistent with the way the system ‘sees’ the original images, the averaging process washes out a lot of the colour information and diversity of edges. Percepts are now blobs of information that tend toward high density in the middle and low density at the edges, leading to a tendency to a pointillist aesthetic. I’m not sure what the best solution would be to both maintain the consistency of the representation of the percepts and their internal structure. I could always add more percepts, but that could decrease the feasibility of prediction for dream generation. There are currently 50,000 percepts, which is consistent with the 600,000 previously used for the entire film. For prediction, the idea was to learn both the occurrence and position of percepts, which means 100,000 variables, which could be unworkable; corresponding to 1.2 million for the whole film!

These new images are quite visually interesting though, especially when seen as stills at full resolution where the differences in density between centre and edges are readable. Unfortunately this subtlety is not readable in the video version, as the frame changes prevent the attention to such small details. This does open the door to print versions where these details would be emphasized. These more stable percepts make reconstructions that appear much closer to perception, albeit a visually impaired perception. I’ll post video soon.

store-0001848 store-0001848

store-0003902 store-0003902

store-0000351 store-0000351

store-0009904 store-0009904