I realized after looking back at this post, that I did not actually post many of my previous shuffled results and I could not remember them. I re-rendered a few so I could reconsider them compared to the unsorted version; they are posted below. The result I thought was most successful is at the bottom. It still does not have the same balance of flow and photographic readability of the unsorted version.
The following images are a few variations on shifting a smaller subset of large percepts that are shifted to the upper 3/4 (top) and middle (bottom) of the stack. Of these partial stacked explorations, I think the top image is the strongest yet, but it still seems too uniform compared to the unsorted version. I have a couple more ideas for variations, but so far nothing is an improvement on the unsorted version.
Looking back at the sorted and unsorted versions I realized I like aspects of both of them. The sorted version certainly has more flow, but the smaller segments all on top obliterate any photographic reading. The unsorted version is strong because the larger photographic segments are readable, but they also interrupt the flow. The images below show intermediary versions where the segments are sorted, but then a subset of the largest segments are shifted up in the stack (rendering order) to increase the photographic readability. I still think the unsorted version is strongest, probably due to the variation of texture over the image. In these explorations below, the sorting by area means the texture is quite inform. I may try a few more variations where a much smaller section of large percepts moved up higher; in the images below 25% and 15% of the largest percepts are inserted into the middle, top quarter and top 15% of the stack, respectively. I think these partial sorted variations are more intentional than the randomly shuffled explorations; I want larger segments to be near the top, but perhaps not at the top.
I finally got to reading Karen Barad’s book (titled above) and thought I would post my notes here while I reflect on them. After reading I also realized that I had gotten Bohm and Bohr confused in my notes from the Karen Barad Seminar; this has now been corrected. In parallel with the collage production one idea is to reconsider my current Artist Statement and rewrite it to be consistent with Agential Realism. Next, I think I’m going to read Chapter 7 to focus on what is meant by “entanglements”. My notes on chapter 4 are as follows:
Following this I generated a sorted version of this composition where larger segments are behind the smaller segments; this emphasizes greater flow, but at the expense of photo-readable segments being visible. I’ve included the sorted version and a few details below. I was just thinking that perhaps I could include a small subset of the large (or medium) segments in the front of the small ones by manipulating of their order in a more complex way; for example, randomly select a few segments from the large end and insert them on the small end?
After the previous explorations I thought I would focus on the 3,000,000 iteration collages and generated two more options. I still think the previous work is the strongest. I’m going to now generate unsorted, sorted and shuffled versions of that previous composition and decide which is most successful.
Following from previous collages I thought I would try fewer iterations (100,000) and a randomly shuffling the stacking order of percepts. I can’t say I’m happy with these results; the most recent iteration is still the strongest. I’ve included below a few of these explorations. I’m now calculating a couple variations with 3,000,000 training iterations. I’m also going to focus on Barad and (re)framing my thinking about objects in relation to how I’ve been thinking about Machine Subjectivity. This will manifest rewriting my artist statement, and I’ve also been playing with the idea of the artist statement as indeterminate where the specific language is manifested as multiple permutations.
Before starting the recent collage explorations, I had been doing more reading and thinking about entropy, see notes following. I also got a chance to watch a lecture that Sarah Dunsiger sent on entropy and emergence, I’ve included my notes on that below as well.
Entropy
Natural log of the number of states in a system multiplied by a constant
The log reduces very large numbers and does little for small numbers. (e.g. ln(10e06) = ~16, ln(10e02) = ~7
The constant (Boltzmann) is a very small number (~1e-23)
So entropy is a small representation of really large numbers of possible states.
All possible 640×480 images in 8bit have 5e12 possible states and an ‘entropy’ of 4.04e-22. (Does it make any sense to think of entropy of an image?? An image is not dynamical, entropy is about dynamics, not structure.)
Second Law of Thermodynamics:
Entropy of closed systems never decreases (the number of possible states only increases until equilibrium, maximum entropy)
Entropy in open systems may decrease if the environment entropy increases (the number of possible states may decrease if the number of states in the environment increases)
is entropy about the propagation of energy? Does a system with more energy have more states? If it has more states, it looses that energy to the environment (increasing the number of its states in the environment).
is there some analogy in ML? Could the energy be the state of excitement of the initial conditions? The rate of learning?
More entropy means more complexity because more information is needed to represent the potential states of a system.
This seems more about the constraints of the system than the specific energy states.
Sarah Papers
order can be introduced from entropy alone
order from disorder?
the whole often resembles the part (chiral particles make chiral structure)
Entropy and Emergence (Video Lecture)
entropy as a measure of what you don’t know about the state of a system
fewer states means more certainty due to less possibilities.
a high-entropy system is random / has many states and no constraint.
entropy as the minimum number of binary questions one must ask to fully determine the system.
random needs every question whereas a pattern can be compressed
Key take-away: entropy does not indicate disorder because a system may have more ordered states than disordered states.
The following images were generated by combining the segments from both collections of photographs (wide and close). There are a total of 135,226 segments inclusive of both collections. The top image is under-trained over only 50,000 iterations (meaning that ~2/3s of the segments were not presented to the network). The bottom image was trained over 150,000 iterations.
Up to this point I’ve been working with half the photos I shot at TRIUMF, the close-up ones. Today I started working with the medium and wide shots that show larger scale structures, architecture, etc. Rather than ~57,000 segments, the density of the wider images resulted in ~77,000 segments. I think these images are the most successful yet, balancing abstraction and photo-realism as well as order and complexity. The composition ends up with larger areas of colour due to the larger areas of colour at the architectural scale. This was generated with 50,000 iterations and I’m now training a 2,000,000 iteration version.
The following image and details shows the result of a smaller neighbourhood function (1/10 of SOM width) after 2,000,000 training iterations. I’ve also rendered the collage in the descending order by area such that the largest segments are rendered behind the smaller segments. This increases the sense of flow, but I don’t think the very small neighbourhood improves things. I still think the images are more successful when they are more chaotic and I’m training a network on fewer iterations to see what the results look like. With the larger area images in the background, the tension between abstraction and photo-realism is lost. The resulting density of textures are very interesting though.
Using a fitted ellipse for each segment, I’ve now included orientation features. This results in images such as the following that feel like they are really going in the right direction. The top one in particular cues magnetic fields, which is very apt. The bottom image uses a larger neighbourhood function, which leads to a smoother more organized macro-structure; I prefer the top image with more turbulence. I’m now training a version of the top with more iterations to see where that goes.
After the early success using the hue histogram features on the TRIUMF collection, I thought I’d go back to the Robin collection. The results are certainly better than the initial BGR collage, but the muted natural tones and the organic quality of the segments leads to a composition that does not seem to balance order and disorder the way I would like; it’s a little too messy. I’ve included the full frame version with a few full resolution details. I’ve also posted a version at half resolution where the same-sized segments appear twice as large relative to frame.
I’ve made some progress on using the new TRIUMF photographs as material for new collages using the same set of segments. The image on the top is using simple BGR features, and the image on the bottom (and corresponding details) is using a 64 bin histogram of each segment’s hue channel as features. The BGR feature image was trained over 2,000,000 iterations while the hue histogram image was trained over 50,000 iterations; both images use a max neighbourhood size of 0.2. I’m going to also try exploring some orientation features. I’m now training a 500,000 iteration version.
I started working through some ideas for a new collage following from my previous works using cinematic material. Robin Gleason donated some photos of her material collections to start with. I think the main issues are that
The diversity of tones in a photograph means there is much more detail than appears and when one resorts components by colour, we end up with something that often resembles a gradient.
The quality of the edges from this organic source material means there is little meta-structure to appreciate and the size of segments means their content becomes merely texture and looses all photographic realism.
It will be interesting to see whether the hard-edge apparatus photographs will allow the preservation those hard edges. Also I’ll be going from 22 photographs to over a 100, so the size of segments can be increased (in theory). The following images shows a full-resolution collage and a few details; the ~50,000 segments were organized by mean colour similarity using a under-trained Self-Organized Map (SOM). I also included a few other visualizations of some SOM (not painted using the segments) results that show the lack of interesting structure. I also plan to explore using features other than mean colour, which should allow for more complexity.
On Wednesday I had the opportunity to spend a couple hours amongst the TRIUMF beam lines to take photographs for the project. I’m just posting a few photos here of the scrap area behind the shop, where Sarah Dunsiger, Robin Gleason and Karen Kazmer were doing a material exploration of the scrap materials.
After months I’ve finally finished reading the Karen Barad papers that where provided as part of their symposium at UBC. The following is my notes from the symposium, as well as my responses to the readings. These are lightly edited and clarified, and if I’m inspired to respond to the notes, I’ll include that in square brackets.
Troubling Time/s and Ecologies of Nothingness, Re-turning, Re-remembering, and Facing the Incalculable.
The following is direct copy-paste from my notes. I have had some time to reflect on this, but I’m strongly leaning towards Karen Barad’s Agential Realist Interpretation, which I will post about after doing some more reading.
Bell’s Theorem and the EPR Paradox
Quantum entanglement means that one of these things is very different than we tend to accept:
Locality: distance has no meaning in some cases
Realism: reality does not exist without observation “counterfactual definiteness” coexistence of everything possible.
Freedom of choice: the universe and all of our actions are deterministic.
“namely (i) reality (that microscopic objects have real properties determining the outcomes of quantum mechanical measurements), and (ii) locality (that reality in one location is not influenced by measurements performed simultaneously at a distant location). ” (wikipedia) (more…)
In one of my earlier posts I mentioned Bell’s Theorem and I’ve been spending some time reading thinking about this in relation to the EPR Paradox. That splintered off into many interesting and different directions for further reading and consideration, listed below. My interest in this area connects with my epistemological inquiry, and the idea of objects and subjects as being mutually constructive.
EPR Paradox
Counterfactual Definiteness
Observation
Quantum Discoherence
Free Will
The Copenhagen Interpretation
De Broglie-Bohm Theory
The Transactional Interpretation
Quantum Bayesianism
Update: I wrote this months ago and have not had a chance to return to it until now. After a week of seminars with Karen Barad, I’m inspired to return to this tough work but it’ll take a few posts to get from where I was at the time or writing the above to where I am now.
I keep hoping to keep up with my reading and reflections for this project and I am not succeeding. I’m spending most of my time just catching up, rather than reading what I’ve already flagged as interesting and considering what has already been discussed. Unfortunately this post will not be any different; it’s a quick description of the last meeting and some quick notes of things to go back to.
The third meeting was a focus on artistic practise, so Karen, Robin and I talked a bit about our practises and I was struck by some very interesting intersections between Robin and I that seemed to resonate significantly with Sarah and Karen. Following are the notes taken by Robin during the meeting; I’ll provide some keywords below. (more…)
I did not get a chance to write a post about our second meeting, so I’m doing so now before our third meeting! During the second meeting we were introduced to team member Dr. Seven J. Barnes (who also served on my PhD committee). I thought I would look through my notes and list interesting terms, phrases and concepts that came up in the second meeting:
Primordial Soup
Consciousness as emergent property
Magnetic Frustration: Inability to find the most favourable alignment of spins
Does this mean the macro structure is and difficult to predict from the micro structure? Equal probability of being in different states?
What does frustration mean in terms of dynamics over time?
Spin Glass used to model dynamics in biological neural networks.
Spin Glass seems very interesting in itself, and I’m still left a little lost in the dynamics. Is the state of spin static in spin glass as long as temperature is steady? What happens in the phase transitions of spin glass? (Is a spin glass a transitional state of a ferromagnet?)
Perturbation
Phase-transition
Nucleation
What is the grand theory in Condensed Matter Physics?
Skyrmion
Metastability
Metaplasticity in learning systems?
(spooky?) Action at a distance.
I also wrote down a few ideas for possible “subtopics” in emergence:
Last night a subset of the LOoW folks got together at Emily Carr for a presentation on emergence from an artistic perspective and have some informal discussions at a pub.
I just wanted to reflect a little here and focus on things that really stood out from the discussion. The physicists in the room brought up the idea of emergence (as the emergence of surprising behaviour due to the interactions of numerous believed-to-be understood components) in condensed matter physics. In the discussion some tensions between particle physics and condensed matter physics became visible. I’ll need to return to condensed matter physics later on (as I had not heard that term), and a quick look at the wikipedia page seems to indicate we’re talking about modelling complexity at the transition between states (solid to liquid, etc.). Through the discussion I kept thinking about disciplines and these various senses of “emergence” (e.g. coming from the dark into the light), and I realized that my initial and implicit sense of emergence comes from the influence of fractals, ALife, autopoiesis, self-organization through my artistic career. I realized that my sense of emergence is really about complexity science as a meta-discipline. (more…)
On September 28th I was present for the first meeting of the third phase of the Leaning Out of Windows project. I’m really excited about this project as I had previously been inspired by physics in my artistic work (“Engineered“) and have been looking to get back into that body of knowledge. I applied for a few COLLIDE residency awards at CERN over the last couple of years and made video submissions available here and here.
As part of this meeting we did a tour of TRIUMF and I’ll include additional photos from the tour below. The meeting (and tour) were overwhelming to say the least and I still need some time to integrate and reflect. This blog post is devoted my initial musings and some very preliminary project ideas. (more…)
