Abstraction and Meaning
(Human vs Machine Creativity)

I have said that my interest in art, and my approach to art making, is not in terms of the traditional role of artistic “expression”, but rather art is used as a methodology to explore “expression”, or more specifically examine the notion of meaning itself. I have talked about being more interested in “doing” than “representing” and in “exploring” over “expressing”. This results from an dissatisfaction early my my B.F.A. with contemporary art where I could not glean the relation between the title and text accompanying a work, and the form of the work itself. I found this very frustrating and often considered it a failure of the work. Part of my interest in using computational mechanisms, and scientific knowledge, to build artworks is to formalize and make rigorous the relation between the concept (what the work is supposed to be) and the form (what the work is).

This interest in the relation between concept and form (meaning and message) has stuck with me, and yet I don’t reflect on it often. I’ve realized that the series of “Context Machines” is composed of works that explore the relation between the world “out there” that cameras can “see” and the world of representations that agent’s can interpret. In essence, the dreaming machine series is about how the relation between signifier and signified is formed. What makes a word or symbol different than anything else? There is nothing in the sound of a word that different from a sound of gibberish. Meaning is in our culture, and our shared experience of the world, and allows us to attribute shared meanings to physical forms. Some would argue that meaning is causal, that effects out in the world simply initiate processes that lead to representations.

Lets take two assumptions at this point: 1. We assume that there is a real physical world that is our there that is not a function of our own perception. 2. We assume that the world is infinitely dense, that its filled with continuous organic processes and properties that are not discrete. When we are children we learn to associate certain sensor impressions of the world—say a fluffy animal with four legs, with a second concurrent sensory impression—the sounds that correspond to the word “dog” coming from our care-giver. With enough presentations and repetitions we learn to associate our sensor impressions of this fuzzy creature and a set of sounds that we can eventually both hear and speak. At that point we think that all fluffy animals with four legs are dogs, including cats. Our care-givers correct us through repetition, and we eventually differentiate between dogs and cats, despite their degree of overlap of features, animate, two eyes, cold nose, four paws, claws, soft fur, etc. If we assemble a list of all of the features shared by dogs and cats and measured their degree for dogs and cats we would find a very high degree of overlap. There is no single line that entirely separates dogs and cats, all lines would put some dogs on the cat side and some cats on the dog side. (I would argue you could do exactly the same thing with all properties of males and females.) We could separate the two groups using something other than a line, a curve the snakes back and forth between dog and cat, but that simply makes the point that dogs and cats cannot be differentiated on the basis of visual features alone. A supervisor (care-giver) is required to partition the continuous space into concepts that represent, at best, clusters of features, or perhaps only centres of clusters.

This learning of concepts from continuous reality is a process of abstraction. The continuity of features is replaced with a Boolean relation: Animal A is a dog or its a cat, there is no in-between. These concepts are highly useful in that they allow us to think about dogs without being concerned with all the features of dogs. Without this ability we would be easily overwhelmed by the continuous variation out in the world. It would be difficult to know what to pay attention to, what sensory values in the world are important. Concepts break the world into pieces in order to make sense of it, to make sharable meaning.

We can infer, and prove statements of relation between terms without being concerned with the definition of the terms themselves. Our concepts become highly complex and  nested, and can abstract so far from the world as encapsulate ideas such “justice” or “exponential”. The process of generating concepts is highly complex and still very much up for debate. For the purposes of this text, we need only be concerned with the idea that we can use concepts in place of the real thing, and that concepts are somewhat arbitrary partitions of a continuous space. When we are being creative, we are often thinking through our systems of concepts, using them as atoms in the constructions of ideas and artifacts.

There are also computational systems that are meant to be “creative” some of these model human creativity, and are therefore concerned with concepts and analogies, and metaphor. There are other systems that are inspired by biology that look at creativity as a search in a space. If we do not know what we want, but we know what its properties are, we can program a machine to find something that fits those required properties. Such systems have been used to design electrical circuits and antenna through the artificial evolution and simulation of many arrangements of components. In short, these evolutionary methods start with many thousands of random locations in a search space (each point is a particular arrangement of components), and refine and move around those points (through mutation and breeding) as driven by the fitness measured by how well the arrangement performs in simulation. Such a system has little notion of the concept of what a resistor or diode does. It randomly combines components until they work. Some of these designs are not even understood by human engineers, because they use components in such a way as to exploit their features directly. They do not depend on a cultural knowledge of components and what they do in various circumstances the way that human engineers do. These systems can find things that a human would never think of, because they don’t fit into the conceptual systems with which we make sense of the world.

It could be argued that true creativity transcends these conceptual categories by working at a lower level, by being concerned with continuous changes in form and movement that may not be conceptual at all, but shift our cognition to a deeper plane dependent on the interactions of so many low level processes, that we cannot be conscious of all of them. Choices unfold organically and tacitly, some without us even realizing it.

The kind of abstraction used in our conceptual system reaches all the way down into our perceptions. We know that our perceptual systems are constructive and highly influenced by task demands, and by our expectations and conceptual systems. We see not what is actually in the world, those real continuous values, we see concepts: dogs, chairs, the sky and trees. We know the resolution we perceive does not match the resolution of our eyes—what we perceive is constructed from what we have seen in the past what our concepts lead us to predict. Visual perception as been described as “inherently creative” such that the wonders of what we perceive is constructed from our internal processes as much as from external stimulus. It is hard to say how much is expectation and how much is reality, as we cannot access those continuous parameters directly, we are always biased by our concepts and expectations all the way down to our very perception.

We are constantly playing a game of equilibrium with the world, where it impacts our sensory organs with an infinity of variation while our brains and minds are constantly trying to keep that chaos at bay, breaking the world into pieces that fit into the ever growing complexity of our conceptual systems. Perhaps creativity is in the tension between our expectations of the world, and that which the world provides.

My interest in making artwork where there is a rigorous connection between concept and object reflects the way that we make meaning in the world through shared experience and our ability to conceptualize and abstract. Not only is this relation between concept and object inherent in the work on dreaming machines, but it is also the purpose of the machines themselves. These artworks attempt to make sense of their world. They don’t have concepts, but there is so much material in the world that it must be reduced for the system to have any long-term memory of it. No two images are ever identical, and thus the system must determine if X is similar enough to Y that they can be considered the same. These “percepts” are the systems simple abstraction of the world. They lack the depth and complexity of a human child, or even the perceptual abilities of a rodent or insect. The systems’ dreams and perceptions are the reflection of their attempt to make sense of the world. In perception their images are reproductions of sensory images from the camera, constrained by the world out there. While they dream this constraint is lost, the system’s activity is the result of activations between these percepts themselves, that are no longer tied to the world. They flow and fade in and out, the systems imaginations of what the world is running out of control.

Perhaps perception is the root of creativity, a balance between our expectations of the world, and the worlds impact on us. It’s a play between two levels of knowledge, the continuous complex flow of information from the world and our abstract representations of it. Perhaps creativity must move beyond the level of the concept, as it’s the tension of relation between the complex reality and the abstract representation that gives representation meaning and power—the tension between what they are and what they represent. Perceptions are waking dreams, while dreams are our expectations and cultural knowledge manifesting their own autonomy. The perception will never perfectly match the reality as concepts are always limited in complexity. Perhaps it’s the failures in the systems perception (and perhaps our own) that reflect most what we want the world to be.

While we are constantly attempting to see different things in the world as the same, perhaps art is about showing that what appears to be the same thing, is in fact an infinity of possibility.