Musicians create music and visual artists create images. I am becoming increasingly aware of the commonalities I share with musicians rather than other visual artists. The creative process of a sound artist or an electronic musician is very similar to my own. The fundamental basis of both what I do and what a musician does is the creation of structure, or perhaps more specifically the making of a score or program that nurtures the creation of structure. My own practice emphasizes structure (in the form of language, image, sound or even a set of related ideas) regardless of my chosen medium.
[Published in Vagueterrain "Generative Art", 2006]
My primary creative tool is a computer, which, by definition is a machine that manipulates only structure since it has no understanding of semantic content. Why is structure so important to my work and why does a project need a certain amount of structural complexity to keep me interested? Does structure have intrinsic meaning?
the nature of consciousness
Consciousness is the process of turning noise into pattern. When we look around the world and listen we do not see or hear the cascade of unimaginable amounts of interacting particles, from waves of light scattering off objects to the molecules that carry the sound we hear; rather, we see a defined, understandable structure of our world. We do not see the world as science defines it. We instead collectively participate in the process of creating the structure that comprises our world. This process happens in our minds, not out in the world and raises the question of interpretation. In a broad sense, my exploration of structure in an artistic context is as much about an exploration of meaning and interpretation as it is of structure. The purpose of my artwork is the thought, inspiration and dialog that surround the structure and interpretation of the process the art system carries out.
Noise and pattern are not two concepts that are mutually exclusive. In fact, noise and pattern are markers at two ends of a single continuum that classify all types of structure. We consider noise as the uniformly distributed froth we see when the television is not tuned, the space we hear between radio frequencies and the data we get beamed into our computers running seti@home. Pattern is the recognition of the repetition of an event. We see or hear something that reminds us of a sight or a sound we saw or heard not long before. Rhythm is found as our perception continues to bind these repeating events into groups. Noise itself is unpredictable because we never know what will happen next; however, pattern is predictable since it refers to itself over time.
I consider the vast area between noise and pattern as chaos. Chaos is not concerned with randomness nor pattern as distinct entities, but rather the relationship of the two. A structure near, but not at, the pattern marker is a predictable rhythm. The repetition of the pattern does not continue forever. Now and then the pattern breaks and we perceive what we did not expect. Structure near the end of noise is much the same; it manifests itself as an ongoing meaningless jumble where once in a while a moment of clarity and repetition forms. For absolute pattern to exist the repetition of events would have to occur infinitely. If a break formed in the pattern it would no longer be absolute and would shift slightly into the chaotic realm. The same is true for absolute noise; it would have to be unpredictable forever and at no moment could one event resemble a previous event. Both pure noise and pure pattern are therefore limitless. This continuum of structure measures the extent to which consciousness has made that structure meaningful; thus, from noise we get pattern.
laws, transformation, iteration.
A generative process is made possible by three steps: laws (or limits), transformation and iteration. Chaotic equations, fractals, noise generators, weather and perhaps every structure is made possible by these three tenants.
Laws are the limitations that dictate what is possible in a particular structure. Without defining limits, we cannot know if something is noise, chaos or pattern. We also cannot know if a structure is uniformly distributed without limiting the scope of analysis. Without laws or limits, structure is pure and therefore impenetrable and impossible to know. The limits of a generative system are either the first to be defined or, in many cases, are innate properties of the system itself.
Transformation is the playground for creativity. It is a choice that steers the result in a certain direction but, due to the emergent and chaotic behaviour of generative systems, the result cannot be known in advance. One makes a choice and views the result. The artist is always one step behind; s/he explores the system without knowing the path taken. The point of interface between the will of the creative mind and the generative process is the transformation. We can define transformation by the change of any variable in the system while it runs, from filtering certain values out to spatial transformation like rotation, shifting or inverting.
Iteration is the core of the generative system. It is a feedback process that nourishes the next step with the results of the previous, thus creating a cycle. Any small change, or transformation, between the input and the feedback creates a very large impact in the overall result. The reasoning for this is because the transformation is multiplied for each iteration of the system. Most “noise” generators in computers are really not sources of noise at all. They are actually equations that generate what appear to be noise through this very same iterative process. From pattern we get noise.
the art of doing
I have always been particularly interested in creating through the process of exploration. In this case, the result is not the purpose but instead the path of creation; what the project does and how it is done is its central purpose. The aim is not to be aesthetically constructed in a predetermined way or express a certain idea but to inspire creation in others. Oftentimes the result of the collaboration between the system and the artist ends up being a surprise. The artist can be as inspired by the result as s/he hopes the viewer could be.
In my arts education I was trained to develop a project initiated by a concept. This conceptual intention would have to be maintained throughout the creative process as it was this concept or intention that was considered the purpose of the work. I initially thought of this concept as an idea that the work was intended to express, as that was largely the type of work I was exposed to. Over time my interpretation of the meaning of “concept” ceased to be an intention and became a starting point in a negotiation between myself and the world around me. The generative system is not limited to the artistic tool but is a process that involves the system, the audience and the world around them.
In 2000, I created a work entitled “Seed”. The project was created at a transitional point in my artistic practice. The concept of “Seed” includes both an element of expression as well as an element of action (what the piece is doing). The work was meant to express my own fear for the future as medicine treats life as a commodity to a greater and greater extent. I was particularly fascinated with the repercussions of reducing life to genetic code due to the success of the human genome project. I imagined a process where a living creature could have its genetic code scanned into a computer system, whereby allowing this virtual copy of that life to be edited and manipulated non-physically. The code would be “improved” upon so that it could be manifested as a new living being completely separate from both the original and the non-physical copy.
What “Seed” does is very simple; it acts out a process similar to the one above using the image of the audience to symbolize their genetic code. The audience is “scanned” by a camera that feeds the computer with data representing the genetics of the audience. An image is created from this data; specifically, an array of codes where each element corresponds to a particular pixel in the image. Each pixel is considered a genetic base-pair. These codes are arranged to create an on-screen image that resembles the image scanned by the camera. The image is projected onto a screen behind the viewer in the direct line of sight of the camera. During the next iteration, the scan of the audience now includes both the audience at that moment combined with the on-screen image created in the last iteration. When there is no viewer the computer can only see the image it generates itself. When a viewer stands in front of the camera the image of the viewer is mixed with the feedback process. This creates a feedback loop where the simple transformation of the image, from pixels to codes, yields a highly complex structure.
The complexity of the structure that results from this process is considered a new organism. The organism is seeded by the audience as they provide the “genetic” code that contributes to the diversity of the organism. The genetics of the organism are represented on a second screen as genetic base-pairs (ATGC). Similar to how the image of the audience symbolizes its genetic essence, the plotting of the new organism’s base-pairs are its symbolic manifestation.
In 2003, I read “Chaos, The Making of a New Science” by James Gleick. This was the book that introduced me to the world of chaos and the complex relationship between noise and order. While reading the book, I started writing a few programs to compute chaotic equations in C which eventually lead to me writing the first version of the “chaos” externals for Pure-Data. In the process of researching and writing, I was particularly inspired by the historical connection between weather modeling and chaos. Two specific equations, the Lorenz attractor and the Rossler attractor, piqued my interest. The Lorenz attractor is the result of a few simple fluid dynamics equations. The Rossler attractor is based on equations to calculate chemical equilibrium. This book inspired my final BFA project, “Oracle”. The purpose of “Oracle” is to give a voice to nature.
“Oracle” uses a live feed of current weather conditions as the variables in a multiplication of Rossler and Lorenz attractors. For each weather report, retrieved each thirty minutes, the variables of the equations are set by the humidity, wind direction, wind speed, temperature, humidity, etc. For each of these thirty-minute blocks, the equations iterate using those same variables, resulting in a set of three numbers. These numbers serve as coordinates in a three-dimensional volume that is populated by a small vocabulary of six hundred words in six different languages. For each iteration, three words are chosen that correspond to each of the three dimensions of the volume. The projection is a mix of video images of earth, air, fire and water, and are combined based on the wind direction. The resultant words of the chaotic equations fade in and out of this ground.
The complex and unpredictable system of weather is used to drive a set of chaotic equations that are themselves unpredictable. As an artist, I chose the vocabulary of the system and transformed the results of the equations to be contained within a closed cube. The choice of what word should come next is an emergent result of the whole system that includes both the limitations imposed by the artist and the limitations of the weather variables.
Like many other art students after finishing their degrees, I went into a bit of a creative slump. It was at this point that I started to approach computer image-making as improvisation. I consider this work the same process as electronic musical improvisation, except rather than creating sound, I created images. These images were made using the GEM (Graphics Environment for Multimedia) library for Pure-Data. A huge creative revolution happened when Erich Berger exposed me to creating feedback loops in GEM. Erich had been using the technique in a number of performances for years and I was very attracted to a totally digital method of visual feedback. The technique differed from video feedback in a few interesting ways:
- Almost no degradation of the image
- The frame rate is not locked to 30fps, thus allowing movement not possible in video
- GEM allows a huge array of digital methods for transforming the image
I started using feedback in my performance “Threads” but soon after went to work creating a Pure-Data patch that focused specifically on the feedback itself. I titled this project “Self-Similar” to draw a connection between the project and fractal systems. “Self-Similar” is what Benoit Mandlebrot calls the attribute of fractals where the part resembles the whole. “Self-Similar”, like all my feedback patches, copies the whole image, transforms it and uses the modified version in the next iteration at sixty iterations per second. The feedback process in “Self-Similar” differs from the feedback process in “Threads” in that, rather than a single feedback process, “Self-Similar” uses three such processes simultaneously. The patch works as follows:
- Place a simple shape in a larger ground
- Copy the whole image
- Transform the copy of the image and place it onto the ground (Feedback A)
- Copy the whole image (which now includes the original and the result of Feedback A)
- Transform the new copy and place it onto the ground (Feedback B)
- Copy the whole image (which now includes the original, the result of Feedback A and the result of Feedback B)
- And so on for Feedback C
This way each of the feedback processes uses a different transformation; so, the transformation in Feedback B effects the next generation of Feedback A. All three feedback processes interact to create very complex structures from the simple transformations.
“Reflex” is my latest improvisational work. It is a creative departure for me since it involves both the creation of sound and image simultaneously. “Reflex” stands out as a generative work since the equation used is not something I wrote nor fully understand. “Reflex” is an exploration of Pure-Data and is only possible in Pure-Data. Pure-Data is composed of two parts: a GUI and a DSP engine. These two parts communicate through a local network socket. What “Reflex” does is hijack the data that is transmitted from the GUI to the DSP and feeds the information back into Pure-Data to be used as raw material. Since the “input” is a coded version of what is happening in the GUI, there is an automatic connection between the image (the Pure-Data GUI) and the sound (the coded representation of the GUI activity as seen by the DSP). The socket data is interpreted in two ways; first, in a raw form that simply feeds an oscillator and second, in an interpreted form where the patch searches for certain keywords and extracts codes to be presented in the image and transform the sound. The image we view is both the logic and diagram of the program as well as the hidden communication that happens inside Pure-Data.
The generative process of “Reflex” works as follows:
- Start with a simple patch that sonifies the socket data
- In response to the sound, build the patch to transform the sound
- Any changes in the patch effect the socket data
- This socket data then gets fed into a future iteration of the patch