“Self-Other Organizing Structure #1”
Seizures, Blindness & Short-Term Memory

PDF Document

[B. D. R. Bogart. Self-Other Organizing Structure 1: Seizures, Blindness & Short-Term Memory. In Andrew Brouse, editor, Proceedings of the Second International Pure Data Conference, Montréal; August 21-26, 2007, pages 1–9. Pure Data Conference 2007, 2007.]

“Self-Other Organizing Structure #1” (SOOS1) is the first in a series of site-specific responsive installations. Rather than depending on the artist to define how these works relate to their site, the task is given to the artwork itself. The structure of the artwork changes in response to continuous stimulus from its context. Context is framed as parameters perceivable by the system that make its place in space and time unique.

As viewers enter the installation space they are able to see out onto a public street through a large window. Hanging in front of the window is a diptych of screens. One screen, the memory system, shows an abstracted grid of images from the street beyond. The free association system visualized on the second screen presents a montage of images from the systems past experience. A third screen shows the systems view of the world as seen through its camera. Below the two screens is a small robot camera that pans and tilts to examine the world around it.

1 Introduction

This paper describes and frames the first artifact in a body of work that aims to create artworks that find their own relationship to their context. These artifacts are embodied, meaning that are manifested in a physical form and are effected by the world around them. My research aims to use artistic inquiry to develop theory that binds ideas from responsive artworks, artificial intelligence, the philosophy of Merleau-Ponty and site-specific art through the practice of creating embodied artifacts-as-processes. Artifacts-as-processes are artifacts–objects created as a result of human activity–where their material of composition is language that encodes a particular process (software). By embodied I am referring to processes that are causally connected to the outside world through sensors and/or actuators. How can an artifact–even a process–find a relationship to its context? Artifacts such as SOOS1 form a relationship with their context by first being embodied so that can have access to their context, and second by having their structure changed in the process of embodiment.

The paper begins by weaving the theory which has been, and is still being, developed in this research project. The theory is focused on the fundamental relationship between an artwork (artifact), the artist (author) and the world in which they are embodied.

1.1 The RealizationInterpretation Loop

The artistic process in the creation of artifacts is made up of two iterative sub-processes, realization and interpretation. Realization is the path of intention from the artist to the world, where interpretation is the path of intention from the world back to the artist. Realization happens when the artist chooses to change the world in some way, to make a choice than manifests physically, for example choosing the colour yellow for particular region of a painting. The assumption is that this manifestation somehow encodes the intentions of the artist–yellowness in a particular spot means something for the artist and is meant to represent that meaning–but in reality this process flattens the process of creation into a representation that is no longer connected to the ideas that informed its manifestation. What the painter thinks yellow means may not be the same as what others do. The artifact is divorced from its concept–the abstract intention (expression) of the artist.

Interpretation is when the artist observes and experiences the results of the realization for example seeing yellow in a particular spot on a canvas. The tension between what the artist intended and what the artifact is actually offering is the context in which the next choices of realization are framed. The painter may decide that the tone of yellow is not quite what he or she wanted and adds more white to the paint. Realization and interpretation then become a dialectic iterating over the artifact where the next choices are the synthesis which are manifested as changes to the artifact. The artist’s concept colours his or her subjective interpretation of the artifact, which biases him or her to interpret the artifact in a way that may only be valid for his or herself. For those subjects that where not part of the realizationinterpretation loop are likely to interpret the artwork differently than the artist.

1.2 Artistic Inquiry

The artifacts of modernist artwork are often seen as products of artistic genius. There is a mythology surrounding the “creative genius” and the artifacts the–predominantly male–artist creates. The artifact is a record of genius which is collected and fetishized. Since the artifact is manifested in the physical world, why should the artist have more authority to define the meaning of an artifact than those subjects not biased by knowledge of the realizationinterpretation loop?

Some contemporary approaches to art practice reject this notion of artwork as expression of genius and break the mythos of creativity by shifting the emphasis away from the artifact toward the process itself. What is produced when the purpose of the work is not the creation of an artifact but an exploration of creative process? There are two products of this inquiry. The first is the artifact-as-process itself. The second is the knowledge the results from the artistic inquiry. This knowledge is manifested both in the artifact-as-process and around it through documentation, rhetoric, sharing and discussion.

This artistic inquiry is centered on the artistic practices of responsive electronic media, and site-specificity. For a survey of electronic media art see “Information Arts” [?]. The discipline of site-specific artwork aims to create work that gives “…itself up to its environmental context, being formally determined or directed by it” [?]. Minon Kwon [?] and Nick Kaye [?] provide a background on the site-specificity. Since the aim of this body of work is to explore the qualities of embodied creativity–through the development of artifacts-as-processes that find their own relationship to their context–it is only through artistic inquiry, as apposed to scientific inquiry, that this subject can be appropriately explored.

1.3 Embodiment

SOOS1 is an embodied system. It is realized as a physical structure in the physical world and is effected by the world it inhabits. Once embodied the material–the software that encodes the process–ceases to be a representation and is executed in the physical context. The material shifts from the representation of a process to action in the physical world. This embodiment is informed by the philosophy of Maurice Merleau-Ponty where the mind (realm of the concept) is not independent of the body (realm of the world). I apply Merleau-Ponty’s rejection of dualism in the mind-body to the dualism between the artistic intention (realization) and the meaning of the artifact (interpretation). In order to unify the realization and interpretation the artist must be able to accept the artifact-as-process as it acts in the world. The interaction between the software and the physical world is not totally deterministic–as it is in a simulated environment–the artist must relinquish control and allow the process to be driven equally by the realization (thesis) and interpretation (antithesis) the synthesis of which is encoded in the artifact-as-process. In the case of SOOS1 the entire project is dependent on the artifact-as-process acting outside of the intentions of the artist.

2 Growing Form from Context

In order to have SOOS1 find its own relationship to its context–and since the umbrella of this work is about exploring creative processes–it is natural to look to cognitive science as a source for how an embodied entity can both relate to its context and act in a way that is not entirely predetermined. A primary application of cognitive science happens in the discipline of artificial intelligence which seeks to create software that exhibits some of the properties of human beings. In order for SOOS1 to find its relationship to context the use of unsupervised connectionist artificial intelligence approaches are appropriate as the behaviour of the system is not dependent on a knowledge-base provided to it. Since SOOS1 is an embodied system, the physical environment becomes the “training” data for the artificial intelligence.

2.1 Methodology of Artistic Inquiry

As this research project is contextualized as a primarily artistic inquiry it is important here to describe how the creation of these artifacts will approached. The software development process will happen in an embodied context–the software will be build-up piece by piece while the system is connected to its context–parts will be initially developed in isolation and attached to the rest of the system as early as possible. Software development will happen in two modes, the first mode will be an intuitive approach that serves to get the basics of the system up and running–with arbitrary choices and placeholders–so that the system can be quickly evaluated in context. The second mode of grounded refinement will be going back over the work produced in the first mode, removing arbitrary choices by situating them in the theory behind such systems as Kohonen “Self-Organized Maps” and the theories of creativity such as present in the “cognitive mechanisms underlying the creative process” [?]. Random variables and placeholders are replaced by variables that refer to aspects of the embodied context. The software development happens in the Pure-Data visual programming system, and each step of development is managed by the subversion version control system. Each iteration (change of software) is evaluated on two bases, the artist’s phenomenological experience of the behaviour of the artwork, and the artist’s phenomenological experience of the software development process itself.

2.2 Why use Artificial Intelligence?

In order to answer this question I first need to define artificial intelligence. A general definition from [?] states that AI is “part of computer science concerned with designing intelligent computer systems, that is, systems that exhibit the characteristics we associate with intelligence in human behaviour—understanding language, learning, reasoning, solving problems and so on.” Although this definition does not directly mention creativity, Boden argues that machines can “appear” to be creative [?] in the same way that machines could be thought of as intelligent according to Turing. Stephen Wilson considers AI’s relationship to art:

Artificial Intelligence is one of these fields of inquiry that reaches beyond its technical boundaries. At its root it is an investigation into the nature of being human, the nature of intelligence, the limits of machines, and our limits as artifact makers. I felt that, in spite of falling in and out of public favor, it was one of the grand intellectual undertakings of our times and that the arts ought to address the questions, challenges, and opportunities it generated.[?]1

In this project I expect the system to be creative by it defining its own relationship to its context. Further I expect that the artwork makes creative choices that manifest themselves the the physical context of the work. AI is the only discipline–through its roots in cognitive science–that explores those questions of creativity through the creation of systems that embody aspects of the human mind. For this reason AI is the first logical discipline to look towards for technique and theory.

What techniques and processes from AI could allow an artwork to form its own relationship to context? To use non-AI software techniques I, as the artist, would still be determining how the work relates to its context rather than it finding its own connection. Those AI techniques, such as self-organizing maps–that allow the system to reorganize itself based on sensor input–are a likely requirement to build the mechanisms for indeterminate contextual response.

3 SOOS1 Architecture

The SOOS1 architecture is made up two interrelated mechanisms that work together to allow a creative act in response to the artifact-as-process’s embodied context. The memory system is the mechanism through which the system stores its previous experience. The memory system is based on a Kohonen Self-Organizing Map which compares all of its previous experience to its current experience in order to situate current experience in terms of previous experience. On top of the memory system is an independent connectionist network. The purpose of this connectionist network is to allow a creative free association to propagate through the memories and is based on the work of Liane Gabora [?].

The direction the camera looks at is controlled by a random variable which is seeded by the current time. A new camera position is stimulated when there is no activity in the free-association system. The longer the free-association the longer the camera will stare at the stimulus that initiated it.2

3.1 Memory System

As the camera explores its context the system creates a field of experience which is organized using a class of Artificial Neural Networks (ANNs) known as Kohonen Self-Organizing Maps (SOMs). The camera image is fed into the computer as a full-frame 30fps video stream. Each 4 seconds the 12×12 node SOM is fed with a twelve pixel (4×3) RGB sub-sampled snapshot of the video stream. A SOM associates each input image with a particular output.3The outputs can be thought of as categories of experience. When the outputs (categories) are plotted on a 2D Cartesian grid the inputs are then represented as a “feature” map where images that are similar tend to move closer together and images that are dissimilar tend to get repelled. This organizes the experience into a series of regions that contain similar experiences that are separated from areas of dissimilar experiences.

Since the system is constantly experiencing its context, the SOM must be continuously training in order to incorporate new experiences into its structure. To accomplish this, both the learning and neighbourhood functions–those functions that control the rate at which the self-organization evolves and is refined–are controlled by a cosine equation, scaled to range from 0 to 1, that operates on a cycle of approximately 42 minutes. The learning and neighbourhood rates control how many nodes should be changed for each new input. In a normal SOM the learning and neighbourhood rates start high–where each new input changes the weights of all the nodes–and slowly decrease so fewer and fewer nodes are changed by new stimulus as training continues. The learning rate eventually hits 0 when training is considered complete and the SOM accurately represents its training data. This choice allows for the SOM to respond to a continuous flow of new data and still be able to effectively–though not always perfectly–sort it into a constantly reorganizing field of memory. Kohonen networks were intended to work on a finite data-set. By using a cyclical symmetrical function for the learning rate, which allows the integration of new experience, there is an equal amount of time the network is undoing its previous work (based on previous experience) as it is contributing to it. This is due to the learning rate increasing over time during some cycles rather than constantly decreasing. The effect of this is that the network moves between cycles of large (approximate), and small (refined) placement of items in the map. SOOS1 currently uses the same decreasing cosine equation for both neighbourhood and learning rates.

A single unsubsampled image is stored for each node in the network, which is represented as a grid of captured images on one of the screens. The poor resolution of the abstracted image, that is fed into the SOM, makes the system very poor at differentiating spacial relationships between images. A sharp image and blurry image are seen as identical to the SOM. Since the camera video stream is sampled each 4 seconds the SOM is only able to store approximately 10min worth of experience. Since the SOM selectively replaces previous experience with new experiences–selected when the new experience is similar enough to the past experience that it fits in the same category–the memory is greatly increased for those experiences that are dissimilar to the most frequent ones.

3.2 Free Association System

Each time a new experience is perceived that perception sets forth a stimulation within the content of the memory, calling up similar experiences from the past. These new stimuli in turn stimulate other experiences, traversing the memory from the similar to the dissimilar. As the traversal progresses the energy in each stimulation decreases. Each subsequent experience is stimulated less than the previous one. As the free association traces a trajectory through the systems experience the memories intersected by that path are visualized as a cinematic montage on one of the screens.

The model of stimulation and propagation is a custom connectionist network made up of pure-data abstractions. When the camera looks in a direction in physical space, but before its image is added to the SOM, the SOM node that most resembles that input is sent a signal which activates it. Each node that is activated chooses a range of directions at random in which to propagate that signal. The signal sent to its neighbours is decreased by a percentage so that the cascade of activations falls off proportional to the distance between the initial activation and each node. In addition when a node is activated it sends an inhibition signal to the node which activated it. For a temporal delay that node will not propagate any new signals. The inhibition and directional control of propagation was needed to keep the system from over-stimulating itself. Early implementations simply used up all the resources on the hardware only moments after the initial stimulation. This over-stimulation behaviour corresponds to seizure activity in the human brain.

The mechanism behind this process is inspired by the work of Liane Gabora on the “cognitive mechanisms underlying the creative process”. Gabora’s theory considers creativity as a controlled form of free-association. The cascade of activations resemble how free association could work in the human mind. In Gabora’s theory the network of memories is different in two ways compared to what has been implemented in SOOS1. Firstly Gabora considers memory as sparse, whereas the SOM organizes content into an organized spacial grid where all nodes are associated with some input after training. Second SOOS1 stores entire images whereas Gabora’s model of creativity considers each memory node as micro-features of stimulus rather than as entire regions of stimulus.

4 Machine Creativity

Boden defines creativity as “…the ability to come up with ideas or artifacts that are new, surprising and valuable[?]. In my research domain the aspect of newness is the focus above surprise and value. As SOOS1 is meant to structure itself based on its embodied negotiation in its environment, newness comes from its ability to be different for each new context, as well as to change over time as its context shifts. The diversity and complexity of the real-world environment should guarantee that the system never receives an identical stimulus twice. The system should exhibit “surprise” at minimum to the same extent as its context. The value of the project is not in its creative act, but in the process that makes it possible. Boden specifies three classes of creativity:

  • Combinatorial creativity is linking together known ideas that are not already associated.
  • Exploratory creativity is accomplished by moving through the space of possibilities.
  • Transformational creativity is the alteration of the space of possibilities.

Combinatorial creativity is inevitable in a connectionist network that supports learning. This is because the shift of the unit weights changes the topology of the network, which is combining the stimulus from the inputs in various ways. Exploratory creativity is also present in these systems, since the space of possibilities is limited by the number of units in the network. In order for a connectionist network to exhibit transformational creativity it would have to be able to change the space of possibilities. SOOS1’s current combination of the SOM and a model of free-association allow it to be exploratorally creative since the free-association traverses through its memory. At the very least the memory, at a snapshot in time, serves as the space of possibilities from which it can choose to be creative. Since the space of possibilities in the memory system is a constantly shifting field of experience SOOS1 is also transformationally creative through its ability to add to its space of possibilities over time. Due to this the same memory traversal (which is already unlikely to repeat itself) occurring at different times would to yield totally different results. As the SOM is 12×12 grid of possibilities at a fixed moment in time it has a fixed space of possibilities. The use of an Adaptive Resonance Theory (ART) network would allow the memory system to create a new category for a new stimulus without effecting the categories of previous experiences. Then the space of possibilities would increase over time as the system gains more experiences.

Consider creativity as a two step process. Some generator, the kernel of creativity, creates a “new” stimulus. This stimulus then goes through a process of evaluation that filters all but the most “fit” ideas. In my case the generator for creativity is the context. Boden largely concentrates on the evaluation aspects of creativity and spends little time on the generator. Boden’s conception of creativity in fact sets up an emphasis on the evaluation of creativity above the seed that makes it possible. In SOOS1 there is no mechanism that serves the role of the evaluator. That is not to say that SOOS1 should not be able to evaluate its own creativity, but that that evaluation should not be prespecified but come as a result of its embodied process. It is unclear at this stage how context could provide criteria for evaluation. SOOS1 is a generator for creativity. Boden’s argument can be summed up in one statement: A creation can only be considered “creative” if it has been successfully evaluated as such.4 Of course these two steps are both required for a creative result. Emphasizing one over the other is to only create a partial model of creativity. Worse still would be to reduce creativity to evaluation since without the seed the mechanism of evaluation has nothing to evaluate. The result of the first step in isolation may not create something highly creative, but the result of the second step in isolation creates nothing at all. Without the evaluation the seed cannot produce something that is “new, surprising and valuable”. The hierarchy is clear, creation (if not creativity), is the domain of the generator, not the evaluator. As a counter to most of the literature in the area I aim to put more focus on the seed of creativity as apposed to its evaluation. The long-term challenge SOOS1 aims to address is the possibility of a seed of creativity not based on randomness but on an embodied process. This aspect of the research connected with artificial life research which is tied to abiogenisis5. The seedevaluation problem is analogous to a central concern of abiogenisis. Was it random fluctuations of early organic molecules, or some form of self-organization, or natural selection, or process as of yet undiscovered, that made life possible? The theories in abiogenisis are a source of technical and philosophical ideas important to the creation of artworks that relate to their context and are not predetermined in structure.

4.1 Machines That are Intended to be Creative

This section will discuss a small subset of artistic projects that involve aspects of machine creativity. They are all systems that have been implemented in computer systems. These projects involve both connectionist and non-connectionist approaches.

One of the most notable examples of a “creative” machine are the AARON programs written by Harold Cohen starting in 1973 and continuing to the present. As a collective of programs AARON can “create” in a number of different painting styles. Each style uses a different variant of AARON which implements a different set of compositional rules. Some examples of these variants are “abstract AARON” which creates abstract landscapes, “acrobat AARON” which creates acrobatic figures, and “jungle AARON” that creates scenes of figures in a complex jungle ground that evoke Gauguin. AARON programs contain sets of rules that encode specific compositional and stylistic rules that are specified by Cohen. Each element in the paintings–the figures, grounds, and objects–is each a representation of the model those rules encode. These rules are chosen by the system based on weighted randomness [?] and applied to create paintings that are drawn by a robot. AARON has no feedback of the results of its actions on the canvas as is then totally blind. Due to this the painting has no effect on AARON’s internal structure and therefore the system is not embodied and exhibits the ultimate example of modernist creation where the internal model (concept) is realized in that perfect theoretical vision–regardless of the actual properties of the artifact in the world–and there is no interpretation of the artifact in the machine’s creative process. This fact shows AARON to be an extension of the modernist conception of the artistic genius. I would argue it could only be considered creative in a symbiotic relationship with its creator. Cohen believes that this software system is a natural approach to art-making because artistic composition is rule-based. While I can agree that graphic composition in a certain particular style of painting can be considered rule-based It does not follow that all aspects of artistic creation are. The AARON software exhibits combinatorial and exploratory creativity, but not transformational creativity since it is unable to compose any choice that has not already been defined by rules specified by Cohen. Further without an ability to perceive the results of its action in the world it will never be able to reflect on its own process.

In 1981 David Cope started writing “Emmy” or “Experiments in Musical Intelligence” in order to deal with a creative block in his own composition. The project started as an effort to automate the compositional process, by using the style of Cope’s own compositions to date. The software uses a variation of Augmented Transition Networks (ATNs) which were created to model the syntax of natural languages. He used this as a basis of a system that models the structure of musical compositions and creates “signatures” from the common aspects of multiple compositions. These signatures are then used in a second process to combine the elements of the signature into a new work that exhibits the style of the source composer. Clearly using combinatorial creativity the software recombines the structures it sees in source-work. Since the space of possibility is limited to the “signature”, created from data, the system is unable to perform exploratory or transformational creativity. The system is also only fed abstractions of compositions as source material, and is unable to perceive, let alone evaluate, the results of its processes.

David Rokeby has created two works that can be considered creative. “The Giver of Names” was first exhibited in Toronto, Canada in 1997. The system perceives the outside world through a video camera pointed at a pedestal. The floor around the pedestal is scattered with children’s toys the audience is free to place in the camera’s view. The software attempts to give names to the objects in its view. Associating the colour and shape with concepts in its knowledge-base the system creates a free writing passage inspired by those objects. The system certainly shows combinatorial creativity by pulling words from its relational database to create texts. It is unclear if the system exhibits exploratory creativity since it is unclear if its network of associations change in response to experience. Further the choice of where to begin within the associative database is not a result of agency in the system.

“n-Cha(n)t” was first installed in 2001 in Banff, Canada and builds on some of the ideas of language and interpretation that are embodied in “The Giver of Names”. “n-Cha(n)t” is a cluster of independent systems that are connected in what could be considered a connectionist network through ambient audio. Each of the nodes is able to both hear6 and speak7 by accessing a relational database similar to the one used in “The Giver of Names”. The hearing process attempts to interpret sound from a microphone input and translate it into text. When the system recognizes a word through the hearing process it is then passed onto the speaking process where a voice synthesizer recites it aloud. The hearing apparatus is a highly directional microphone that picks up only sounds nearby. Without any external interaction all nodes chant the same word over and over again, each node picking up the sound from other nodes forming a reinforcing pattern. When a sound from the non-local environment is heard, the nodes differentiate as each hears, and speaks, a different interpretation of the fluctuating environmental sound. Without further interference one of the interpretations takes dominance and eventually all nodes are repeating that dominant pattern. Both “Giver of names” and “n-Cha(n)t” are embodied systems that are both attached to the physical world through sensors that allow it to respond to their context. Going further “n-Cha(n)t” attempts to realize (through speech) and interpret (through speech recognition) the results of its collective action and therefore shows an example of embodied creativity.

George Legrady’s “Pockets Full of Memories” was made possible by a commission from the Centre Pompidou Museum of Modern Art in 2001. Revised versions of the project were revisited in 2003 and exhibited in the Dutch Electronic Arts Festival in Rotterdam, Netherlands. “Pocket’s Full of Memories” is one of the few artistic projects that makes use of a connectionist network. Specifically the system uses an implementation of the the Kohonen SOM to organize content provided by the audience. The installation consists of a large projection and a number of kiosks with flat-bed scanners. The audience is encouraged to scan an image of some artifact in their possession. The kiosk then prompts the participant to answer questions about the meaning of the artifact. The answers to those questions are then stored in a database bound, as meta-data, to the images from the scanner. This meta-data is fed into the Kohonen network which plots the images in a projection. The position of the artifact in the projection is a result of the categorization process of the Kohonen network. Artifacts attached to similar meta-data are plotted closer together than artifacts with dissimilar attributes. The Kohonen map used in this project is a collaboration between George Legrady and Dr. Timo Honkela, who conducts research into artificial systems to study cognitive processes.

5 Connectionist Approaches to Artificial Intelligence

In “A Brief History of Connectionism” [?] Medler traces the roots of connectionism from Aristotelian associationism to contemporary connectionist research. Connectionism is a revision of some ideas rooted in empiricism and associationism. Both associationism and empiricism consider the role of the environment as integral to human behaviour:

Empiricism emphasizes the role of culture, education and life experiences as determinants of human abilities and proclivities, while associationism identifies pairwise links between individual elements of experience, either subjective or behavioral, as the main process of such psychological change. [?]

In terms of the importance of context these roots of connectionism are methodologically compatible with my research interests. Connectionism in the contemporary sense, coined as “New Connectionism” by Thorndike, “is characterized by computationally powerful networks that can be fully trained” [?]. These networks act as both “…very powerful information processors…” and as “…arbitrary pattern classifiers” [?].

Connectionist systems are networks of simple units that combine to form complex structures that act as Parallel Distributed Processors (PDPs). Connectionist networks are inspired by human neurophysiology.

5.1 How Do Connectionist Approaches̛ Relate to this Project?

The central purpose of this research program, to allow artworks to define their own relationship to context without the artist defining them a priori, closely fits with connectionist approaches according to Walker:

The recent origins of PDP are in “random self-organizing networks” and its goal frequently seems to be to account for perception with the minimum of innate preconditions.

PDPs are “random self-organizing” in that the units are interconnected to a degree defined by a “weight” associated with each connection. The structure of the network is changed by the adjustment of the weights. The change of weights results in some units to be connected to a greater extent than others. Often initially randomized the weights are then tuned through a training process. The training process allows the network to learn by associating certain inputs with certain outputs. The memory system in SOOS1 is then a pattern classifier that aims to organize experience by classification. A connectionist model is also used in the free association module where the network is a medium through which signals are propagated. The nodes in the free association module are all connected to the same degree.8

5.2 Overview Survey of Historical̛ Connectionist Approaches

As a general overview of some connectionist approaches to artificial intelligence I will be discussing the Perceptron, Pandemonium, Kohonen Networks, Multi-Layer Perceptrons and the General Delta Rule. These serve as examples of significant works and not as an exhaustive survey9.

Conceptualized by Rosenblatt in 1957 the Perceptron “has come to represent the genesis of machine pattern recognition” as described by Medler [?]. The Perceptron is a network of nodes, organized into a hierarchy. A subset of these nodes serve as inputs, which are connected to a subset of “Association” nodes, which are in turn connected to a subset of “Response” nodes. Different types of Perceptrons have different network topologies. For example the α-Perceptron only allows a limited number of response units connected to the association units, and “Mark 1” Perceptrons only allowed weights within a limited range10, restricting the variation of topologies possible in the network structure. The “Mark 1” Perceptrons in particular were highly criticized for their lack of computational power by Minky and Papert [?].

Selfridge in 1959 published “Pandemonium: A paradigm for learning.” [?] where one of the first examples of a neurophysiology inspired PDP network is described. Pandemonium was a network capable of adapting itself to solve problems that could not be entirely specified in advance. This model was among the first to include a network of nodes, organized into four separate layers, each unit processing in parallel. Each node had a weight associated with each connection. The initial state of the weights was predetermined for a particular task a priori. The weights of nodes in two particular layers11 were altered through a supervised hill-climbing learning method. The nodes only in these layers were subject to a second learning system, which can be considered a genetic algorithm, which was used to maintain the most successful nodes and remove the unsuccessful ones. The successful nodes were mutated or bred together in order to create new nodes as variations of their clone/parents. Pandemonium was quite successful being able to recognize a series of dots and dashes, from manually keyed Morse code, and acting as a optical character recognition system able to recognize ten hand-written characters.

Also known as a Self-Organized Map (SOM) a Kohonen network is designed expressly for the purpose of classification. Similar to other models it consists of a number of nodes connected in a network with weighted connections. Unlike Pandemonium there are only two layers of nodes. The input nodes are connected directly to the output nodes. The network is one of the first to make use of an unsupervised12 learning technique. A result of this learning method is that it is not possible to know which output node will be associated with which class of input before executing the categorization process. The association between nodes and inputs is then a function of the self-organization of the network and will vary even when using the same input data. Kohonen networks cluster input by value, where inputs that are most similar stay nearby on the “feature map” and inputs that are different stay far apart. As the frequency of an input pattern increases so does the size of the associated area of the feature map.

Multi-Layer Perceptrons (MLPs) are Perceptrons that include a number of “hidden layers” that stand between the input and output layers. These Perceptrons are significantly more powerful then the “Mark 1”, but until very recently there was no way of training MLPs. Without training the topology of the network must be predetermined and is designed to solve a particular problem. The advent of the Generalized Delta Rule (DGR)13, by Rumelhart, Hinton, and Williams, allows MLPs to be trained. The GDR is a generalization of the delta rule learning system created by Widrow and Hoff for Adaline networks. The GDR allows the supervised training of a Multi-Layer Perceptron regardless of the organization of the network. This allows the Perceptron model to break free of most of the computational limitations highlighted by Minsky and Papert. The Multi-Layer Perceptron is in the class of general PDP networks.

6 Conclusion & Future Work

SOOS1 is the first is what I hope to be a long and varied body of work. The system is, at the time of writing, running in a long-term installation at Simon Fraser University where it will remain the platform of development for my thesis work. This stage of development concludes the first phase of intuitive development. The next stage will be to reflect on the behaviour of the system, and using that knowledge go back through the software to reconsider the arbitrary and intuitive choices.

Short term goals for SOOS include experiments to see the results of both increasing the size of the SOM (above 12×12 nodes) will be a reasonable way to increase the short-term memory of the system. I will also explore how much data it is practical to send into the SOM beyond the 4×3 pixel image so that it is able to make a more fine-grained assessment of the relation between experiences. The free-association model needs to be reevaluated. At this time it is possible for a memory stimulation to activate only a single node, without that signal being propagated through the network, due to the random direction of propagation. In order to make the free-association model more closely resemble neuron function there needs to be a reinforcement of patterns that are often propagated through the network. Using such a model could be a hint to an approach for memories of creative experiences since the more often a particular node gets fired from a particular neighbour the more easily it would be able to do so in the future, which would store a particular pattern of free-association in the network.

The medium term goals are to remove the arbitrary random variables, and replace those with variables from the sensed environment. The random variable that controls the direction the camera is looking at should be connected to some, as of yet undetermined, parameter of the free-association. So the choice of what the next stimulus to the network will be a result of the previous stimuli of the network.

In the long-term the SOOS installations are intended to become permanent and self-sufficient. This brings a whole range a new issues from a physical embodied stand-point to keep the installations up and running in the long term. By self-sufficient I mean that the installations should not be dependent on outside infrastructure, power nor telecommunications.

This exploration of embodied creative machines is highly suited to the use of connectionist approaches to artificial intelligence. In particular unsupervised PDP networks are ideal as they are literally design to change their structure in response to sensory input. These methods make the goal of creating systems that respond to their context without being depending on a priori knowledge possible.