System Architecture

Posted: February 17, 2012 at 4:59 pm

Following is a diagram of the current conception of the system. It is a high level overview where many details are omitted. A number of modules have been added from the last diagram, which are filled in blue. After looking more at LIDA it has become clear that it will not be that useful for this system. That being said, there are overlaps between these modules and some of the LIDA modules.

The object and spatial centric processors deal with dynamic and static segments of the stimulus, as the code to cluster instances in both cases is different. The perceptual buffer is the current state of the perceptual system. It includes all active percepts that have not been habituated to. Habituated percepts are those that are visually present, but because they have been seen so often, are not perceptually present. Both habituated and non-habituated percepts are of the same structure and stored in the same location, they only differ in how they are constructed. The conceptual network is a set of interlinked networks that represent higher level representations of the relations between percepts. This module will be discussed later. Finally, the circadian clock is a process that modulates the explicit wake-sleep cycle. It is influenced by the stimulus (brightness), and the state of the conceptual system and contributes to when the machine is sleeping and dreaming.

The preceding diagram represents the structure of the conceptual system. Each orange circle is a precept, which is tied to two sets of low-level features: where in space and time, and what in terms of colour and area. Unlike in a normal ANN, each feature is its own network. Area nodes can only attach to area nodes and so on. For each percept there is a strength attached to each feature. How this strength is determined is discussed in the next section. The strength of the percept is the sum of the strengths of the features.

In the above example the “small red” percept is activated. The area feature is the strongest, so it passes activation to the area feature of the “small green” percept because they are closest in feature space. The strength of the connection is inversely proportional to the distance between those features (similar percepts pass signals more easily than dissimilar percepts). Each time a signal is propagated that link it is strengthened. Once “small green” is activated, that activation is passed to its strongest feature: the green aspect of the “red-green” feature. Again, the activation is passed to the percept whose red-green feature is most similar to, and so on.

The reason for multiple linked networks is to enable associative links based on features, and allow link strengths to be feature-specific. Activation can pass between percepts through multiple paths, enriching the complexity of their interconnections. As activation is only passed through the strongest feature of a percept, that reduces activation of the network, and still allows activation to pass between networks, allowing associations as depicted above.


How is the strength of each feature determined for a particular percept? In the process of constructing percepts a density function is computed for each feature. This results in a probability distribution for each feature. The distance between the distribution and the stimulus feature provides a measure of the relative novelty of that feature. As all features are summed for a single percept, this indicates the novelty of the percept across all features.


Perception is a multistage process where raw stimulus data is abstracted into percepts that root the conceptual system. First, sensor images are segmented into contiguous regions of similar colour bordered by strong edges. As this aspect of the project is less emphasized, these regions are not meant to match human perception, but simply a means to break a raw image into patches for future processing. For each incoming frame the resulting patches are compared to all patches from previous frames. There are two methods of comparison, one tuned to static background patches and a second tuned to dynamic moving patches. When matches between patches are found, then those patches are merged into a percept. A percept is a unit of perception that is abstracted from stimulus data, and represents objects beyond individual presentations. When a stimulus results in the update or construction of a percept, that percept is activated in the conceptual system. The degree of activation is modulated by a bias that is proportional to the number of merges in the percept. This bias reflects the habituation of the system to that stimulus. If a percept has been activated often in the past, the bias makes it less likely to activate in the conceptual system.


When the system is dreaming it is cut off from sensory stimulus. Activation is not driven by stimulus but results from activation within the conceptual system as inspired by latent activation. As in a waking state, the activation within the conceptual system takes some time to dissipate, even after the disconnection of the sensory system. This latent activation is the starting point of a dream. The qualities of activation are modulated in a dream state compared to the waking state, and in the absence of a continuous stimulus drive, the associative process (propagation of activation) spreads through the system, calling up percepts, even habituated ones, freely.

What is the function of the dream? One hypothesis is that sleep and dreaming are part of memory maintenance. For example, a slow wave sleep (SWS) like state could decrease the activation of neurons and decrease connections between percepts (pruning). Afterwards a REM-like sleep state could rebuild the remaining connections, leaving only the strongest. Another possible function of dreaming in the system is the offline processing of the distances between features.

Hallucination and Day Dreaming

One of the major contributions of this work is a conception of perception and dreaming in relation to mental imagery. In short, the constructive aspect of perception is enabled by the same processes that enable mental imagery. We also argue that mental imagery and dreaming are highly related, and in fact the process of waking perception is as much dreaming as it is mental imagery. During waking, the associative and generative image-making abilities of the mind-brain are tightly coupled to stimulus. In the absence of stimulus (or in the case of all stimulus being habituated to) dreams and mental images are not constrained by stimulus and free propagate across broad associations.

What of the states in between? As a stimulus becomes habituated, the conceptual system is less activated, even in the presence of stimulus. If a large portion of the visual scene is habituated to, then a day dreaming or hallucination may ensue. In both cases a dream-like association will occur at the same time as latent perceptual activation, causing a fusion of imagination and perception.

Circadian Clock

What controls the state of the system? In humans, our sleep states are managed by a circadian clock which is influenced by the brightness of visual stimulus, exercise, social interaction, etc.. In the system, the circadian clock could be influenced by both the degree of activity in the conceptual network, and the brightness of the stimulus. The state of the system would then be a function of the degree of habituation to the current stimulus and the state of the circadian clock.


The system enables two rudimentary types of learning. First, habituation is the simplest form of learning where a continuous stimulus is increasingly ignored. Second, the linking of percepts in the conceptual network is the learning of relations between percepts.

Open Questions

My plan was to use hierarchical representations of concepts, but it is unclear how that would fit into the current conception. Firstly, it makes the current conceptual network more complex than it already is (adding features that represent the position of a percept in a hierarchy). Secondly, hierarchy is implicit in the pattern of activation in the conceptual network. The small activation of a small number of percepts corresponds to a small or flat hierarchy, while a larger activation where many percepts are active corresponds to a deep complex hierarchy.

How to deal with possible O(n²) complexity in distance functions? As the number of percepts increases, the number of distances between instances increases exponentially, as each percept must calculate its distance to all other percepts. One idea is to do this calculation offline during the dreaming state (when segmentation processes are minimal). This would mean that the cached distance values used in the conceptual system may be inaccurate, which could lead to unexpected associations between percepts. Another option is to prioritize the distance functions for outlier percepts that are less habituated.

If the system has a SWS-like state, will not all concepts be so minimally activated that there is no starting point for the next REM cycle? It would be best to keep the system activated, even to a lesser degree, at all times. This would be similar to brain behaviour, which is always active to some degree (until death).

If a distinction between REM and SWS sleep exists in the system, how do the qualities of those states (frequency, amplitude and synchrony) impact the function of those states on the system? ie. How can a high amplitude low frequency activation of many concepts at the same time diminish link strength?

There has been no mention in this conception of short or long term memory.