Posted: May 1, 2014 at 5:33 pm

In order to deal with the problem of static dreams Philippe asked me to create a synthetic data set that has particular temporal properties. The idea is that we can use it to get a sense of both the distribution of percepts over time and the resemblance / boundedness of dreaming and mind wandering compared to perception. The data-set is 2000 frames and contains three objects: a blue circle that gets bigger, a red circle that gets smaller, and a green rectangle that moves from the left to the right. The background toggles between white and grey every 200 frames. Additionally, there was a single all black frame at the end of the data-set to mark epochs. Following are the first and last frames of the synthetic data set, not including the trailing black frame:

The change in background colour triggers arousal and therefore perception in the system. Thus the data-set was sampled 10 times over the 2000 frames. The max number of percepts was set to 30 to allow each state of each shape to have its own percept where no merging is necessary. The DM system was trained on these frames and repeated over 50 epochs. Following is a plot of the test:

All of the dreams and mind-wandering results of this test showed static results. The following image shows all percepts dumped by the system after one epoch stacked on top of one another:

I extracted the state vectors (presence of clusters) that correspond to training iterations to obtain the following:

As there were enough clusters, the training samples fed to the predictor are stable over every epoch. I extracted one epoch from this test and wrote a new program to investigate MLP learning of this data, whose results follow. First we’ll look at a number of tests running over 15 epochs where the aim to learn the sequence. In the resulting images below each column is the network output and corresponds to each input column. The output images are offset by one pixel as no t-1 pattern exists. Both thresholded and raw output values and shown below.

Input Sequence	3 Hidden Units Constant Learning Rate	15 Hidden Units Constant Learning Rate	15 Hidden Units Decreasing Learning Rate

The example with three hidden units corresponds to the current implementation of the Dreaming Machine where the number of hidden units is set to a 10th of the number of network inputs. The network with three hidden units did not learn the sequence well no matter how many epochs were used. When the number of hidden units is half the number of inputs, learning over 15 epochs is more successful. The following plot shows the error rates of various tests after each epoch:

For this data-set, over 15 epochs, the difference between using a linearly decreasing learning rate from 1 to 0 and a constant learning rate of 1, was not significant in terms of the quality of results (although the decreasing learning rate does report lower error). As shown above, the sequence is well learned in all cases when 15 hidden units are used. Unfortunately, this sequential learning is not manifest in feedback in the predictor. While this network does produce state t+1 when fed state t, feeding the network state t does not lead to outputting states beyond t+1 (t+2, t+3, etc.) when feeding t+1 back as an input to the network. The following table shows these results:

Constant Learning Rate Discretized Feedback	Constant Learning Rate Raw Feedback	Decreasing Learning Rate Discretized Feedback	Decreasing Learning Rate Raw Feedback

While the network will produce the correct first state (t+1 from t), if t+1 is fed back as an input, then the learned sequence is not reproduced. In order to augment the feedback process we used two methods: In discretized feedback the continuous output units are thresholded before they are fed back into the network. In raw feedback the continuous output of the network is fed back without modification. In any case, the network stabilizes after a small number of iterations at which point it either produces a periodic or static pattern. The parameters used to train these networks did not appear as significant as the initial network weights to the networks feedback behaviour. For example, the following results are the output of non-discretized feedback in the same network using a constant learning rate, 15 hidden units and 15 epochs where the only difference is the initial network weights:

From what we have learned here we have two concrete implications for the Dreaming Machine: (a) the number of hidden units may need to be increased considering the number of inputs in the full system (4000 percepts); and (b) even if the network is learning predictions, feedback will not manifest those predictions. The periodic and static dreams of the system shown to date are caused by the lack of predictions being manifest in network feedback. This is a significant problem, and likely means a different machine learning method for the predictor is needed. Following is the code used to generate these experiments: