Posted: May 1, 2014 at 5:33 pm
In order to deal with the problem of static dreams Philippe asked me to create a synthetic data set that has particular temporal properties. The idea is that we can use it to get a sense of both the distribution of percepts over time and the resemblance / boundedness of dreaming and mind wandering compared to perception. The data-set is 2000 frames and contains three objects: a blue circle that gets bigger, a red circle that gets smaller, and a green rectangle that moves from the left to the right. The background toggles between white and grey every 200 frames. Additionally, there was a single all black frame at the end of the data-set to mark epochs. Following are the first and last frames of the synthetic data set, not including the trailing black frame:
The change in background colour triggers arousal and therefore perception in the system. Thus the data-set was sampled 10 times over the 2000 frames. The max number of percepts was set to 30 to allow each state of each shape to have its own percept where no merging is necessary. The DM system was trained on these frames and repeated over 50 epochs. Following is a plot of the test:
All of the dreams and mind-wandering results of this test showed static results. The following image shows all percepts dumped by the system after one epoch stacked on top of one another:
I extracted the state vectors (presence of clusters) that correspond to training iterations to obtain the following:
As there were enough clusters, the training samples fed to the predictor are stable over every epoch. I extracted one epoch from this test and wrote a new program to investigate MLP learning of this data, whose results follow. First we’ll look at a number of tests running over 15 epochs where the aim to learn the sequence. In the resulting images below each column is the network output and corresponds to each input column. The output images are offset by one pixel as no t-1 pattern exists. Both thresholded and raw output values and shown below.
Input Sequence | 3 Hidden Units Constant Learning Rate |
15 Hidden Units Constant Learning Rate |
15 Hidden Units Decreasing Learning Rate |
---|---|---|---|
![]() |
![]() ![]() |
![]() ![]() |
![]() ![]() |
The example with three hidden units corresponds to the current implementation of the Dreaming Machine where the number of hidden units is set to a 10th of the number of network inputs. The network with three hidden units did not learn the sequence well no matter how many epochs were used. When the number of hidden units is half the number of inputs, learning over 15 epochs is more successful. The following plot shows the error rates of various tests after each epoch:
For this data-set, over 15 epochs, the difference between using a linearly decreasing learning rate from 1 to 0 and a constant learning rate of 1, was not significant in terms of the quality of results (although the decreasing learning rate does report lower error). As shown above, the sequence is well learned in all cases when 15 hidden units are used. Unfortunately, this sequential learning is not manifest in feedback in the predictor. While this network does produce state t+1 when fed state t, feeding the network state t does not lead to outputting states beyond t+1 (t+2, t+3, etc.) when feeding t+1 back as an input to the network. The following table shows these results:
Constant Learning Rate Discretized Feedback |
Constant Learning Rate Raw Feedback |
Decreasing Learning Rate Discretized Feedback |
Decreasing Learning Rate Raw Feedback |
---|---|---|---|
![]() ![]() |
![]() ![]() |
![]() ![]() |
![]() ![]() |
While the network will produce the correct first state (t+1 from t), if t+1 is fed back as an input, then the learned sequence is not reproduced. In order to augment the feedback process we used two methods: In discretized feedback the continuous output units are thresholded before they are fed back into the network. In raw feedback the continuous output of the network is fed back without modification. In any case, the network stabilizes after a small number of iterations at which point it either produces a periodic or static pattern. The parameters used to train these networks did not appear as significant as the initial network weights to the networks feedback behaviour. For example, the following results are the output of non-discretized feedback in the same network using a constant learning rate, 15 hidden units and 15 epochs where the only difference is the initial network weights:
![]() ![]() |
![]() ![]() |
![]() ![]() |
![]() ![]() |
From what we have learned here we have two concrete implications for the Dreaming Machine: (a) the number of hidden units may need to be increased considering the number of inputs in the full system (4000 percepts); and (b) even if the network is learning predictions, feedback will not manifest those predictions. The periodic and static dreams of the system shown to date are caused by the lack of predictions being manifest in network feedback. This is a significant problem, and likely means a different machine learning method for the predictor is needed. Following is the code used to generate these experiments:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 | #include "floatfann.h" #include "fann_cpp.h" #include <ios> #include <iostream> #include <fstream> #include <sstream> #include <sys/time.h> #include "opencv2/highgui/highgui.hpp" #include "opencv2/core/core.hpp" #include "opencv2/imgproc/imgproc.hpp" #define maxIterations 50 using namespace std; using namespace cv; // get the current time (for rough profiling) double getTime() { timeval rawtime; double time; gettimeofday(&rawtime, NULL); time = rawtime.tv_sec+(rawtime.tv_usec/1000000.0); return(time); } // Test function that demonstrates usage of the fann C++ wrapper void run(string inputData, string networkFilename, string imageFilename) { double t1, t2; const unsigned int num_layers = 3; //const float desired_error = 0.01f; //const unsigned int max_iterations = 1000; const unsigned int iterations_between_reports = 1; string imageBasename = imageFilename.substr(0, imageFilename.rfind(".")); // load datafile FANN::training_data data; if (data.read_train_from_file(inputData)) { FANN::neural_net net; net.create_standard(3, data.num_input_train_data(), int(data.num_input_train_data()/2), data.num_input_train_data()); net.set_learning_rate(1); // was 2, 0.7 // steepness of activation function (slope of middle of sigmoid) net.set_activation_steepness_hidden(1.0); net.set_activation_steepness_output(1.0); // Types of activation functions: // FANN::SIGMOID is e.g. in the 0 - 1 range while FANN::SIGMOID_SYMMETRIC is in the -1 - 1 range and FANN::LINEAR is unbound. net.set_activation_function_hidden(FANN::SIGMOID_SYMMETRIC_STEPWISE); net.set_activation_function_output(FANN::SIGMOID_SYMMETRIC_STEPWISE); //net.print_parameters(); // scale data from 0-1 to -1 to 1 //t1 = getTime(); data.scale_train_data(-1, 1); // approximately 1.35359seconds for 41,886 inputs //t2 = getTime(); //cout << "data scale time: " << t2-t1 << endl; // Initialize and train the network with the data // TODO try changing this. net.randomize_weights(-1, 1); // Train network int epochs = 15; double learningRate = 1.0; for (int i=0; i<epochs; i++) { // epoch training //t1 = getTime(); //net.set_learning_rate(learningRate-((1.0/epochs)*i)); cout << "15epochs_15hidden_constantLearningRate_rawFeedback learningRate " << i << " " << learningRate-((1.0/epochs)*i) << endl; cout << "15epochs_15hidden_constantLearningRate_rawFeedback error " << i << " " << net.train_epoch(data) << endl; //t2 = getTime(); //cout << "canonical_100000_hidden/5_clampFeedback time " << i << " " << t2-t1 <<endl; } // automatic //net.train_on_data(data, 1.3840422E9, 1000, 0.001f); // Generate image from network Mat masterImageFeedback = Mat(data.num_input_train_data(), maxIterations, CV_8UC1); Mat masterImageFeedbackCont = Mat(data.num_input_train_data(), maxIterations, CV_8UC1); Mat masterImageIndex = Mat(data.num_input_train_data(), data.length_train_data(), CV_8UC1); Mat masterImageIndexCont = Mat(data.num_input_train_data(), data.length_train_data(), CV_8UC1); // Validate by each index // for each data point (row) for (unsigned int i = 0; i < data.length_train_data(); i++) { fann_type *calc_out = net.run(data.get_input()[i]); // for each input dimention (col) for (unsigned int j = 0; j < data.num_input_train_data(); j++) { float value = calc_out[j]; //float value = data.get_input()[i][j]; // populate image // continuous masterImageIndexCont.at<uchar>(j,i) = char( (value+1)*128 ); // discretized if (value > 0) masterImageIndex.at<uchar>(j,i) = char(255); else masterImageIndex.at<uchar>(j,i) = char(0); } } // Feedback validation // set initial input to first training sample fann_type *previousOutput = data.get_input()[0]; for (unsigned int i = 0; i < maxIterations; i++) { fann_type *tmpOutput = net.run(previousOutput); // TODO discretize values to -1 to 1 fann_type newInput[data.num_input_train_data()]; for (unsigned int j = 0; j < data.num_input_train_data(); j++) { newInput[j] = tmpOutput[j]; // populate image // continuous masterImageFeedbackCont.at<uchar>(j,i) = char( (tmpOutput[j]+1)*128 ); // discretized if (tmpOutput[j] > 0) { masterImageFeedback.at<uchar>(j,i) = char(255); //newInput[j] = 1; // discretized feedback } else { masterImageFeedback.at<uchar>(j,i) = char(0); //newInput[j] = -1; // discretized feedback } //cout << int(masterImage.at<uchar>(j,i)) << endl; } previousOutput = newInput; // feedback loop. } // Write masterImage to file imwrite(imageBasename+"-valFeedback.png", masterImageFeedback); imwrite(imageBasename+"-valFeedbackCont.png", masterImageFeedbackCont); imwrite(imageBasename+"-valIndex.png", masterImageIndex); imwrite(imageBasename+"-valIndexCont.png", masterImageIndexCont); // Save the network in floating point and fixed point net.save(networkFilename); //unsigned int decimal_point = net.save_to_fixed("xor_fixed.net"); //data.save_train_to_fixed("learn_sequence.data", decimal_point); } else cout << "Data file could not be loaded" << endl; } /* Startup function. Syncronizes C and C++ output, calls the test function and reports any exceptions */ int main(int argc, char* argv[]) { if (argc != 4) { cout << "Usage: " << argv[0] << " [input data] [network filename] [output image basename]..." << endl; cout << "Exiting..." << endl; exit(1); } else { try { std::ios::sync_with_stdio(); // Syncronize cout and printf output run(argv[1], argv[2], argv[3]); } catch (...) { cerr << endl << "Abnormal exception." << endl; } } exit(0); } /******************************************************************************/ |