Clustering Test 8 (45,000 frames, 2000BG, 1000FG)
Posted: April 30, 2013 at 8:46 am
This is the first long term test covering material over an approximately 12 hour period, shown in the following plot. In this case there are 2000 foreground clusters and 1000 background clusters. The mean rendering time is 2.6seconds with a variance of 0.28.
Clustering and Performance Test (50 clusters, 20,000 frames)
Posted: April 25, 2013 at 10:13 am
The spiking clustering time has been solved! The change in the code was to mark both clusters and scratch units for removal after a merge, whereas previously only clusters would be marked. This test had a very small number of clusters (less than the number of regions in a single frame) so that any issues with spiking clustering time would appear earlier. The following plot shows quite stable time for clustering and rendering, so we’re ready for some longer-term tests:
Posted: April 24, 2013 at 9:10 am
After a few more short tests, it is clear that the spikes are actually caused by the clustering algorithm. Some additional testing showed that multiple percepts may have the same distance to their nearest cluster. The code to calculate the minimum distance assumed an upper limit of 100, since the initial plan was to use features normalized to 0-1. At some point I changed the distance function for foreground percepts to only use colour features, and since CIELuv distances are perceptually correct, there is no need to normalize them. The result is that distances could exceed 100, thus the calculation of the minimum is incorrect in some cases. While debugging I also found that the number of new units could exceed the number of scratch units. It is expected that this is due to the same scratch unit being merged with multiple clusters. I assumed that this was due to the false minimums, but after changing the upper distance limit to the max possible distance in Luv colourspace, the spikes in clustering time still persist. So the same scratch percept is merged in multiple clusters for some other yet unknown reason. The clustering method is BSAS, but only until we have gotten to the max number of clusters, at which case we’re using our own method similar to the SOM where a scratch unit (input) is merged with the closest cluster (Best Matching Unit), which may explain these problems. As we have a fixed number of clusters, and many inputs, the clustering algorithm used after the fixed number of clusters has been reached is even more crucial. The following plot seems to indicate that indeed the problem is caused when multiple clusters are updated by a single percept. Note the spike in “extraNewUnitsFG”, which is the number of updated clusters (numNewUnitsFG) minus the number of scratch percepts (numScratchFG).
Performance Test A (20,000 frames)
Posted: April 22, 2013 at 3:44 pm
The increasing time to render each frame certainly slows down testing, and would cause future problems processing more than 20,000 frames. What I have done is rewrite the rendering code so that the segmentation uses RGB internally (rather than BGR), and put the draw function inside the percept class so that percepts for segmentation and rendering are stored in the same class. In the previous tests, rendering would take over 8 seconds per frame by 20,000 frames and apparently would continue to increase linearly. Rendering is certainly faster with the new implementation, but unfortunately the test did not get to the goal of 20,000 frames because it used too much memory to continue. Following shows the progress of this latest test… (more…)
Posted: April 16, 2013 at 1:15 pm
Following are some thoughts on Reinforcement Learning (RL) in relation to the current conception of the project. Note this this assumes a basic introductory understanding of RL. (more…)
Clustering Test 7
Posted: April 15, 2013 at 5:47 pm
Following are a few plots of the performance of the last test (20,000 frames, 1500 clusters). The most important observation is that it is not at all the clustering that causes increasing processing times, but the actual rendering itself.
Clustering Test 6 (20,000 frames, max 2000 percepts)
Posted: April 9, 2013 at 9:18 am
This second test went well, although the processing time for each frame shows what looks like it could be an exponential increase. I presume this is due to the comparison of each newly segmented percept to the set of all existing percepts:
Clustering Test 5 (10,000 Frames)
Posted: April 4, 2013 at 11:39 am
The long-term clustering test was successful. This is the first time in this project that I’ve gotten clustering code to deal with the massive amount of data from real-world frames. These 10,000 frames represent about 3 hours ending near dusk. Following is a set of plots that show the behaviour of the system over time, images follow. (more…)
Clustering Test 4
Posted: April 3, 2013 at 10:15 am
For those that are highly detail oriented you may notice that although all the previous clustering tests are supposed to be 300 frames long, but they are actually not. I spent a week finding the problem of why the openframeworks draw() method would not be called on some frames, and then get called again later, all at seemingly unpredictable times. Function by function I gutted my program until all that was left was an empty shell producing debugging output, and yet the problem persisted! At that point I was convinced this must have been a bug in openframeworks and filed a github issue. I thought I should test on another machine, I did so and the problem did not occur. Then I realized that draw() is not called when the window is minimized. My development machine is headless and I use it via an ssh X11 tunnel and vnc (to run with opengl). Turns out it was the screen-saver that was causing the problem: draw() would not be called when the screen-saver was engaged. So I’m back on track. Following is a 1000 frame test, as usual at very high resolution and with the original frames on the left. In this case I limit background percepts to 1000, so once 1000 percepts have been reached, then new segmented regions are merged with the nearest cluster, no matter how far away it is. I also ran a 10,000 frame test overnight; I’ll look at that data and summarize in a future post.