After quite some frustration I have managed to move the segmentation and clustering code into a separate thread. This thread runs as fast as possible, and the main rendering process checks in on the thread between frames to see if new data (percepts) are ready to be rendered. The main thread then makes a local copy of the percepts and then renders them. The main reason for this change is that the rendering rate was always expected to be much faster than the expensive segmentation rate, and keeping it separate keeps segmentation from blocking rendering. Following is a plot from a ~90,000 frame test of the threaded code.
Two main issues are the ever increasing sumActivation and the apparent increase of the time taken to copy percepts from thread into renderer during the night. A segmented frame is likely to have only 100 percepts, so the sumActivation should rarely be over 100, let alone over 2000 as shown here. This problem was due to left over activation in the last frame continuing in the next frame. In the current version of the code, the activation of all percepts is reset to 0 before clustering. It is yet unclear as to the cause of the increasing time to copy foreground percepts during the night, but the longest time is still only 8ms. We’ll have to see if this increase persists in the next long test.