After almost two months, and seeking the help of quite a number of people who know a lot more about C++ than me, the problem I’ve been facing with the memory leak has been (at least partially) resolved, thanks to this post. It turns out that this whole time the way I was measuring ram usage was incorrect (maxrss via rusage vs rss via /proc). Thus all memory plots for the whole life of this project were inaccurate. This does not effect the fact that the system did eventually crash after using too much memory, but at least it means I should be able to actually find the problem. The likely cause of the real leak was the blind shallow copying of percepUnits from the thread into the renderer. Now we’ll render to a small (<10) set of FBOs in the thread, and then only share the FBOs, and not the whole list of instances. Unit tests have shown that at least foreground segmentation, image grabbing from the camera, and the percepUnit constructor are all leak free
During this debugging process I did do some refactoring concerning how Mats get passed between some functions, and combining backgroundMasks() and segmentBackground() into one function. In testing for leaks, I have realized there is an issue with this slightly modified background segmenter: while memory appears somewhat stable, the amount of time it takes to process a frame is up to 80s!! I have no idea where this problem was introduced, but seems to be related to very dark (night) input frames.
I have learned a lot since I made this segmenter, and I have to wonder how much I should bother debugging this known to be slow method rather than rewriting the whole thing from scratch. I tried segmenting with a canny filter, and with mean-shift filtering, but I never tried using them together! While mean-shift is a little slower, canny is really quite fast, and with a little morphology to smooth structure out initially, the processing time could be even faster than the current 2s per frame. Additionally, the machine has been upgraded since those early segmentation experiments. I’m currently running the code in cachegrind (5 frames) to get a sense of what function is using all this CPU time. If its a simple bug, I can just fix that and move ahead confirming the lack of a memory leak. If it turns out to be something more serious, I think I should dump this floodFill version and do something like this:
- equalization of image? (slow)
- morphology (fast on gpu)
- mean-shift filtering (fast on gpu?)
- canny (fast)
- findContours (fast)