The following video is 100 frames of current work in progress. I was not happy with the motion-blur effect so I went back to the copying of the moving objects into the frame and tweaked some other parameters. I think it’s going well, but I’m not happy with the hard edges of the moving objects, especially at 540p. The motion blur is not great either. Maybe the harder edges are ok for the larger objects, but less so the spots in the trees. Part of the idea of including those moving objects was to anchor the abstraction into the current frame; now that I’m also increasing the effect of the reorganization process in the areas of movement does that have a similar effect?
I did a quick run without inserting the foreground into the state of the network and I’m certainly much happier with the general aesthetic. There is even a sense of movement away from the camera along Broadway that is still readable. The following gallery shows the same sequence as above, except where foreground objects are not included.
I’m just not sure, from an audience perspective, that the movement of the scene would be readable without some subtle reference to foreground objects. Up to this point is to seed the network state with the moving objects to effect the reorganization process. An alternative could be just to present those moving objects visually without them effecting the reorganization process… This would really miss out on some of the complexity of the disintegration of moving objects, for example in the following frames.
The obvious answer is just to take more time and allow the algorithm to obliterate the seed of the moving objects more, but that means taking more time when there is already a time crunch. Another idea is to use the current frame mask (that shows where the moving objects are) to augment the learning process. This may not do what I imagine though… After some reflection, I don’t think it would do anything more concrete. I’m right now doing a run to get the aesthetic I want, training time be damned! (see below.)
I’ve head back from a few of my GPU inquires, and it does not look like upgrading my GPU is possible. I’m now looking at seeing if I can borrow a faster GPU or potentially buying a “gaming” PC from FreeGeek; the problem with the latter is that its certainly a lot more money for a whole new system (even an older one) and the GPUs are much more in the middle of the road and actually have fewer CUDA cores than my current GPU. It’s unclear to me how this will effect performance, but it strikes me that more parallelism is what I need than raw speed to do the training faster.