Feature Extraction (between frames for 5 subsequent labelled patches)

Posted: April 13, 2012 at 3:48 pm

Since I’ve been flying by the seat of my pants, I thought I should collect some real data on the feature extraction process before continuing. The only measure I could think of for seeing the distribution of features for a particular patch of pixels was to segment and hand label patches. I’ve done so for a small set of 5 subsequent frames, and following are the results.

CIELuv Histogram


Above shows the variance of the correlation of LUV hists between frames for each labelled (reference) patch. The colour shows the mean of the correlation. Clearly there is a lot of variation, and it differs a lot between patches. The mean of the correlation is extremely small (0.07) across frames, channels, and patches. The mean of other channels, excluding the channel with the least variance (L) is 0.06, 0.076  for U and 0.045 for V. Seems histogram is not a great indicator that the patch is the same between frames.

Position and Aspect

For position and aspect these features have much less variance (the largest distance in position between two labelled patches is 3 pixels. That same patch also shows a spike of variance in the aspect ratio, where the largest difference is .15.

This means that position and aspect are reasonable features for clustering, but the former obviously only works on patches that are not moving. In order to track moving patches, it seems I’m going to have a problem. I should experiment with SURF, SIFT, and template matching but those are supposed to be quite slow, how slow I won’t know until trying it.