Since it’s time to explore some clustering methods for these perceptual patches, I first calculated a RGB histogram for each patch, making use of the mask. In attempting to make sense of the structure of the openCV hist (1 channel, but 3 dimensions) I realized that perhaps the hist is not appropriate for this project. The central question is: what is important about colour for the purpose of associating perceptual units? Perhaps the histogram should be abstracted into a set of concepts (for lack of a better term) representing major colour clusters.
The proposal is to use the CIE Luv colourspace (which separates luminosity and colour, and is perceptually oriented, using blue-yellow and red-green channels and where distance in the colour-space is proportional to perceived difference). Rather than using a histogram of the colour space directly, the idea is to break it up into notions of individual colours: darkness-lightness, redness-greenness, blueness-yellowness. The question becomes how to break up the colour-space into these abstractions? One possibility is to use hard thresholds to break the space up into ranges corresponding to each colour notion. This opens up a number of other questions, how many colour notions should there be? What about orange, cyan and magenta? Any division of a continuous space into notions of individual colours seems arbitrary. This issues reaches to a core aspect of this project, the link between symbolic atomic pieces and a continuous sensory space. How does the range between red and orange turn into two ideas? Perhaps they are clusters in themselves, a centre surrounded by outliers that belong to varying degrees.
Lets take a concrete example, the u channel, from blue to yellow. We could break it into two regions, the blue part and the yellow part. The problem is that at the border red and yellow are identical. We could pad the two notions, The first third being blue, the second third being left undetermined, and the final third being yellow. This would likely provide a reasonable abstraction of colour, but what if a patch’s hist lies in the undetermined area? In developmental psychology it is the role of the caregiver that allows the partitioning of a continuous space into fuzzy symbolic notions, through the act of naming objects and attributes. Of course this requires a care-giver, and is therefore incompatible with this project. Another notion is a probabilistic one, where the colour value defines the probability that the patch matches a particular colour notion. The mapping of colour values to probability is still problematic. Does the chance of blueness drop to 0 where the change of yellow becomes 1? Or does the chance of blueness drop off to 0 at a threshold between blue and yellow?
According to the CIEluv diagram below there do appear borders between blue/yellow and red/green. Its unclear if these borders are an artifact of the plot, or have meaning in terms of the colour model. I have emailed the neurophilosphers (whose supervisor specializes in colour) on these issues and will post any conclusions or further insight.
Here are some refs I’m going to look through for ways of tackling this: