Very simple question about color perception

Thanks so much for the reference.
“Which input it is carries the information of which color it is”, this is the key point, who knows which input carries what color information. Assume that you are one neuron at a high level layer, and there are two inputs connected to you, the first one is a red detector(meaning where red color comes it will fire), the second one is a green detector, now the firstone fires, how do you know it means that it see a red color, because you don’t know the first one is a red color detector, what all you know is just that your have two inputs and know one of them fires.
I think this is a more common problem, as we know, SDRs representing different concepts are also very different, so the high level layer can very easily distinguish them, however, when the input is not complex it will not work, and is very simple just like the two base colors (red and green) in the retina, LGN and even V1 area, only two kinds of cells will fire for them respectively. e.g. in V1, there is one kind of neurons which will fires when the red color fills its receptive field and another kind of neurons which will fires when the green color fills its receptive field. So the high level visual cortex area will see some neurons fire when we see big red(we can simplify the environment where there is only one big screen in front of our eyes, and the screen just show red color or green color), then the question will become how we distinguish red color and green color, except that the brain has very different encoding ways for different colors. For example, one way to distinguish differenct colors is to encode wave length of light into fire rate, however, it seems that our brain is not able to compare two color. So the reference has the answer about this?
to @rhyolight again, how HTM network represents different colors in sensory input? just RGB values? a different ways from human beings.

There are still people working on this in nupic.vision, but I don’t know anything about it. Older versions of HTM used to do video processing, but they were abandoned because of new more biologically-constrained theory implemented in the current theory.

I don’t remember the book specifically addressing your underlying question, but it’s well worth reading (and will clarify some points on how color processing works in primate brains). I think the answer to your deeper question is that “red” is the label we learn to assign to particular patterns of input. There’s no intrinsic red-ness to red-on/green-off cells, you can’t tell them apart from green-on/red-off cells until you start showing different colors and examining the resulting activity. Over months of early experience you learn that these patterns are present when the Big People say “red”, and also when the round crunchy sweet food is in front of you, and gradually you weave your own personal fabric of associations around that specific pattern of input.

1 Like

Great!

“red” is the label we learn to assign to particular patterns of input

Yes, that’s right! The key is particular patterns.
What are the differences between red pattern and green pattern?
In physics, the only different is that they have different wave-lengths.

Different photoreceptors have different pigments and thus respond to different wavelength ranges. Most of us have three varieties of cones and so three different color signals that project to the next stage. So far as anyone can tell, the arrangement of projections is random and so it’s a learning problem (the first learning problem, really) to turn the input into behaviorally relevant information.

1 Like

Sorry, I didn’t understand this sentence? Is that mean in biology, different colors will have different random representing values?

There is a theory about this issue called labeled line theory.

labeled lines theory: Different qualities of a percept such as hot versus cold, sweet versus salty are encoded by separate group of cells. The information within a series of cells transmitting information to the brain remains within separate channels or lines. @karchie

The cells in the retina (which is itself part of the brain) are arranged in layers, but very differently to the neocortex. On the outside, photoreceptors change their activity in response to light, and a complex circuit of cells in the middle layers use integration and inhibition to feed signals to the output retinal ganglion cells on the inside of the retina. Some of these cells encode the color of the image, but only a small number of these encode the intensity of red, green and blue (as happens in a digital camera). Most of them rather encode the opposition of two colours (red vs green, blue vs yellow). As @dfx mentions, we have cells which each respond to one direction of these axes (as well as the ON-off, OFF-on axis rod-fed cells), so there are four types: R-g, G-r, B-y, Y-b.

Perhaps the best coverage of recent discoveries of retinal structure and function and colour vision by Christof Koch.

The individual projections of RGCs is indeed random, but starting in early development, “training programs” of spreading activation are run in the fetal retina which send patterned signals up through the visual pathways, and so help organise the specialisation of groups of cells in V1 upwards, and so the organisation in cortex looks very organised (mosaic-like) and not random (see around 1hr in to the second talk).

4 Likes

Wow, thanks a lot, a clear description about the basic structure. But I still have some questions?

How does it encode a color? Does this mechanism put very special information into the color encoding? Is the special information same for the same colors and different for different colors?

@fergalbyrne good point on the spatial arrangement: early learning is a self-organizing process that, starting with random arrangement, produces still-random but statistically consistent spatial structure in the projections (e.g., center-surround in retina/thalamus, blobs in V1). Ken Miller did really pretty theory on this 20 years ago for ocular dominance and orientation columns.

1 Like

Well, the notion of “value” is a little suspect here. There are photoreceptors, each kind has its own color sensitivity (there’s the start of your labeled lines), and then where those lines go is random but shaped by a learning process (as @fergalbyrne mentioned). Over time the brain learns to make sense of these inputs.

1 Like

@dfx In general, a neuron “encodes something” when its firing correlates with the presence of that thing. Depending on what you’re encoding, how you’re measuring, and over what timescale, this might involve its rate of firing, its probability of firing, or simply whether its firing at all. Different sensory neurons have evolved to change these in different ways - so for example rod photoreceptors lower their rate of firing as the number of photons increases, while some force-sensitive skin receptors raise theirs as the force increases.

To simplify in HTM we just say that a neuron is a 1 (firing) or a 0 (not firing) in a given (artificial) timestep, indicating the binary presence or absence of a feature. This models the kind of non-cortical preprocessing which prepares sensory signals for the neocortex.

In the case of “colour-encoding” neurons, they usually produce spikes for the onset of a particular “colour”, and spike faster in proportion to its rate of intensity increase (or vice versa for offset-sensitive neurons). By “colour” I mean the broad spectrum response of the corresponding cone receptor.

1 Like

It’s a little complicated. Neural coding tends to be more about contrasts and edges (center-surround and on/off response types, to start) and temporal changes than a direct representation of the external world. So a given cell increasing its firing rate might mean “this part of the visual field is now more red and/or less green than it was a little while ago.” The same cell will decrease its rate if that same place becomes more green/less red, or if nothing changes.

@karchie is correct. For reasons of efficiency, most sensory information is in the form of rates of change (spatial/temporal), or differences. These signals are integrated higher up to produce more persistent representations. See https://www.youtube.com/watch?v=QxJ-RTbpNXw for a camera based on this idea.

I am moving this topic into the new Neuroscience subcategory, since it doesn’t really have anything to do with SP, but is more about the neuroscience.

1 Like

Yes, firing rate could represents stronger or weaker, lighter or darker. But when the same firing rate occurs for different kinds of sensory input, how do you know which input represents what type of receptor.
For example, In HTM, when there are two neurons are both 1, how do you which neuron represents a red color or a skin pressing or a sound input?
In another words, if no labeled information, how dose a robot brain to know whether a sensory input is from its ear, its eyes, or just the lower level neurons?

@dfx the simple answer is that you (ie your neurons) don’t know what the “meaning” of an input is. If you’re getting input right now from a particular axon, you just know that “whatever that axon represents” is happening right now. Each neuron has access to a relatively small number of possible inputs, and all it does is learn to decide to fire or not, based on how many and what combination of inputs it is getting right now. If it fires, its output represents something to other neurons which again is just “whatever that axon represents”.

So, at the level of individual neurons, there is no sense in which any individual neuron is aware that “this signal means orange” or “this signal means green”. Yet somehow we (as minds) do have that awareness. How this works in detail is called The Hard Problem in neuroscience (and psychology and philosophy).

One very poor answer is that our reptilian brains have evolved deep structures for representing or labelling things in the physical world, such as “mother”, “tribe member”, “friend”, “threat”, along with solid objects, liquids, animals, etc. (see Japanese Counter Word for this surfacing in the Japanese language). The evidence for this is found in birds and reptiles, who instinctively “know” what such things are just by seeing or hearing them. In the mind of a reptile, the “meaning” of a percept is simply the presence of activation in one or more of these innate “label structures”, and the “label structures” only have meaning in that they have evolved to trigger some kind of relevant behaviour. For an extreme version of this, see Julodimorpha bakewelli which caused the outlawing of brown beer bottles in Australia).

As we experience the world as babies, these structures are (hand-wave) activated by the old brain as it receives the appropriate sensory stimuli, and our cortex (probably in or near the inferior temporal lobe, where object recognition happens) learns to identify these associations too. The reptilian mechanisms develop more quickly than the cortical ones, but feedback loops involving ever-improving cortical functions allow the reptilian components to do their job even better, and before long (maybe well before 12 months of age) the two parts of the brain are so interdependent that perception itself is a distributed phenomenon and becomes this conscious thing the philosophers call “qualia” and others call “feeling”. Many people believe that the qualia is not directly perceived by the neocortex, but is inferred from previous associations between similar perceptions and the identifying signals coming from non-cortical centres.

4 Likes

On the “meaning” of individual neurons vs large-scale patterns, try Hofstadter’s Careenium

1 Like

Dear @fergalbyrne, thanks so much for your clear and detailed answer, now I’m no longer confused any more. :slight_smile: So Great!