I really enjoyed these postings, and I find gmirey’s and Bitking’s thoughts and shares to be highly informative. Wow Bitking! You do need a bigger hat. Maybe one made in spandex.
The effort seems to have stalled a little, maybe I can help out. There is a general misunderstanding of the brain’s process framework and I see it reflected in some of your struggles to duplicate V1. In my own research, I found this misperception of the brain’s algorithm separates mind science from computer science.
Any chance at a realistic V1 implementation is dependent on first understanding that the brain’s process begins outside of the visual subsystem. I will try to summarize the correct model the best I can. As Bitking points out in one of his other postings, there are many thalamic connections and it makes for a complicated diagram. So, I will try to give a basic explanation without getting lost in detail.
All lifeforms on this planet perceive their environment using sensory systems of some type. This perception is accomplished through a rhythmic measurement of space. A biological process maintains state by performing and applying these measurements based on linear time. Since there exist no world clock (Einstein), the rhythm is set by each individual lifeform and measured relative to that lifeform (observer). Life is by its own nature impaled on the arrow of time.
In the human brain, time relativity is calibrated through the hypothalamus which uses the optic charisma like a pulsar to establish a circadian rhythm that regulates the cycling of the biological process. By performing this function, the hypothalamus serializes sensory measurements and synchronizes the execution of both hemispheres.
Each hemisphere will be individually controlled by the thalamus. The thalamus cycles the linear firing sequence using a sophisticated set of nuclei. More importantly, the thalamus maintains a state of neural synchronization between itself and all the lobes, cortexes, and layers through these nuclei. The reason why all thalamic connections are reciprocal is because two-way communication is mandatory to maintain a synchronous state. By doing this, the brain creates a hierarchically structured quantum state shared in by all of its biological components. For example, the lateral geniculate nucleus shares the same state as primary visual cortex (V1) at one level and the pulvinar shares state with the secondary visual cortex (M5) at another.
Neural synchronization allows the brain to exist in a single entangled state as sensory measurements are cycled, processed, and applied. Since the thalamus is cycled, it has awareness of both time and state. It uses this knowledge to translate all sensory data into space-time (Minkowski). Comparing two slices of space-time at a fixed interval allows the thalamus to produce a measurement of motion.
This is NOT the motion definition derived from the superior colliculus (rods). Rods are an encoding characteristic specific to visual sensory data. Thalamic motion is actually a set of instructions on how to change state and it doesn’t matter what sensory or object format is being measured. A downstream component simply applies these instructions to keep the states synchronized in real-time.
When the Thalamus conducts the measurement, it will categorize any changes (movements) in state as either predictable or unpredictable. Unpredictable states are accumulated and their change instructions are encoded for synchronization. Predictable states are discarded because their results are already known by the downstream component due to its shared state with thalamus. The knowledge is passed between components by the sheer existence of the neural pulse. Nature is amazing in its efficiency.
The problem when discussing the Bayesian nature of the brain is that people’s perceptions of prediction is based only on top-level cognitive objects. What is the chance a rabbit will jump left, right, or straight? While this is an activity of prediction, it misses the importance that that prediction plays in regulating and filtering sensory data within the thalamus. The rabbit is just a consolidated object that is composed of tens/hundreds of thousands of smaller predictions ranging from the chance that the rabbit will alter color to the chance that there will change in a visual cone. The brain’s prediction is granular and has application on all levels.
As motion is bound up through components like the Occipital and Parietal Lobe, it gains greater and greater hierarchical abstraction (objectification, states-within-states). For example, sensory patterns become fingers becomes hands becomes arms and so on. Top-down knowledge is then passed back down the pulvinar and used to group smaller objects for measurement; thereby, refining prediction capability. So, the sensory motion of a finger is predictable if the hand is performing a certain activity that may have been predicted by a particular arm movement.
By doing hierarchical predication in a synchronized state, the brain increases data transfer efficiency by 100,000% or more as top-down information flows down to the geniculate/V1 level. This has significant implications for humans because the efficiency boost allows us to shift activity and subsequent energy use to higher brain functions. So, the less we have to do in the bottom results in the more we can do on top. A wonderful subject for another time.
Thalamic motion is also the principle reason that neural communication has been so difficult to decipher. As an outside observer who does not share synchronous state, there is no foundation to understand the significance of the motion encoding. For example, one neuron may contain instructions for an entire image, a second may only carry instructions to update a few cones/rods, and a third carries nothing. From an outsider’s perspective, it appears chaotic.
I have also gotten lost in the visual sensory data format as you have in your effort. Dealing with visual sensory data is no easy task and is one of the most challenging formats within the brain to duplicate. The good news is that visual data drives the process, but does not define the process. So, you can substitute simpler data forms to accelerate your construction of V1 and diversify its application. You will find that the brain’s motion algorithm works just as efficiently on computer data frames as it does on cones and rods.
I have covered a lot of ground and have glazed over some very important topics. I am willing to explore some details either publically or in private messages towards subject matter pertinent to your V1 programming goals. With all the interrelationships of V1, it is not an easy task. So, I truly appreciate your efforts and will help where I can.
Interesting Side-Note:
As the brain binds motion up the hierarchy, the top state is maintained in the prefrontal cortex and can be best described as our state of conscious reality. Among other connections, the prefrontal cortex shares state with the mediodoral nuclei (thalamus) and the hypothalamus. The synchronized brain will bind two forms of motion through these connections, sensory perception via the mediodorsal nuclei and thought via the hypothalamus. So, our conscious reality has two principle binding points within a synchronized process that performs linear measurements of space. I guess it wasn’t that “hard” a problem after all.