# Jeff Hawkins on Reference Frame Rotations - April 19, 2021

**URL:** <https://discourse.numenta.org/t/jeff-hawkins-on-reference-frame-rotations-april-19-2021/8487>\
**Category:** Current Research\
**Created:** [April 23, 2021, 8:27pm UTC](https://discourse.numenta.org/t/jeff-hawkins-on-reference-frame-rotations-april-19-2021/8487 "2021-04-23T20:27:37Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![clai](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/clai/32/5365_2.png) [@clai](https://discourse.numenta.org/u/clai)\
**Post date:** [April 23, 2021, 8:27pm UTC](https://discourse.numenta.org/t/jeff-hawkins-on-reference-frame-rotations-april-19-2021/8487/1 "2021-04-23T20:27:37Z")

</div>

[![](https://img.youtube.com/vi/iyX4AVuuz6c/maxresdefault.jpg "Jeff Hawkins on Reference Frame Rotations - April 19, 2021") ](https://www.youtube.com/watch?v=iyX4AVuuz6c)

Previously, Jeff Hawkins has discussed the possibility that reference frame rotations might occur locally in a cortical column. In this research meeting, he proposes an alternate possibility that movement and sensed features are translated in the thalamus. Using vision as an example, Jeff gives an overview of the thalamus and discusses the role and mechanism that thalamo cortical cells might play.

[Jeff Hawkins on Reference Frame Transformation in the Thalamus](https://youtu.be/eQv-MjnTodM)

---

<div class="post-metadata">

**Author:** ![petre.a](https://avatars.discourse-cdn.com/v4/letter/p/da6949/32.png) [@petre.a](https://discourse.numenta.org/u/petre.a)\
**Post date:** [May 6, 2021, 3:48pm UTC](https://discourse.numenta.org/t/jeff-hawkins-on-reference-frame-rotations-april-19-2021/8487/2 "2021-05-06T15:48:09Z")

</div>

I am not a neuroscientist and, naturally, I have a stupid idea….  
What if L5 “motor” output doesn’t contain motor vectors but some sort of data resembling sensory (some sort of “Augmented Reality Glasses” trick)?  
Maybe neocortex is “cooking” some “fake” sensory data with the clear intent to trick the old brain to do something for him?

---

<div class="post-metadata">

**Author:** ![maxerbubba](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/maxerbubba/32/4804_2.png) [@maxerbubba](https://discourse.numenta.org/u/maxerbubba)\
**Post date:** [May 7, 2021, 3:49am UTC](https://discourse.numenta.org/t/jeff-hawkins-on-reference-frame-rotations-april-19-2021/8487/3 "2021-05-07T03:49:43Z")

</div>

I had a realization that “thalamus is a router” one day but I wasn’t sure what to do about it. I am very excited to see where this goes!

I was picturing each transformation “level” being its own dimension, where each transformation can be stacked. Like a gimbal, each with separate independent axis that lead to a wide range of motion. (I’ve been calling it a “Neural gimbal” 😀)  
For example: when reading your phone screen:

- retina
- eye left/right
- eye up/down
- neck left/right
- neck up/down
- torso
- shoulder
- arm
- hand
- finger  
Each level might be a “hop” through the thalamus, to translate retina pixel to phone screen pixel.

Keeping track of this in real-time requires some sort of feedback to tell if the state estimation is “in focus”.

Now that I mention it, it probably wouldn’t be too hard to train a deep learning model in a simulator to detect joint movements with optical flow, and estimate independent transforms for each joint. Might be fun. I’m Not sure how the system would deal with 2 possible explanations for a given inverse kinematic a solution, though

---

<div class="post-metadata">

**Author:** ![Bitking](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/bitking/32/3099_2.png) [@Bitking](https://discourse.numenta.org/u/Bitking)\
**Post date:** [May 7, 2021, 5:07am UTC](https://discourse.numenta.org/t/jeff-hawkins-on-reference-frame-rotations-april-19-2021/8487/4 "2021-05-07T05:07:04Z")

</div>

Perhaps you should get familiar with the posture control system?  
This is the [inverse kinematics](https://en.wikipedia.org/wiki/Inverse_kinematics) system that maps from your vestibular system to the support points.

 ![800px-Modele_cinematique_corps_humain.svg](https://canada1.discourse-cdn.com/flex030/uploads/numenta/original/2X/6/673be564b9cba8d614e5ab9dc35d2958bafadd08.png)

> **[Cortical control of posture](https://en.wikipedia.org/wiki/Cortical_control_of_posture)**
>
> Postural control refers to the maintenance of body posture in space. Central nervous system interprets the sensory inputs to produce motor output to maintain upright posture.
> Postural control is defined as achievement, maintenance or regulation of balance during any static posture or dynamic activity for the regulation of stability and orientation. The interaction of the individual with the task and the environment develops postural control. Stability refers to maintenance of the centre of mass...

 ![Posture control system](https://canada1.discourse-cdn.com/flex030/uploads/numenta/original/2X/6/6634e70a9fd9ef17ca4bf1ce11971ad1071921a4.png)

 ![slide_41](https://canada1.discourse-cdn.com/flex030/uploads/numenta/original/2X/d/d5071bc6cda3cc760aebc6fb8269f82597cdb5b0.jpeg)  
 ![the-vestibular-system-16-638](https://canada1.discourse-cdn.com/flex030/uploads/numenta/original/2X/1/1bd2340cef753f35d8984578d4157acee2cfd186.jpeg)

---

<div class="post-metadata">

**Author:** ![maxerbubba](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/maxerbubba/32/4804_2.png) [@maxerbubba](https://discourse.numenta.org/u/maxerbubba)\
**Post date:** [May 7, 2021, 5:41am UTC](https://discourse.numenta.org/t/jeff-hawkins-on-reference-frame-rotations-april-19-2021/8487/5 "2021-05-07T05:41:07Z")

</div>

> [@clai](#):
>
> Jeff Hawkins on Reference Frame Transformation in the Thalamus

Oops I was commenting on the bottom link to video from January. Still exciting!

But here’s a fun idea for Jeff: What if the thalamus transformation is like a muscle? That is, commands sent to thalamus (via the branching axon from Layer6) is a motor command… and the thalamus is the “muscle” that “moves the reference frame”. So to see the results of the action in the real world, the brain needs to use “sensors” to measure the effect of that motor command, and adjust accordingly.

So then a branching axon sends (a) a motor command to “flex” the thalamic “muscle” and (b) an efference copy so that higher levels can decode the lower-level action that was taken

Maybe I misunderstand, but might be interesting.

 ![image](https://canada1.discourse-cdn.com/flex030/uploads/numenta/original/2X/6/6f3466dcdbb62ffa4e8438f7e017ae93f71a795f.jpeg)

---

<div class="post-metadata">

**Author:** ![CollinsEM](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/collinsem/32/160_2.png) [@CollinsEM](https://discourse.numenta.org/u/CollinsEM)\
**Post date:** [May 7, 2021, 2:06pm UTC](https://discourse.numenta.org/t/jeff-hawkins-on-reference-frame-rotations-april-19-2021/8487/6 "2021-05-07T14:06:11Z")

</div>

I’ve had a thought in recent days that the efference copy of the motor output might be related to voting. Essentially each column is attempting to nudge it’s neighbors towards a common (or at least complimentary) representation. This would be sort of like multiple clocks on the wall gradually synchronizing their oscillations with one another. The resulting firing patterns would correspond to a lowest energy configuration for the local network. The purpose of this synchronisation is so that the columns in the local cortical region can pool their influence on other regions through their combined motor outputs.

I’ve had another thought this morning related to grid cells, reference frames, and rotations of k-vectors in high dimensional spaces using rotors (similar to quaternions). I still need to work out the math, but if my intuition is correct, it should be possible to generate high dimensional grid cell module representations (states) and their allowed state transitions in such a way as to directly encode the topology of a low dimensional continuous manifold using nothing but binary vectors with constant sparsity.