# An HTM compatible theory of attention

**URL:** <https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640>\
**Category:** Numenta Theory\
**Tags:** attention\
**Created:** [April 7, 2022, 10:39pm UTC](https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640 "2022-04-07T22:39:27Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Panaman](https://avatars.discourse-cdn.com/v4/letter/p/3da27b/32.png) [@Panaman](https://discourse.numenta.org/u/Panaman)\
**Post date:** [April 7, 2022, 10:39pm UTC](https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640/1 "2022-04-07T22:39:27Z")

</div>

I’ve been working hard to grok HTM theory. If I’m understanding correctly, the mechanism behind “attention” would best be explained by SDRs inhibiting more parts of the brain as they are recruited to model something intensely (the net effect pushing other conscious thoughts out of awareness). Does this sound right (or at least indicate that I’m starting to think out HTM theory correctly)?

---

<div class="post-metadata">

**Author:** ![CollinsEM](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/collinsem/32/160_2.png) [@CollinsEM](https://discourse.numenta.org/u/CollinsEM)\
**Post date:** [April 7, 2022, 11:20pm UTC](https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640/2 "2022-04-07T23:20:58Z")

</div>

I tend to think of attention as a behavior designed to reduce uncertainty and/or surprise. That is to say that our attention is directed towards thoughts and/or obtaining sensory inputs that allow us to gather additional data to resolve existing uncertainties or learn new novel sequences.

HTM excels at generating predictive states that are consistent with previously observed patterns. When a novel pattern is observed and columns burst in response to being activated without having been in a predictive state, then the thalamus is gated to flood the cortex with additional signals and information to assist with learning a new sequence.

While these ‘surprise’ events will certainly grab the attention, and focus the agent on learning the new sequence, novelty is not the same as uncertainty. Uncertainty can exist if the system is making multiple predictions that are all plausible given the current context. In that case, attention could be used to generate behaviors that would allow the agent to acquire additional information (input from senses or recall of stored memories) that might reduce the number of simultaneous predictions.

---

<div class="post-metadata">

**Author:** ![Panaman](https://avatars.discourse-cdn.com/v4/letter/p/3da27b/32.png) [@Panaman](https://discourse.numenta.org/u/Panaman)\
**Post date:** [April 7, 2022, 11:59pm UTC](https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640/3 "2022-04-07T23:59:43Z")

</div>

While performing this behavior, many columns are recruited to model observed patterns right? As more parts of the brain are employed, doesn’t this have a inhibitory effect… essentially filtering other unrelated patterns from participating in conscious thought?

I’m imagining the enforced sparcity leaving no room for conscious thought other than what you are actively modeling (Attention).

---

<div class="post-metadata">

**Author:** ![EEProf](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/eeprof/32/5653_2.png) [@EEProf](https://discourse.numenta.org/u/EEProf)\
**Post date:** [April 8, 2022, 2:23pm UTC](https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640/4 "2022-04-08T14:23:40Z")

</div>

> [@CollinsEM](#):
>
> I tend to think of attention as a behavior designed to reduce uncertainty and/or surprise.

You might find Graziano’s work on _Attention Schema Theory_ (AST) provocative.

[https://grazianolab.princeton.edu](https://grazianolab.princeton.edu)

---

<div class="post-metadata">

**Author:** ![DanML](https://avatars.discourse-cdn.com/v4/letter/d/edb3f5/32.png) [@DanML](https://discourse.numenta.org/u/DanML)\
**Post date:** [April 8, 2022, 3:01pm UTC](https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640/5 "2022-04-08T15:01:13Z")

</div>

> [@EEProf](#):
>
> Graziano’s work on _Attention Schema Theory_

Personally, I found AST quite ground breaking - and a good step away from the Theory of Consciousness to real application. AST predicts things you can test - and he has.

(As an aside, I also enjoyed his starting point from the applied world of proprioception, which itself highlights many non-conscious aspects of how bodies really work, and predicting what happens when it fails).

---

<div class="post-metadata">

**Author:** ![BrainVx](https://avatars.discourse-cdn.com/v4/letter/b/ed8c4c/32.png) [@BrainVx](https://discourse.numenta.org/u/BrainVx)\
**Post date:** [April 8, 2022, 7:05pm UTC](https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640/6 "2022-04-08T19:05:03Z")

</div>

This is along the lines of how I’m thinking the process works :

Goal investigation → Unrecognised pattern (sub cortical instigator)  
Unrecognised pattern → Surprise (SDR mini column inhibition / winner takes all)  
Surprise → Attention (thalamic focus / wider column inhibition)  
Attention → Working memory (PFC mini column identification)  
Working memory → Thought (PFC/other/no idea)  
Thought → Goal investigation

All surprise ends up in working memory, just depends on how quickly it fades out… we can’t avoid or block this process.

Surprise is only created in the SDR process once (within 200mS theta wave cycle - borrowing from Max’s hypothesis - then it is no longer a surprise and is then a known pattern) so this surprise is then held in working memory… to fade out or not…

---

<div class="post-metadata">

**Author:** ![dmac](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/dmac/32/5842_2.png) [@dmac](https://discourse.numenta.org/u/dmac)\
**Post date:** [April 9, 2022, 11:57am UTC](https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640/7 "2022-04-09T11:57:21Z")

</div>

> [@L5tt cells are attentional (paper summaries)](https://discourse.numenta.org/t/l5tt-cells-are-attentional-paper-summaries/9435):
>
> [1] [High-frequency burst spiking in layer 5 thick-tufted pyramids of rat primary somatosensory cortex encodes exploratory touch](https://www.nature.com/articles/s42003-021-02241-8) [2] [Active dendritic currents gate descending cortical outputs in perception](https://www.nature.com/articles/s41593-020-0677-8) Main Point L5tt cells cause the brain to notice a stimulus. To do so, they burst, triggering detection via thalamus, SC, and striatum. [[1](https://www.nature.com/articles/s42003-021-02241-8)] L5tt cells can burst upon touch. This study just measured spiking while the rat whisked and touched an object. Many L5tt cells were capable of respon…

---

<div class="post-metadata">

**Author:** ![Prakash](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/prakash/32/6117_2.png) [@Prakash](https://discourse.numenta.org/u/Prakash)\
**Post date:** [September 4, 2022, 12:43pm UTC](https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640/8 "2022-09-04T12:43:01Z")

</div>

Attention is insufficiency of adequate inputs to the brain. Which leads to incomplete or partial thoughts formation that will eventually fades away. If you want to increase attention try to increase the inputs or stop the chaos and gets fresh input…eg. When you are out of focus in your classroom, stop your breath untill you feel the pain in the chest this cause your existing thoughts to disappear, now you have cleared your thoughts, watch the class immediately don’t take any other inputs. I don’t know there may be another way to clean our thoughts but physical pain has more potential clean our thoughts than any other things.

---

<div class="post-metadata">

**Author:** ![JarvisGoBrr](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/jarvisgobrr/32/6147_2.png) [@JarvisGoBrr](https://discourse.numenta.org/u/JarvisGoBrr)\
**Post date:** [September 4, 2022, 4:39pm UTC](https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640/9 "2022-09-04T16:39:46Z")

</div>

I think you are lumping top-down and bottom-up inputs together but when we talk about attention, we usually mean the phenomenon of bottom up inputs content being gated/modulated by top-down expectation.

---

<div class="post-metadata">

**Author:** ![Casey](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/casey/32/2874_2.png) [@Casey](https://discourse.numenta.org/u/Casey)\
**Post date:** [September 4, 2022, 8:48pm UTC](https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640/10 "2022-09-04T20:48:25Z")

</div>

> [@Prakash](#):
>
> Attention is insufficiency of adequate inputs to the brain.

Do you mean like a filter? I think another part of it is surprise. If you read about the burst / tonic firing modes in the thalamus, one way to interpret that is surprise.

> **off topic**
>
> > [@Prakash](#):
> >
> > I don’t know there may be another way to clean our thoughts but physical pain has more potential clean our thoughts than any other things.
> 
> Pain causes adrenaline which causes focus. The problem with that is it’s a cheap way to pay attention, like music or inspirational thoughts, rather than wiring your brain to just do stuff which part of it doesn’t want to do, without a push from extraneous stuff. You shouldn’t need to do well in classes to feel good about yourself. I hate how society acts like intellectual ability makes people better or worse than others. I’m not immune to any of this, it’s just stuff I’ve thought about a lot.

---

<div class="post-metadata">

**Author:** ![Prakash](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/prakash/32/6117_2.png) [@Prakash](https://discourse.numenta.org/u/Prakash)\
**Post date:** [September 5, 2022, 12:39pm UTC](https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640/11 "2022-09-05T12:39:11Z")

</div>

Actually i believe that we can form thoughts only when there is enough inputs to actually create a meaningfull thoughts. If inputs are low or higher we can’t form a thought… so a adequate amount is necessary

---

<div class="post-metadata">

**Author:** ![Prakash](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/prakash/32/6117_2.png) [@Prakash](https://discourse.numenta.org/u/Prakash)\
**Post date:** [September 5, 2022, 12:42pm UTC](https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640/12 "2022-09-05T12:42:55Z")

</div>

> [@Casey](#):
>
> You shouldn’t need to do well in classes to feel good about yourself

You are right, but im living in a place where students compete each other with their marks… Its this place that demands me to stop my breath to get attention… I don’t want anyone to follow this… Sry thou…

---

<div class="post-metadata">

**Author:** ![Prakash](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/prakash/32/6117_2.png) [@Prakash](https://discourse.numenta.org/u/Prakash)\
**Post date:** [September 5, 2022, 12:45pm UTC](https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640/13 "2022-09-05T12:45:38Z")

</div>

Agreed… is top-down and bottom-up work with each other simultaneously to give attention?

---

<div class="post-metadata">

**Author:** ![Casey](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/casey/32/2874_2.png) [@Casey](https://discourse.numenta.org/u/Casey)\
**Post date:** [September 5, 2022, 10:02pm UTC](https://discourse.numenta.org/t/an-htm-compatible-theory-of-attention/9640/14 "2022-09-05T22:02:13Z")

</div>

No need to apologize, holding your breath isn’t a big deal.