# Untangling Sequences: Behavior vs. External Causes
**URL:** https://discourse.numenta.org/t/untangling-sequences-behavior-vs-external-causes/2883
**Category:** Related Papers
**Tags:** behavior
**Created:** [September 26, 2017, 3:45pm UTC](https://discourse.numenta.org/t/untangling-sequences-behavior-vs-external-causes/2883 "2017-09-26T15:45:55Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@rhyolight](https://discourse.numenta.org/u/rhyolight)
#### Post date: [September 26, 2017, 3:45pm UTC](https://discourse.numenta.org/t/untangling-sequences-behavior-vs-external-causes/2883/1 "2017-09-26T15:45:55Z")
For better understanding how the Layers and Columns theory builds upon Temporal Memory theory:
> **[Untangling Sequences: Behavior vs. External Causes](https://www.biorxiv.org/content/early/2017/09/19/190678)**
>
> There are two fundamental reasons why sensory inputs to the brain change over time. Sensory inputs can change due to external factors or they can change due to our own behavior. Interpreting behavior-generated changes requires knowledge of how the...
by @subutai and @jhawkins
---
### Author:  [@gidmeister](https://discourse.numenta.org/u/gidmeister)
#### Post date: [September 28, 2017, 8:07pm UTC](https://discourse.numenta.org/t/untangling-sequences-behavior-vs-external-causes/2883/2 "2017-09-28T20:07:54Z")
I’m unclear on three basic points.
1. The temporal sequence layer and the sensorimotor sequence layers are traversed by the same mini-column. So does that mean that when a burst happens, it activates every cell in one minicolumn in both layers? And conversely if a predicted cell exists, it inhibits every other cell in its minicolumn in both layers?
(In another part of the article, you say that having just one layer, where half the cells have a location input, and the other half have context inputs from adjacent cells, would also work. This sounds as if the layers are independent)
2. There is a difference between incorrect predictions and no predictions. Does the article say that the temporal sequence layer simply makes very few predictions when faced with a sequence caused by feeling an object?
3. Does the object layer have any effect on the temporal sequence layer?
As a far-fetched application, maybe you could use this (just the sensorimotor part) for encryption. You could have two patterns, one as the context pattern in the sensorimotor layer, and the other as the sensory pattern feeding that layer. Only with the correct context (location) pattern, and the correct sensory pattern, would you get a significant output. So one pattern would be the key to unlock the other.
Anyway, hoping for answers before I do a re-read.
Thanks
---
### Author:  [@subutai](https://discourse.numenta.org/u/subutai)
#### Post date: [September 28, 2017, 8:48pm UTC](https://discourse.numenta.org/t/untangling-sequences-behavior-vs-external-causes/2883/3 "2017-09-28T20:48:44Z")
> [@gidmeister](#):
>
> (In another part of the article, you say that having just one layer, where half the cells have a location input, and the other half have context inputs from adjacent cells, would also work. This sounds as if the layers are independent)
Yes, in this paper the minicolumns in the two layers are independent.
> [@gidmeister](#):
>
> Does the article say that the temporal sequence layer simply makes very few predictions when faced with a sequence caused by feeling an object?
It can make a reasonable number of predictions, but they will be pretty random. The number of correct predictions will be pretty small. You can see this difference in Figure 5A
> [@gidmeister](#):
>
> Does the object layer have any effect on the temporal sequence layer?
Not in this paper. I thought about adding it (it should work fine) but I didn’t have time to experiment with that (was under a deadline 🙂 ). We have done this elsewhere in a paper that is in progress. It would mainly impact noise robustness, which I didn’t really tackle in this paper.
---
### Author:  [@Bitking](https://discourse.numenta.org/u/Bitking)
#### Post date: [February 4, 2018, 6:34pm UTC](https://discourse.numenta.org/t/untangling-sequences-behavior-vs-external-causes/2883/4 "2018-02-04T18:34:34Z")
I was reading a paper with content related to this topic.
In the paper, the authors say that the separation of the layers supports prediction and feedback on the success of that prediction. This includes considerable research to support the model referenced in the paper.
It goes on to build an intriguing system where the perception error and related learning is distributed along the what and where stream using local processing. Both the bottom up and top down information flow is described with an excellent exposition of how this all fits together. This model is more in-line with plausible biological function than anything coming out of the deep learning camp.
The part relevant to this discussion starts on page 4, left-hand side of the page. “How are the prediction and actual outcome separately represented, and how is the timing of the prediction and outcome coordinated & organized?” [1]
[1] Deep Predictive Learning: A Comprehensive Model of Three Visual Streams
[https://arxiv.org/abs/1709.04654](https://arxiv.org/abs/1709.04654)
---
### Author:  [@rhyolight](https://discourse.numenta.org/u/rhyolight)
#### Post date: [February 21, 2018, 5:12pm UTC](https://discourse.numenta.org/t/untangling-sequences-behavior-vs-external-causes/2883/5 "2018-02-21T17:12:31Z")
5 posts were split to a new topic: [When I look at what deep neural networks do](https://discourse.numenta.org/t/when-i-look-at-what-deep-neural-networks-do/3498)
---
### Author:  [@rogert](https://discourse.numenta.org/u/rogert)
#### Post date: [July 12, 2019, 5:29pm UTC](https://discourse.numenta.org/t/untangling-sequences-behavior-vs-external-causes/2883/6 "2019-07-12T17:29:20Z")
The HTM-scheme [untangling\_sequences](https://github.com/rogerturner/HTM-scheme/tree/master/projects/untangling_sequences) project implements temporal sequence and sensorimotor inference algorithms combined in one minicolumn (@gidmeister 's point 1 above), apical feedback to both parts of layer 4, and multiple cortical columns.
It uses the HTM-scheme translations of column\_pooler.py and apical\_tiebreak\_temporal\_memory.py (which aim to be “htmresearch compatible”; attm is extended to allow unified minicolumn bursting)
I’ve put example output in a [separate topic](https://discourse.numenta.org/t/untangling-sequences-revisited/6295)
---
### Author:  [@rogert](https://discourse.numenta.org/u/rogert)
#### Post date: [September 12, 2019, 2:01pm UTC](https://discourse.numenta.org/t/untangling-sequences-behavior-vs-external-causes/2883/7 "2019-09-12T14:01:17Z")
(September 2019 update): [untangling\_sequences](https://github.com/rogerturner/HTM-scheme/tree/master/projects/untangling_sequences) is now an experiment in configuring an HTM model closer to the biology of Layer 4. It implements 3 classes of L4 excitatory cells with interconnections, as well as the apical feedback from L2/3, as specified in figures 8 and 9 of [Izhikevich & Edelman 2008 Large-scale model of mammalian thalamocortical systems SI](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.126.1325&rep=rep1&type=pdf):
When run with the biologically plausible 100 minicolumns per cortical column, the characteristic model behaviour of distinguishing sensorimotor inputs from temporal sequences is still clear:
