# Continuous online sequence learning with an unsupervised neural network model

**URL:** <https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573>\
**Category:** Related Papers\
**Created:** [May 25, 2016, 3:37pm UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573 "2016-05-25T15:37:54Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [May 25, 2016, 3:37pm UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/1 "2016-05-25T15:37:54Z")

</div>

[![](https://canada1.discourse-cdn.com/flex030/uploads/numenta/original/1X/25591e2549cfe3018216ef5250f3de9197264c51.png)](http://arxiv.org/abs/1512.05463)

By @ycui, @subutai, & @jhawkins

Please discuss [this paper](http://arxiv.org/abs/1512.05463) below.

---

<div class="post-metadata">

**Author:** ![ycui](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/ycui/32/253_2.png) [@ycui](https://discourse.numenta.org/u/ycui)\
**Post date:** [November 1, 2016, 7:16pm UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/2 "2016-11-01T19:16:36Z")

</div>

The paper is now published in Neural Computation. The final version is available [here](http://www.mitpressjournals.org/doi/abs/10.1162/NECO_a_00893#.WBjX8eErLUb)

---

<div class="post-metadata">

**Author:** ![blue2](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/blue2/32/5050_2.png) [@blue2](https://discourse.numenta.org/u/blue2)\
**Post date:** [February 10, 2018, 3:56pm UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/3 "2018-02-10T15:56:04Z")

</div>

Is the code available for this somewhere? Thanks.

---

<div class="post-metadata">

**Author:** ![blue2](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/blue2/32/5050_2.png) [@blue2](https://discourse.numenta.org/u/blue2)\
**Post date:** [February 14, 2018, 12:40am UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/4 "2018-02-14T00:40:54Z")

</div>

Anyone? @ycui @subutai @jhawkins? Pretty please?

I’d like to replicate your findings and experiment with decoding  
predictions. Currently reading the Bridle 1990 paper.

Thanks.

---

<div class="post-metadata">

**Author:** ![Bitking](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/bitking/32/3099_2.png) [@Bitking](https://discourse.numenta.org/u/Bitking)\
**Post date:** [February 14, 2018, 2:13am UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/5 "2018-02-14T02:13:45Z")

</div>

Perhaps what you are looking for can be found here:

> **[numenta/htmpapers](https://github.com/numenta/htmpapers)**
>
> Numenta published papers code and data. Contribute to numenta/htmpapers development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![blue2](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/blue2/32/5050_2.png) [@blue2](https://discourse.numenta.org/u/blue2)\
**Post date:** [February 14, 2018, 2:46am UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/6 "2018-02-14T02:46:31Z")

</div>

Presumably that’s where it should be but I can only see code pertaining  
to two other papers there.

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [February 14, 2018, 4:19pm UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/8 "2018-02-14T16:19:17Z")

</div>

The paper says it is in [https://github.com/numenta/htmresearch/](https://github.com/numenta/htmresearch/) and I assume it is in [here](https://github.com/numenta/htmresearch/tree/master/projects) somewhere. I’ll ask @subutai today.

---

<div class="post-metadata">

**Author:** ![subutai](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/subutai/32/106_2.png) [@subutai](https://discourse.numenta.org/u/subutai)\
**Post date:** [February 14, 2018, 5:47pm UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/9 "2018-02-14T17:47:52Z")

</div>

> [@blue2](#):
>
> Anyone? @ycui @subutai @jhawkins? Pretty please?
> 
> I’d like to replicate your findings and experiment with decoding
> 
> predictions. Currently reading the Bridle 1990 paper.

Hi @blue2 - sorry for the late response. Just returned from [this dendrites workshop](https://eitnconf-080218.sciencesconf.org) (which was great).

The original code for the paper is here: [https://github.com/numenta/htmresearch/tree/master/projects/sequence\_prediction](https://github.com/numenta/htmresearch/tree/master/projects/sequence_prediction)

We are slowly cleaning it up, and I can’t promise that it is easy to use right now or fully reproducible. Our goal is to put a cleaned up and properly versioned/reproducible copy here:

> **[GitHub - numenta/htmpapers: Numenta published papers code and data](https://github.com/numenta/htmpapers)**
>
> Numenta published papers code and data. Contribute to numenta/htmpapers development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [February 14, 2018, 7:48pm UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/10 "2018-02-14T19:48:47Z")

</div>

Split this topic on February 14, 2018, 7:48pm UTC

A post was split to a new topic: [Dendrites workshop](https://discourse.numenta.org/t/dendrites-workshop/3467)

---

<div class="post-metadata">

**Author:** ![blue2](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/blue2/32/5050_2.png) [@blue2](https://discourse.numenta.org/u/blue2)\
**Post date:** [February 14, 2018, 11:51pm UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/11 "2018-02-14T23:51:20Z")

</div>

Thanks much appreciated.

---

<div class="post-metadata">

**Author:** ![SirStone](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sirstone/32/5169_2.png) [@SirStone](https://discourse.numenta.org/u/SirStone)\
**Post date:** [June 23, 2020, 9:36am UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/12 "2020-06-23T09:36:14Z")

</div>

Hello  
Sorry if I’m resuming this post but I could not find any answer searching in the forum to my questions about this paper.

I’m trying to create an implementation in Java for the HTM algorithm that could fit the hardware and software limitations I have, so implementing the just the only required to make it go thorough from start to end, for now doesn’t matter if it’s the cutting edge version, it needs just to be up to the job.  
As the picked version of the algorithm I choose the one in this paper because the title says exactly what I’m in need to achieve: Unsupervised continuous online sequence learning.  
I’ve read this paper and a lot of other resources, so my confusion comes from that, I can’t understand if in this paper the spatial pooler learning part is “included mathematically” or if the paper assumes it is already implemented without the need of explain it or if the algorithm doesn’t require to have any permanence values of the column (proximal) synapses. From the paper it says only that the input is matched with the columns and a set Wt of winning columns is received.

Second question is about the “unsupervised” term used in the title; from my knowledge unsupervised means that no “label” is used to mark the input, but the paper clearly uses the target for the SDR classifier, this for me, is not unsupervised learning so I think the title is misleading. Am I wrong?

Thanks in advance

---

<div class="post-metadata">

**Author:** ![sheiser1](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sheiser1/32/766_2.png) [@sheiser1](https://discourse.numenta.org/u/sheiser1)\
**Post date:** [June 23, 2020, 5:18pm UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/13 "2020-06-23T17:18:37Z")

</div>

> [@SirStone](#):
>
> I’m trying to create an implementation in Java for the HTM algorithm that could fit the hardware and software limitations I have

I assume you have seen HTM Java but it doesn’t fit you limitations?

> [@SirStone](#):
>
> I can’t understand if in this paper the spatial pooler learning part is “included mathematically” or if the paper assumes it is already implemented without the need of explain it or if the algorithm doesn’t require to have any permanence values of the column (proximal) synapses.

I’m not sure if SP is used in this paper’s implementation, I assume so since it normally is. If you do make your own implementation I’d recommend starting without SP, just feed the encoding right into TM. The TM is the most non-trivial part, so I’d only add SP once you have fully validated your TM. There’s pseudocode for both in the “Biological and Machine Intelligence” (BAMI) book:

> **[Biological and Machine Intelligence (BAMI)](https://www.numenta.com/resources/htm/biological-and-machine-intelligence/)**
>
> Biological and Machine Intelligence (BAMI) is a living book authored by Numenta researchers and engineers. Its purpose is to document our theoretical framework for both biological and machine intelligence.

> [@SirStone](#):
>
> Second question is about the “unsupervised” term used in the title; from my knowledge unsupervised means that no “label” is used to mark the input, but the paper clearly uses the target for the SDR classifier, this for me, is not unsupervised learning so I think the title is misleading. Am I wrong?

HTM is really _Time-Supervised_. Often in ML the goal is to predict one variable’s value from another (Y from X). If your algorithm uses known Y’s in training its _Supervised_, and if it doesn’t its _Unsupervised_.

What HTM is doing is predicting the value of X at the next timestep (t+1) from X at the current timestep (t). When X(t+1) arrives, it is compared to the predicted value(s) from X(t) and the model is updated. It is basically supervised by the future.

So in practice the model can’t use the target value X(t+1), because that time hasn’t occurred yet. The SDR classifier, which generates predictions in the raw data format, uses the current raw input (X) along with the current prediction(s) for t+1 – which are states of the TM (SDRs). To record prediction accuracy, the raw value at (t+1) is eventually compared to the predicted raw value from the SDR classifier at the previous time step.

---

<div class="post-metadata">

**Author:** ![SirStone](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sirstone/32/5169_2.png) [@SirStone](https://discourse.numenta.org/u/SirStone)\
**Post date:** [June 24, 2020, 11:04am UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/14 "2020-06-24T11:04:59Z")

</div>

> [@sheiser1](#):
>
> I assume you have seen HTM Java but it doesn’t fit you limitations?

Nope, my limitations are:  
-needs to be in Java  
-can’t use external libraries, I’ve tried with the source code but HTM.java itself uses several external libraries  
-can’t excess a certain number of CPU cycles per input fed to the algorithm  
-when saving the object for persistance the sum of saved files can’t excess a certain amount of KB.

> [@sheiser1](#):
>
> I’m not sure if SP is used in this paper’s implementation, I assume so since it normally is. If you do make your own implementation I’d recommend starting without SP, just feed the encoding right into TM. The TM is the most non-trivial part, so I’d only add SP once you have fully validated your TM. There’s pseudocode for both in the “Biological and Machine Intelligence” (BAMI) book:

As I said, the SP is included, it’s the learning part of it that is not clear if it’s there or not.  
Literally I’ve already implemented more than once both SP and TM, but with the wrong tools for the project and not respecting the limits. Now I’ve coded myself a set of sparse vector/matrix objects with the needed functions to handle them with the HTM tasks but had to rewrite everything.  
What do you think If I go with a minimised SP? initialised with random connections and never learn them, without boosting the SP should act exactly as with learning on.

---

<div class="post-metadata">

**Author:** ![sheiser1](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sheiser1/32/766_2.png) [@sheiser1](https://discourse.numenta.org/u/sheiser1)\
**Post date:** [June 24, 2020, 6:14pm UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/15 "2020-06-24T18:14:05Z")

</div>

> [@SirStone](#):
>
> What do you think If I go with a minimised SP?

I’d bet you’ll be ok without SP learning. Without learning though I’m not sure the point of SP.  
It seems you could just use an encoder with 2% sparsity (n=2048, w=40 if you have 1 input field).  
That would drop the need to calculate the overlap scores for SP.

---

<div class="post-metadata">

**Author:** ![SirStone](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sirstone/32/5169_2.png) [@SirStone](https://discourse.numenta.org/u/SirStone)\
**Post date:** [April 17, 2021, 3:39pm UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/16 "2021-04-17T15:39:40Z")

</div>

Hello, I’m back since almost 1 year to resume my pet project.  
Since last time I’ve spent more free time of mine in literature and got more confidence with the HTM theory and the world of machine learning in general.  
I’m still in with this my project of implementing the HTM algorithm in Java with very strict bounds for implementation, memory usage and cpu cycles.  
I’ve found a very performant way to represent and work with structures that are made by bits and are sparse and I’ve lowered by a lot the cpu cycles of my SP implementation but now I’m hitting the TM complexity that I need to master in order to understand how to optimise it for my project.  
In order to do this I wanted to have a diagram and I’ve used Vizgraph to produce one of the TM based on the 2-step description in the “Offical Temporal Memory Algorithm Details” and I would like if someone could take a look at it and check if is correct.  
I’m new to this Vizgraph, so the diagram is quite ugly to see but I don’t want to wait that I learn how make it nicer before understanding if it’s correct or not.  
[Temporal Memory Algorithm Graph](https://lab.davidecappellini.me/TemporalMemory.dot.html)

Hope that I’m not enraging anyone posting here and breaking rules I don’t know about and I thank already anyone that will be kind enough to look into this graph of mine.  
Cheers

---

<div class="post-metadata">

**Author:** ![dmac](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/dmac/32/5842_2.png) [@dmac](https://discourse.numenta.org/u/dmac)\
**Post date:** [April 17, 2021, 5:29pm UTC](https://discourse.numenta.org/t/continuous-online-sequence-learning-with-an-unsupervised-neural-network-model/573/17 "2021-04-17T17:29:59Z")

</div>

> 7| grow new synapses to active cells in t-1

I think this is not quite right, it should read:

> 7| grow new synapses to winner cells in t-1

When a mini-column bursts then many cells activate but only one of them is the designated “winner” cell, and only that cell learns in response to the mini-column activating.