# HTM for IT

**URL:** <https://discourse.numenta.org/t/htm-for-it/6588>\
**Category:** Applications\
**Created:** [September 15, 2019, 8:35am UTC](https://discourse.numenta.org/t/htm-for-it/6588 "2019-09-15T08:35:45Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![life\_happy](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/life_happy/32/4502_2.png) [@life\_happy](https://discourse.numenta.org/u/life_happy)\
**Post date:** [October 2, 2019, 12:10pm UTC](https://discourse.numenta.org/t/htm-for-it/6588/21 "2019-10-02T12:10:52Z")

</div>

Thank you @sheiser1  
I found this  
[http://nupic.docs.numenta.org/1.0.3/api/algorithms/anomaly-detection.html](http://nupic.docs.numenta.org/1.0.3/api/algorithms/anomaly-detection.html)  
I have ư questions, hope get advice from you and others in thí forum

1. is there any quickstart for above anomaly detection algorithm code
2. in the above code they use the active columns and predicted columns to calculate the anomaly score. But as i know the output of SP is active columns indecies í ok but the output of TM is predicted cells (not columns). Can any one explain ?  
Thank you very much

_class_`nupic.algorithms.anomaly.` `Anomaly` ( _slidingWindowSize=None_ , _mode=‘pure’_ , _binaryAnomalyThreshold=None_ )

`compute` ( _activeColumns_ , _predictedColumns_ , _inputValue=None_ , _timestamp=None_ )[¶](http://nupic.docs.numenta.org/1.0.3/api/algorithms/anomaly-detection.html#nupic.algorithms.anomaly.Anomaly.compute)

Compute the anomaly score as the percent of active columns not predicted.

Parameters: \* **activeColumns** – array of active column indices

- **predictedColumns** – array of columns indices predicted in this step (used for anomaly in step T+1)
- **inputValue** – (optional) value of current input to encoders (eg “cat” for category encoder) (used in anomaly-likelihood)
- **timestamp** – (optional) date timestamp when the sample occured (used in anomaly-likelihood)  
Returns: the computed anomaly score; float 0…1

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [October 2, 2019, 6:13pm UTC](https://discourse.numenta.org/t/htm-for-it/6588/22 "2019-10-02T18:13:47Z")

</div>

> [@life\_happy](#):
>
> - is there any quickstart for above anomaly detection algorithm code

Sorry, no.

> [@life\_happy](#):
>
> in the above code they use the active columns and predicted columns to calculate the anomaly score. But as i know the output of SP is active columns indecies í ok but the output of TM is predicted cells (not columns).

When the TM predicts cells, they are in minicolumns structures. To see if the prediction was correct, we compare the next set of active minicolumns from the SP and see if the predicted cells were within those minicolumns.

---

<div class="post-metadata">

**Author:** ![life\_happy](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/life_happy/32/4502_2.png) [@life\_happy](https://discourse.numenta.org/u/life_happy)\
**Post date:** [October 3, 2019, 1:31am UTC](https://discourse.numenta.org/t/htm-for-it/6588/23 "2019-10-03T01:31:45Z")

</div>

thank you @rhyolight. i will study more

---

<div class="post-metadata">

**Author:** ![sheiser1](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sheiser1/32/766_2.png) [@sheiser1](https://discourse.numenta.org/u/sheiser1)\
**Post date:** [October 3, 2019, 2:46am UTC](https://discourse.numenta.org/t/htm-for-it/6588/24 "2019-10-03T02:46:25Z")

</div>

> [@rhyolight](#):
>
> When the TM predicts cells, they are in minicolumns structures. To see if the prediction was correct, we compare the next set of active minicolumns from the SP and see if the predicted cells were within those minicolumns

Yes that sounds right to me, and I’d recommend starting here:

[https://nupic.docs.numenta.org/1.0.3/quick-start/algorithms.html](https://nupic.docs.numenta.org/1.0.3/quick-start/algorithms.html)

I’d say first get familiar with how to instantiate encoders, SP and TM objects and feed them data.

Once you’re there you can TM.getPreictedCells(), then for each of these cells get their corresponding column with TM.columnFromCell(cell).

From there you can store those columns as like ‘previouslyPredictedColumns’ and compare them to the columns activated by the SP at the next time step. The proportion of the SP active columns which were also previouslyPredicted is the anomaly score.

Then the next step is to feed the anomaly scores into an anomaly likelihood object to get the likelihood values.

---

<div class="post-metadata">

**Author:** ![life\_happy](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/life_happy/32/4502_2.png) [@life\_happy](https://discourse.numenta.org/u/life_happy)\
**Post date:** [October 6, 2019, 9:19am UTC](https://discourse.numenta.org/t/htm-for-it/6588/25 "2019-10-06T09:19:40Z")

</div>

thank you very much for your advice. that 's a clear route

---

<div class="post-metadata">

**Author:** ![life\_happy](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/life_happy/32/4502_2.png) [@life\_happy](https://discourse.numenta.org/u/life_happy)\
**Post date:** [October 16, 2019, 2:23pm UTC](https://discourse.numenta.org/t/htm-for-it/6588/26 "2019-10-16T14:23:51Z")

</div>

thankyou @rhyolight

I have a dump question: In the TM space , we talk about 2-D matrix ( horizon is number of colums and verizon is cells per columns)  
Default value is :  
columnCount: 2048  
cellsPerColumn: 32

But after we do

# Execute Temporal Memory algorithm over active mini-columns.

```
  tm.compute(activeColumnIndices, learn=True)

  activeCells = tm.getActiveCells()  

```

the getActiveCells() function returns the indices of Active cells . It’s a list ( 1-D) not a matrix ( 2-D). How can i convert an index ( that the getActicell() returns) to exact the position of column and the cell in 2-D matrix ?

Thank you very much.

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [October 16, 2019, 2:41pm UTC](https://discourse.numenta.org/t/htm-for-it/6588/27 "2019-10-16T14:41:24Z")

</div>

> [@life\_happy](#):
>
> It’s a list ( 1-D) not a matrix ( 2-D). How can i convert an index ( that the getActicell() returns) to exact the position of column and the cell in 2-D matrix ?

The array is simply flattened, so you have to do the unflattening. There is a minicolumn represented every `cellsPerColumn` elements.

---

<div class="post-metadata">

**Author:** ![life\_happy](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/life_happy/32/4502_2.png) [@life\_happy](https://discourse.numenta.org/u/life_happy)\
**Post date:** [October 18, 2019, 1:09pm UTC](https://discourse.numenta.org/t/htm-for-it/6588/28 "2019-10-18T13:09:49Z")

</div>

> [@rhyolight](#):
>
> The array is simply flattened, so you have to do the unflattening. There is a minicolumn represented every `cellsPerColumn` elements.

thank you. Is there any function to map a cell index to exactly position of column and and cell ?

By the way do we have an simple example about RandomDistributedScalarEncoder ? I searched but i can not find anything. The explaination in the [http://nupic.docs.numenta.org/1.0.5/api/algorithms/encoders.html](http://nupic.docs.numenta.org/1.0.5/api/algorithms/encoders.html) is not easy to understand.

Thank you very much

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [October 18, 2019, 4:14pm UTC](https://discourse.numenta.org/t/htm-for-it/6588/29 "2019-10-18T16:14:19Z")

</div>

> [@life\_happy](#):
>
> Is there any function to map a cell index to exactly position of column and and cell ?

I believe it is just a couple lines of code. Or use numpy.shape().

> [@life\_happy](#):
>
> simple example about RandomDistributedScalarEncoder ?

Check the [nupic walkthrough notebook](https://github.com/numenta/nupic/blob/master/examples/NuPIC%20Walkthrough.ipynb).

---

<div class="post-metadata">

**Author:** ![thanh-binh.to](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/thanh-binh.to/32/1793_2.png) [@thanh-binh.to](https://discourse.numenta.org/u/thanh-binh.to)\
**Post date:** [October 18, 2019, 5:08pm UTC](https://discourse.numenta.org/t/htm-for-it/6588/30 "2019-10-18T17:08:14Z")

</div>

@life\_you can easily write an simple code to split sdr array into nColumns subarrays and each subarray has nCellsPerColums

---

<div class="post-metadata">

**Author:** ![sheiser1](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sheiser1/32/766_2.png) [@sheiser1](https://discourse.numenta.org/u/sheiser1)\
**Post date:** [October 18, 2019, 6:21pm UTC](https://discourse.numenta.org/t/htm-for-it/6588/31 "2019-10-18T18:21:50Z")

</div>

> [@rhyolight](#):
>
> Is there any function to map a cell index to exactly position of column and and cell ?

There’s a function in TM called _columnForCell()_. I think its not in BacktrackingTM, though you can replicate it with:

colIndex = int(cellIndex / SPsize)

Then if you want the position of that cell in the column you could use mod/remainder:

cellPositionInCol = cellIndex % SPsize

---

<div class="post-metadata">

**Author:** ![life\_happy](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/life_happy/32/4502_2.png) [@life\_happy](https://discourse.numenta.org/u/life_happy)\
**Post date:** [October 19, 2019, 1:31pm UTC](https://discourse.numenta.org/t/htm-for-it/6588/32 "2019-10-19T13:31:18Z")

</div>

thank you. i will try

---

<div class="post-metadata">

**Author:** ![life\_happy](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/life_happy/32/4502_2.png) [@life\_happy](https://discourse.numenta.org/u/life_happy)\
**Post date:** [October 19, 2019, 1:31pm UTC](https://discourse.numenta.org/t/htm-for-it/6588/33 "2019-10-19T13:31:37Z")

</div>

thank you

---

<div class="post-metadata">

**Author:** ![life\_happy](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/life_happy/32/4502_2.png) [@life\_happy](https://discourse.numenta.org/u/life_happy)\
**Post date:** [October 19, 2019, 1:33pm UTC](https://discourse.numenta.org/t/htm-for-it/6588/34 "2019-10-19T13:33:49Z")

</div>

thank you very much. I will follow the note book to understand more.

---

<div class="post-metadata">

**Author:** ![life\_happy](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/life_happy/32/4502_2.png) [@life\_happy](https://discourse.numenta.org/u/life_happy)\
**Post date:** [October 21, 2019, 2:23pm UTC](https://discourse.numenta.org/t/htm-for-it/6588/35 "2019-10-21T14:23:00Z")

</div>

Dear all!

Now i can understand Classifier . So i move forward to anomaly detect.

In the document [http://nupic.docs.numenta.org/1.0.3.dev0/api/algorithms/anomaly-detection.html](http://nupic.docs.numenta.org/1.0.3.dev0/api/algorithms/anomaly-detection.html)

I see this:

`compute` ( _activeColumns_ , _predictedColumns_ , _inputValue=None_ , _timestamp=None_ )

Compute the anomaly score as the percent of active columns not predicted.

Parameters: \* **activeColumns** – array of active column indices

- **predictedColumns** – array of columns indices predicted in this step (used for anomaly in step T+1)
- **inputValue** – (optional) value of current input to encoders (eg “cat” for category encoder) (used in anomaly-likelihood)
- **timestamp** – (optional) date timestamp when the sample occured (used in anomaly-likelihood)  
Returns: the computed anomaly score; float 0…1

I have a question:

In Parameters above:

**activeColumns** : i can get it from the SP in this round ( each round is in one data row in the dataset file)

But **predictedColumns** : do i get it from the TM in this round or i have to get it from the last round.

Thank you very much.

---

<div class="post-metadata">

**Author:** ![sheiser1](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/sheiser1/32/766_2.png) [@sheiser1](https://discourse.numenta.org/u/sheiser1)\
**Post date:** [October 21, 2019, 4:24pm UTC](https://discourse.numenta.org/t/htm-for-it/6588/36 "2019-10-21T16:24:28Z")

</div>

> [@life\_happy](#):
>
> **predictedColumns** : do i get it from the TM in this round or i have to get it from the last round

From the last round. The anomaly score is the proportion of activated columns at time (t) which were _not_ predicted at time (t-1).

---

<div class="post-metadata">

**Author:** ![marty1885](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/marty1885/32/1891_2.png) [@marty1885](https://discourse.numenta.org/u/marty1885)\
**Post date:** [October 21, 2019, 4:34pm UTC](https://discourse.numenta.org/t/htm-for-it/6588/37 "2019-10-21T16:34:42Z")

</div>

You use the predicted columns from t-1 and the actual active columns from t. Yes this causes the anomaly score to always be 1 at t = 0 (Since t=-1 doesn’t exist) but you can just ignore that special case.

---

<div class="post-metadata">

**Author:** ![life\_happy](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/life_happy/32/4502_2.png) [@life\_happy](https://discourse.numenta.org/u/life_happy)\
**Post date:** [October 26, 2019, 12:43am UTC](https://discourse.numenta.org/t/htm-for-it/6588/38 "2019-10-26T00:43:36Z")

</div>

after get your guides/advices @marty1885 @sheiser1… i have run successfully the anomaly, anomalylikelihood, loglikelihood for hotgym dataset ( i use agorithms not OPF ). I tried change some paramether in scalar encoder, TM, anomly() and anomalylikelihood() ( eg: resolution, permernace increase/decrase, slidingwindow , learningPeriod …) and i got different anomaly detection results. I would like to check the right anomaly from the dataset.  
I have some question below :

1. As i search and read in this forum , there is no reference result for hotGym datasets. Is it right ?
2. I read the NAB. I am using python 2.7. i found this link [https://github.com/numenta/NAB/tree/master/nab/detectors/numenta](https://github.com/numenta/NAB/tree/master/nab/detectors/numenta)  
Is this link is right for my python 2.7, Nupic and algorithm ? if i use this guide, can i check my algorithm with my own parameter ?
3. Or can i use 1 dataset (datafile) like this [https://github.com/numenta/NAB/blob/master/data/realKnownCause/machine\_temperature\_system\_failure.csv](https://github.com/numenta/NAB/blob/master/data/realKnownCause/machine_temperature_system_failure.csv)  
(i see this datafile look like HotGym datafile) and then run my algorithm and check the result. But how can i know my anomaly detection result is right or not ? Or i check maunally with the result file [https://raw.githubusercontent.com/numenta/NAB/master/results/numenta/realKnownCause/numenta\_machine\_temperature\_system\_failure.csv](https://raw.githubusercontent.com/numenta/NAB/master/results/numenta/realKnownCause/numenta_machine_temperature_system_failure.csv)

timestamp,value,anomaly\_score,raw\_score,label,S(t)\_reward\_low\_FP\_rate,S(t)\_reward\_low\_FN\_rate,S(t)\_standard  
2013-12-02 21:15:00,73.96732207,0.0301029996659,1.0,0,0.0,0.0,0.0  
2013-12-02 21:20:00,74.935882,0.0301029996659,1.0,0,0.0,0.0,0.0  
2013-12-02 21:25:00,76.12416182,1.0,1.0,0,0.0,0.0,0.0

–\> but i dont know what is exacly anomly point in this file

Thank you very much

---

<div class="post-metadata">

**Author:** ![marty1885](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/marty1885/32/1891_2.png) [@marty1885](https://discourse.numenta.org/u/marty1885)\
**Post date:** [October 26, 2019, 2:28am UTC](https://discourse.numenta.org/t/htm-for-it/6588/39 "2019-10-26T02:28:53Z")

</div>

1. People have been using the Hot Gym data a lot. See  
[Hot Gym running in Python 3 via nupic.cpp & pybind](https://discourse.numenta.org/t/hot-gym-running-in-python-3-via-nupic-cpp-pybind/5949)  
[High anomalylikelihood values for hot gym anomaly example](https://discourse.numenta.org/t/high-anomalylikelihood-values-for-hot-gym-anomaly-example/2311)  
etc…

2. Yes, you can modify the parameters there. Please do be aware that the actual model creation happens in a model factory in [OPF](https://github.com/numenta/nupic/blob/50c5fd0dc94f2ffb205544ed11fe82ad5bb0de18/docs/source/guides/opf.md).

3. It should work.

---

<div class="post-metadata">

**Author:** ![life\_happy](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/life_happy/32/4502_2.png) [@life\_happy](https://discourse.numenta.org/u/life_happy)\
**Post date:** [October 27, 2019, 4:54am UTC](https://discourse.numenta.org/t/htm-for-it/6588/40 "2019-10-27T04:54:02Z")

</div>

thank you @marty1885

I found this [https://www.businesswire.com/news/home/20170620005440/en/New-Research-Paper-Numenta-Demonstrates-Results-Machine](https://www.businesswire.com/news/home/20170620005440/en/New-Research-Paper-Numenta-Demonstrates-Results-Machine)  
This paper show that there are 3 anomalies in detail ( 1 is planned, 2 are outages) and a plot it’s very clear.  
[https://plot.ly/~sjd171/2636.embed](https://plot.ly/~sjd171/2636.embed)

All are good for me to check my model.

> [@marty1885](#):
>
> Please do be aware that the actual model creation happens in a model factory in [OPF](https://github.com/numenta/nupic/blob/50c5fd0dc94f2ffb205544ed11fe82ad5bb0de18/docs/source/guides/opf.md).

That you mean i should use OPF for real applications ?

Now i follow algorithm [http://nupic.docs.numenta.org/1.0.3/api/algorithms/index.html](http://nupic.docs.numenta.org/1.0.3/api/algorithms/index.html)

I think it 's esier to understand because it follow the HTM theory. After i understand Algorithm, i will move forward to OPF for real applications . Do you think so ? Is it a right approach ?

Thank you very much.

[Previous page](https://discourse.numenta.org/t/htm-for-it/6588.md?page=1)

[Next page](https://discourse.numenta.org/t/htm-for-it/6588.md?page=3)