# Results Comparison: HTM Studio vs Nupic

**URL:** <https://discourse.numenta.org/t/results-comparison-htm-studio-vs-nupic/2573>\
**Category:** Applications\
**Created:** [July 23, 2017, 7:11am UTC](https://discourse.numenta.org/t/results-comparison-htm-studio-vs-nupic/2573 "2017-07-23T07:11:15Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![wip\_user](https://avatars.discourse-cdn.com/v4/letter/w/848f3c/32.png) [@wip\_user](https://discourse.numenta.org/u/wip_user)\
**Post date:** [July 23, 2017, 7:11am UTC](https://discourse.numenta.org/t/results-comparison-htm-studio-vs-nupic/2573/1 "2017-07-23T07:11:15Z")

</div>

Hi  
Why nupic results are not matching with HTM Studio results ? which one is accurate ?  
Pls see the below screenshot of results .

 ![](https://canada1.discourse-cdn.com/flex030/uploads/numenta/original/2X/e/e8205e35e410b13083b732d3bf643a0e14263bc7.png)

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [July 23, 2017, 1:58pm UTC](https://discourse.numenta.org/t/results-comparison-htm-studio-vs-nupic/2573/2 "2017-07-23T13:58:26Z")

</div>

You are comparing NuPIC’s “raw anomaly score”, which is not the same as anomaly likelihood. If you want a fair comparison, add [`AnomalyLikelihood`](http://nupic.docs.numenta.org/1.0.0/api/algorithms/anomaly-detection.html#module-nupic.algorithms.anomaly_likelihood) to your NuPIC runtime.

---

<div class="post-metadata">

**Author:** ![wip\_user](https://avatars.discourse-cdn.com/v4/letter/w/848f3c/32.png) [@wip\_user](https://discourse.numenta.org/u/wip_user)\
**Post date:** [July 23, 2017, 8:06pm UTC](https://discourse.numenta.org/t/results-comparison-htm-studio-vs-nupic/2573/3 "2017-07-23T20:06:35Z")

</div>

Hi Matt ,  
Thanks for the sharing the link , it is really helpful ,

if we dont pass any parameter in Anomaly.create() in goes with PURE in both Python and Java

As per below documentation, in PURE mode , nupic calculates raw anomaly score .  
MODE\_PURE = 'pure’  
Default mode. The raw anomaly score as computed by computeRawAnomalyScore()

if that is the case , in above image “anomaly\_score” on left side , should match with right side studio raw anomaly\_score .

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [July 24, 2017, 3:47pm UTC](https://discourse.numenta.org/t/results-comparison-htm-studio-vs-nupic/2573/4 "2017-07-24T15:47:01Z")

</div>

> [@wip\_user](#):
>
> if we dont pass any parameter in Anomaly.create() in goes with PURE in both Python and Java

Wait, I thought you were comparing the results of HTM Studio and NuPIC? Neither of these are Java-based…

---

<div class="post-metadata">

**Author:** ![wip\_user](https://avatars.discourse-cdn.com/v4/letter/w/848f3c/32.png) [@wip\_user](https://discourse.numenta.org/u/wip_user)\
**Post date:** [July 24, 2017, 4:43pm UTC](https://discourse.numenta.org/t/results-comparison-htm-studio-vs-nupic/2573/5 "2017-07-24T16:43:29Z")

</div>

Hi Matt (@rhyolight)

Apologies for confusion , I am comparing between Nupic vs HTM studio , screen shot also from the same.

Thanks & Regards  
Hareesh

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [July 24, 2017, 4:50pm UTC](https://discourse.numenta.org/t/results-comparison-htm-studio-vs-nupic/2573/6 "2017-07-24T16:50:18Z")

</div>

It still seems like you are comparing the raw anomaly scores from NuPIC to the anomaly likelihood in HTM Studio. Where is your NuPIC code? I want to see how you are setting up the anomaly / anomaly likelihood class. You should use [`MODE_LIKELIHOOD`](http://nupic.docs.numenta.org/1.0.0/api/algorithms/anomaly-detection.html#nupic.algorithms.anomaly.Anomaly.MODE_LIKELIHOOD) when creating the Anomaly class.

---

<div class="post-metadata">

**Author:** ![wip\_user](https://avatars.discourse-cdn.com/v4/letter/w/848f3c/32.png) [@wip\_user](https://discourse.numenta.org/u/wip_user)\
**Post date:** [July 24, 2017, 4:54pm UTC](https://discourse.numenta.org/t/results-comparison-htm-studio-vs-nupic/2573/7 "2017-07-24T16:54:25Z")

</div>

Hi Matt (@rhyolight)

I am using the below code for anomaly score

> <https://github.com/numenta/nupic/blob/master/examples/opf/clients/hotgym/anomaly/hotgym_anomaly.py>

Added the below lines to likelihood score after model = createModel() line

```python
  anomalyLikelihood = anomaly_likelihood.AnomalyLikelihood()

```

in the loop the below piece code to get likelihood

```python
likelihood = anomalyLikelihood.anomalyProbability(
        modelInput["consumption"], anomalyScore, modelInput["timestamp"]
      )

```

---

<div class="post-metadata">

**Author:** ![wip\_user](https://avatars.discourse-cdn.com/v4/letter/w/848f3c/32.png) [@wip\_user](https://discourse.numenta.org/u/wip_user)\
**Post date:** [July 25, 2017, 11:14pm UTC](https://discourse.numenta.org/t/results-comparison-htm-studio-vs-nupic/2573/8 "2017-07-25T23:14:45Z")

</div>

@rhyolight Could you pls comments on this .

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [July 25, 2017, 11:17pm UTC](https://discourse.numenta.org/t/results-comparison-htm-studio-vs-nupic/2573/9 "2017-07-25T23:17:30Z")

</div>

In the chart, it says you’re plotting the `raw_anomaly_score` from HTM Studio. But you are comparing it to the anomaly likelihood in NuPIC. That still seems like the problem.

---

<div class="post-metadata">

**Author:** ![wip\_user](https://avatars.discourse-cdn.com/v4/letter/w/848f3c/32.png) [@wip\_user](https://discourse.numenta.org/u/wip_user)\
**Post date:** [July 25, 2017, 11:21pm UTC](https://discourse.numenta.org/t/results-comparison-htm-studio-vs-nupic/2573/10 "2017-07-25T23:21:49Z")

</div>

Hi Matt (@rhyolight)

Apologies for troubling you  
anomaly\_score is calculated using pure mode(raw\_anomaly\_score) ,which should equal to “raw\_anomaly\_score” of studio  
 ![image](https://canada1.discourse-cdn.com/flex030/uploads/numenta/original/2X/c/ca35537a15e5e1de2ebb56f4f621a871393d4722.jpg)

Thanks & Regards  
Hareesh

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [July 26, 2017, 12:13am UTC](https://discourse.numenta.org/t/results-comparison-htm-studio-vs-nupic/2573/11 "2017-07-26T00:13:37Z")

</div>

I did not even know that you could get a raw anomaly score out of HTM Studio. You should also be able to get the anomaly likelihood from HTM Studio too, right? More than just a “HIGH” anomaly level, I mean. That is the number you should be comparing to the anomaly likelihood coming out of NuPIC.

Also, I don’t know exactly how models are created and run in HTM Studio, but it is using NuPIC of course. There are probably some small differences in the model configurations between these systems that could cause differences. But overall you should get comparable anomaly likelihood values for the same input data for both systems.