NuPIC: Anomaly Scores report out 0.0 on anomaly/hot_gym/one_gym example

Engineering NuPIC
1

Hello,

When I run the remove_tuesday script, and execute the run script, the anomaly scores in rec-center-hourly_out.csv are all 0.0. I’m currently trying to figure out why the Tuesdays aren’t being set to 5.0, but even if there aren’t obvious anomalies, the score should still be higher than zero, right?

I’ve tried setting the autoDetectWaitRecords property in the rec_center_hourly_model_params, since I saw that it was 4000 in the default xml file, but that had no effect in the result either.

Does anyone know why this may be occurring?

Edit: The remove_tuesday script is working as intended. I was running a diff between the rec-center-hourly file and the backup, and found no differences, so I thought the script wasn’t working. It seems that I accidentally copied over the old backup, so the new backup still has the Tuesdays effectively removed.

1 Like
2

I just re-attempted this example code with the latest NuPIC and I’m not getting the same results that you are. I graphed the output CSV here, and you can see the anomaly likelihood jump up in October:

Screen Shot 2016-12-19 at 8.39.47 AM.png1954×1270 142 KB

Just to be clear, this was not happening before you ran the remove_tuesdays script?

3

I’m using the latest version of NuPIC found on DockerHub. Here are the results I’m getting (Can’t upload files so raw text will have to do):

timestamp,kw_energy_consumption,prediction,anomaly_score,anomaly_likelihood
2010-07-02 00:00:00,21.2,21.2,1.0,0.5
2010-07-02 01:00:00,16.4,16.4,1.0,0.5
2010-07-02 02:00:00,4.7,16.4,1.0,0.5
2010-07-02 03:00:00,4.7,4.7,0.0,0.5
2010-07-02 04:00:00,4.6,4.7,0.0,0.5
2010-07-02 05:00:00,23.5,4.67,0.0,0.5
2010-07-02 06:00:00,47.5,47.5,0.0,0.5
2010-07-02 07:00:00,45.4,45.4,0.0,0.5
2010-07-02 08:00:00,46.1,46.1,0.0,0.5
2010-07-02 09:00:00,41.5,41.5,0.0,0.5
2010-07-02 10:00:00,43.4,43.4,0.0,0.5
2010-07-02 11:00:00,43.8,43.8,0.0,0.5
2010-07-02 12:00:00,37.8,37.8,0.0,0.5
2010-07-02 13:00:00,36.6,36.6,0.0,0.5
2010-07-02 14:00:00,35.7,35.7,0.0,0.5
2010-07-02 15:00:00,38.9,38.9,0.0,0.5
2010-07-02 16:00:00,36.2,36.2,0.0,0.5
2010-07-02 17:00:00,36.6,36.6,0.0,0.5
2010-07-02 18:00:00,37.2,37.2,0.0,0.5
2010-07-02 19:00:00,38.2,38.2,0.0,0.5
2010-07-02 20:00:00,14.1,14.1,0.0,0.5
2010-07-02 21:00:00,5.1,5.1,0.0,0.5
2010-07-02 22:00:00,5.0,5.0,0.0,0.5
2010-07-02 23:00:00,5.9,5.9,0.0,0.5
2010-07-03 00:00:00,22.5,22.5,0.0,0.5
2010-07-03 01:00:00,22.3,22.3,0.0,0.5
2010-07-03 02:00:00,6.0,6.0,0.0,0.5
2010-07-03 03:00:00,4.7,4.7,0.0,0.5
2010-07-03 04:00:00,4.4,4.4,0.0,0.5
2010-07-03 05:00:00,4.8,4.8,0.0,0.5
2010-07-03 06:00:00,23.4,23.4,0.0,0.5
2010-07-03 07:00:00,42.4,42.4,0.0,0.5
2010-07-03 08:00:00,52.0,52.0,0.0,0.5
2010-07-03 09:00:00,50.8,50.8,0.0,0.5

…and so on until around line 390:

2010-07-18 04:00:00,4.7,4.7,0.0,0.041630779742412583
2010-07-18 05:00:00,4.9,4.9,0.0,0.041630779742412583
2010-07-18 06:00:00,5.0,5.0,0.0,0.041630779742412583
2010-07-18 07:00:00,5.5,5.5,0.0,0.041630779742412583
2010-07-18 08:00:00,38.4,38.4,0.0,0.041630779742412583
2010-07-18 09:00:00,46.7,46.7,0.0,0.041630779742412583
2010-07-18 10:00:00,39.3,39.3,0.0,0.041630779742412583
2010-07-18 11:00:00,39.6,39.6,0.0,0.041630779742412583
2010-07-18 12:00:00,39.8,39.8,0.0,0.041630779742412583
2010-07-18 13:00:00,42.3,5.046165833265324,0.0,0.041630779742412583
2010-07-18 14:00:00,44.6,5.046165833265324,0.0,0.041630779742412583
2010-07-18 15:00:00,46.5,46.5,0.0,0.041630779742412583
2010-07-18 16:00:00,40.1,4.699567849668804,0.0,0.041630779742412583
2010-07-18 17:00:00,25.3,5.046165833265324,0.0,0.041630779742412583
2010-07-18 18:00:00,9.6,9.6,0.0,0.041630779742412583
2010-07-18 19:00:00,5.2,5.2,0.0,0.041630779742412583
2010-07-18 20:00:00,5.2,5.2,0.0,0.041630779742412583
2010-07-18 21:00:00,5.2,5.2,0.0,0.041630779742412583
2010-07-18 22:00:00,5.2,5.2,0.0,0.041630779742412583
2010-07-18 23:00:00,15.9,15.9,0.0,0.041630779742412583
2010-07-19 00:00:00,21.0,5.163064416567003,0.0,0.041630779742412583
2010-07-19 01:00:00,16.7,16.7,0.0,0.041630779742412583
2010-07-19 02:00:00,4.9,4.9,0.0,0.041630779742412583
2010-07-19 03:00:00,5.1,5.1,0.0,0.041630779742412583
2010-07-19 04:00:00,4.9,4.9,0.0,0.041630779742412583
2010-07-19 05:00:00,20.9,20.9,0.0,0.041630779742412583
2010-07-19 06:00:00,41.0,41.0,0.0,0.041630779742412583
2010-07-19 07:00:00,37.1,37.1,0.0,0.041630779742412583
2010-07-19 08:00:00,41.1,41.1,0.0,0.041630779742412583
2010-07-19 09:00:00,42.0,5.032231094882482,0.0,0.041630779742412583
2010-07-19 10:00:00,50.1,5.032231094882482,0.0,0.041630779742412583
2010-07-19 11:00:00,42.4,39.96070894515547,0.0,0.041630779742412583
2010-07-19 12:00:00,40.1,40.1,0.0,0.041630779742412583
2010-07-19 13:00:00,38.6,5.032231094882482,0.0,0.041630779742412583
2010-07-19 14:00:00,38.6,38.6,0.0,0.041630779742412583
2010-07-19 15:00:00,38.7,5.032231094882482,0.0,0.041630779742412583
2010-07-19 16:00:00,39.1,5.032231094882482,0.0,0.041630779742412583
2010-07-19 17:00:00,41.9,5.032231094882482,0.0,0.041630779742412583
2010-07-19 18:00:00,45.9,45.9,0.0,0.041630779742412583
2010-07-19 19:00:00,45.1,45.1,0.0,0.041630779742412583
2010-07-19 20:00:00,39.8,5.032231094882482,0.0,0.041630779742412583

I don’t exactly know when this behavior started, I just found out about it when I tried to run my own anomaly models and they were coming up with zero. I tried the example, and came up with the results above.

4

And that output was generated before you ever ran remove_tuesdays.py?

5

Yes.

6

Ok, there is definitely something wrong here. The raw anomaly score should not be 0 like that. And you are running your own data through or the example data?

7

I’m running the example data.

8

To help me troubleshoot, run this command from your shell, which should print out what versions of NuPIC you are using:

python -c 'import pkg_resources;n=pkg_resources.get_distribution("nupic");b=pkg_resources.get_distribution("nupic.bindings");print n.project_name, n.version;print b.project_name, b.version'

For example, on my system this prints out:

nupic 0.5.8.dev0
nupic.bindings 0.4.16
9

Here’s what my system prints out:

nupic 0.5.8.dev0
nupic.bindings 0.4.17.dev0

10

I have the same problem. Any fix yet…?

11

I’ve decided to move to a more stable version of NuPIC until I can look into it more.

I"m now working in nupic 0.5.7, nupic.bindings 0.4.13 and the anomaly scores work.

12

Still using NuPIC on DockerHub? I see only two version the latest and master (very old). Or did you created your own Docker image?

13

Yes, still using NuPIC on docker. I uninstalled NuPIC and reinstalled it via pip. That gave me the stable version.

14

Indeed, it works fine. Modifying and committing is an easy way to keep the image with the stable version. Thanks.

15

@Viridenox Thanks for this report, and @Inton thanks for confirming. I have created:

https://github.com/numenta/nupic/issues/3435

Please continue discussion there if you have Github accounts.