How to deal with big data in nupic

Engineering NuPIC
1

I adapted nyctaxi_anomaly.py from the example to process my data from sensors.

I have processed data with frequency of 1 Hz (86400 data points a day), and it worked well. The running time is about 600 seconds each time.

But when I try 50 Hz and 200 Hz data, it takes too long time.

So I ask you for help on how to deal with this problem.

Thanks！

2 Likes
2

There are some tactics you could look into here: TemporalMemory running very slow after long training

1 Like
3

I have opened an issue a short while ago on HTM.core. They are now looking into the problem. Temporal Memory extreamly slow when benchmarking with random inputs · Issue #745 · htm-community/htm.core · GitHub

If c++ is an option for you. You can also use my HTM library. GitHub - etaler/Etaler: A flexable HTM (Hierarchical Temporal Memory) framework with full GPU support.

1 Like