How to deal with big data in nupic

I adapted nyctaxi_anomaly.py from the example to process my data from sensors.

I have processed data with frequency of 1 Hz (86400 data points a day), and it worked well. The running time is about 600 seconds each time.

But when I try 50 Hz and 200 Hz data, it takes too long time.

So I ask you for help on how to deal with this problem.

Thanks!

2 Likes

There are some tactics you could look into here: TemporalMemory running very slow after long training

1 Like

I have opened an issue a short while ago on HTM.core. They are now looking into the problem. https://github.com/htm-community/htm.core/issues/745

If c++ is an option for you. You can also use my HTM library. https://github.com/etaler/Etaler

1 Like