HTM for IT

Engineering Applications
41

No, I’m just informing you about the implementation details; in case you are tracing the code yourself.

Whatever fits you is good enough for you! However I recommend to read BAMI after you gained a basic understanding of the algorithms.

1 Like
42

thank you very much. Actually, BAMI is the second documment i have read. The first one is “Unsupervised real-time anomaly detection for streaming data 2017”

43

Dear @marty1885 @sheiser1 @rhyolight

I make HTM work with datafile to make prediction or anomaly detect ( i Use algorithms not OPF) . Now i want to use HTM to work ( prediction+anomaly detection) with realtime streaming data source ( from IOT devices…) instead of static data file.

I checked and found on this forum https://github.com/htm-community/influx.htm . But this is for OPF not for Algorithms (http://nupic.docs.numenta.org/1.0.3/quick-start/algorithms.html)

Do you have any advice for me ?

Thank you very much.

44

Hi,

So you are facing the same problem as one of my college is dealing. (He uses Etaler not HTM.core, but the problem is same.)

The simplest solution is to develop a server-client architecture; speaking HTTP/HTTPS, WebSockets, MQTT, IOTA or what not. Where your HTM app is the server and the the IoT device is the client. The client sends sensory data periodically to the server and the server performs anomaly detection upon receiving the data.

This should solve your needs. It should handle tens of devices sending data every minute (quite a high rate) should not be a problem.

2 Likes
45

Actually you can use with anything. The example I wrote used the OPF, but you could do the same thing with the algorithms API. InfluxHTM is agnostic to the HTM backend. However, I am not sure it works with the latest version of InfluxDB. This was a long time ago.

This is a typical problem when you start building HTM systems. I’ve set up a client / server many times. Here is some helpful code for an HTM HTTP server. It uses an old version of NuPIC, but you could adapt the code to work for you.

3 Likes
46

thank you @marty1885 @rhyolight, i think using web server and using webservice to push data from client to server is the only solution.but it will take time because i am not good at web programing.

I will try to monitoring and do prediction/anomaly detection first.

I am going to design a real system with architecture below:

Servers/PCs ( with installed telegraf agent) --> influxdb ( measurement 1) —> python influxdbclient ( query ) --> HTM Algorithm ( do preditction and anomaly detect) —> python influxdbclient ( insert) --> influxDB ( measurement 2) --> Grafana ( to monitoring )

any advice ?

Thank you in advance

1 Like
47

Sounds good! Well, I can’t tell much since I’m also bad at web programming. But this is very close to what we do.

1 Like
48

me too. So i choose all opensource like telegraf+influxdb+grafana to solve my disavantage ( web programming).

49

My advice is to follow the patterns I created in InfluxHTM, where you write temporal streams into the system and can run parallel analysis alongside it at any time. I found this to be useful, because I could start up a new model and generate more predictive data many times alongside the actual data. Influx made it easy to parse through multiple of these predictive streams at once all while comparing to ground truth. If you structure this backend well, it will be easier to create the frontend.

I have a video of using this somewhere…

1 Like
50

thank you . i wil study this.

Btw, i have question about SP and TM. When you run SP, the synape permanences of minicolumns will be learn and update. and TM also update cell synape permanences…

So how can i save these synape status or status of SP+TM to hard disk or database in general ? If not, my laptop must alway turn on to keep these status of SP, TM ( variables in RAM) ??

I have read the serialization import capnp, but it run error and It is just for SP, not for TM.

Any advice ?

Thank you very much.

2 Likes
51

anyone help ? thank you very much

52

Have you read the Serialization Guide?

53

thank you @rhyolight. I have read the link as i typed above "I have read the serialization import capnp, but it run error and It is just for SP, not for TM.

with your advice , i rechecked in nupic code and see in the TM instance also has the tm.write or tm.writetofile function ( in the serialization document just tell about SP) .

I also faced the error when run serialization on nupic python 2.7 on window laptop. I searched in this forum and found some similar cases. I will try to fix this.

Thank you very much.

2 Likes
54

thank you @rhyolight, your idea is great. Although influxHTM you write for OPF ( I dont have time yet to study OPF, i just study Algorithms) , but your idea is so great. I have applied successfully influxDB with HTM algorithms ( with my architecture above). Now i have a realtime data prection & anomaly detection just only one my window 10 laptop ( combine with some opensources software like telegraf, grafana…)

My idea and my code is normal . I am not confident to publish to everyone. But if anyone like, i will send to this forum as my little contribution.

thank you

2 Likes
55

Yes if you’re willing to share go for it!

3 Likes
56

Ok @sheiser1, i will correct my code and send asap. ( now my code is very messy)

1 Like
57

10 posts were split to a new topic: SP / TM connectivity questions

58

I have a question about the prediction ( using SDR Classifier). In NUPIC Algorithm quickstart they wrote:

Execute Temporal Memory algorithm over active mini-columns.

  tm.compute(activeColumnIndices, learn=True)

activeCells = tm.getActiveCells()

  # Get the bucket info for this input value for classification.
  bucketIdx = scalarEncoder.getBucketIndices(consumption)[0]

Run classifier to translate active cells back to scalar value.

  classifierResult = classifier.compute(
    recordNum=count,
    patternNZ=activeCells,
    classification={
      "bucketIdx": bucketIdx,
      "actValue": consumption
    },
    learn=True,
    infer=True
  )

I doubt this code " patternNZ=activeCells" . Why don’t use " patternNZ= tm.getPredictiveCells() " .

Because we want to prediction so we have to do on Predictive cells ??

Thank you very much.

59

i found this link CLA Classifier ( thank you @rhyolight) . I think this is guide about how the classifier can do the prediction. Does any one have a more intituive document ( with more examples )?

thank you very much

60

Thanks, I just updated that document with a better link to the SDRClassifier docs. You should be using this one, not the CLA Classifier.

1 Like