HTM for IT

Engineering Applications
#44

Hi,

So you are facing the same problem as one of my college is dealing. (He uses Etaler not HTM.core, but the problem is same.)

The simplest solution is to develop a server-client architecture; speaking HTTP/HTTPS, WebSockets, MQTT, IOTA or what not. Where your HTM app is the server and the the IoT device is the client. The client sends sensory data periodically to the server and the server performs anomaly detection upon receiving the data.

This should solve your needs. It should handle tens of devices sending data every minute (quite a high rate) should not be a problem.

2 Likes
#45

Actually you can use with anything. The example I wrote used the OPF, but you could do the same thing with the algorithms API. InfluxHTM is agnostic to the HTM backend. However, I am not sure it works with the latest version of InfluxDB. This was a long time ago.

This is a typical problem when you start building HTM systems. I’ve set up a client / server many times. Here is some helpful code for an HTM HTTP server. It uses an old version of NuPIC, but you could adapt the code to work for you.

3 Likes
#46

thank you @marty1885 @rhyolight, i think using web server and using webservice to push data from client to server is the only solution.but it will take time because i am not good at web programing.

I will try to monitoring and do prediction/anomaly detection first.

I am going to design a real system with architecture below:

Servers/PCs ( with installed telegraf agent) --> influxdb ( measurement 1) —> python influxdbclient ( query ) --> HTM Algorithm ( do preditction and anomaly detect) —> python influxdbclient ( insert) --> influxDB ( measurement 2) --> Grafana ( to monitoring )

any advice ?

Thank you in advance

1 Like
#47

Sounds good! Well, I can’t tell much since I’m also bad at web programming. But this is very close to what we do.

1 Like
#48

me too. So i choose all opensource like telegraf+influxdb+grafana to solve my disavantage ( web programming).

#49

My advice is to follow the patterns I created in InfluxHTM, where you write temporal streams into the system and can run parallel analysis alongside it at any time. I found this to be useful, because I could start up a new model and generate more predictive data many times alongside the actual data. Influx made it easy to parse through multiple of these predictive streams at once all while comparing to ground truth. If you structure this backend well, it will be easier to create the frontend.

I have a video of using this somewhere…

1 Like
#50

thank you . i wil study this.

Btw, i have question about SP and TM. When you run SP, the synape permanences of minicolumns will be learn and update. and TM also update cell synape permanences…

So how can i save these synape status or status of SP+TM to hard disk or database in general ? If not, my laptop must alway turn on to keep these status of SP, TM ( variables in RAM) ??

I have read the serialization import capnp, but it run error and It is just for SP, not for TM.

Any advice ?

Thank you very much.

2 Likes
#51

anyone help ? thank you very much

#52

Have you read the Serialization Guide?

#53

thank you @rhyolight. I have read the link as i typed above "I have read the serialization import capnp, but it run error and It is just for SP, not for TM.

with your advice , i rechecked in nupic code and see in the TM instance also has the tm.write or tm.writetofile function ( in the serialization document just tell about SP) .

I also faced the error when run serialization on nupic python 2.7 on window laptop. I searched in this forum and found some similar cases. I will try to fix this.

Thank you very much.

2 Likes
#54

thank you @rhyolight, your idea is great. Although influxHTM you write for OPF ( I dont have time yet to study OPF, i just study Algorithms) , but your idea is so great. I have applied successfully influxDB with HTM algorithms ( with my architecture above). Now i have a realtime data prection & anomaly detection just only one my window 10 laptop ( combine with some opensources software like telegraf, grafana…)

My idea and my code is normal . I am not confident to publish to everyone. But if anyone like, i will send to this forum as my little contribution.

thank you

2 Likes
#55

Yes if you’re willing to share go for it!

3 Likes
#56

Ok @sheiser1, i will correct my code and send asap. ( now my code is very messy)

1 Like
#57

10 posts were split to a new topic: SP / TM connectivity questions

#58

I have a question about the prediction ( using SDR Classifier). In NUPIC Algorithm quickstart they wrote:

Execute Temporal Memory algorithm over active mini-columns.

  tm.compute(activeColumnIndices, learn=True)

activeCells = tm.getActiveCells()

  # Get the bucket info for this input value for classification.
  bucketIdx = scalarEncoder.getBucketIndices(consumption)[0]

Run classifier to translate active cells back to scalar value.

  classifierResult = classifier.compute(
    recordNum=count,
    patternNZ=activeCells,
    classification={
      "bucketIdx": bucketIdx,
      "actValue": consumption
    },
    learn=True,
    infer=True
  )

I doubt this code " patternNZ=activeCells" . Why don’t use " patternNZ= tm.getPredictiveCells() " .

Because we want to prediction so we have to do on Predictive cells ??

Thank you very much.

#59

i found this link CLA Classifier ( thank you @rhyolight) . I think this is guide about how the classifier can do the prediction. Does any one have a more intituive document ( with more examples )?

thank you very much

#60

Thanks, I just updated that document with a better link to the SDRClassifier docs. You should be using this one, not the CLA Classifier.

1 Like
#61

Thank you very much

this link look like just show how to do mapping from TM to bucket. It seem not explain ( in theory) about how to predict t+1, t+3, T+5 as your last link.

Is it right ?

My problem is: i can follow http://nupic.docs.numenta.org/stable/api/algorithms/classifiers.html#module-nupic.algorithms.sdr_classifier and do the prediction in application but i don’t understand the theory how to do that.

#62

The theory of the SDR Classifier is not a brain theory, it is a very simple MLP (Multi Layer Perceptron) network. We don’t have a proper tutorial for it because there are lots of them out there for machine learning stuff like this. I focus mainly on explaining the brain theory.

1 Like
#63

thank you very much @rhyolight . i will study MLP to understand more