Serializing HTMPredictionModel for anomaly detection with cap'n proto


#1

I searched the forum for this topic before posting but couldn’t quite find what I was looking for. I am using the model factory to create opf.htm_prediction_model.HTMPredictionModel, I have made some predictions and I would like to serialize the model.

I found nupic’s prototypes for serializing different components here but I don’t see any for the models? Should we not use the OPF if we would like to do this? Perhaps we need to serialize smaller components instead? It’s a little confusing because the model factory doesn’t require you to build any of those more detailed data structures to use it.


#2

There was discussion on this here:

Basically, OPF is deprecated and you are better off using the Network API if you want to serialize with capnproto.


#3

What about the algorithms API?


#4

With the algorithms API you can more easily save the state of the SP and TM. See here.


#5

@rhyolight

Under the hood the algorithms api is manipulating the spatial and temporal pooler? Problem is that I just want to serialize the Anomalylikelihood model, not the underlying parts as I don’t create those manually myself (like in the serialization example)


#6

See details about the algorithms API here. It is the SP and TM. There is no AnomalyLikelihood model when you use it. That is a part of the OPF. See a quickstart for each of the 3 APIs here.


#7

@jacobeverist

I got the HTMPredictionModel to serialize, but a single model is 2.9 MB (20 samples). I got the AnomalyLikelihood model to serialize too (1000 samples) and it was 26KB.


#8

How did you do it? Did you just call model.save() on it? Or did you pull out the algorithms and serialize them with capnp?


#9

HTMPredictionModel has write/read methods that work with capnproto:

htm_prediction_model.py (line 1321)

right above it at line 1317 there is a method getSchema(), which returns the correct capnproto schema file


#10

That’s great, I had no idea.


#11

Thanks for finding this. I will try it out when I get a chance this week.