Minimun set for anomaly detection

Hi,
I’m trying to perform most simple anomaly detection on time-series data.
My data format is something like:

timestamp dataValue

Example:
1498659225838 -8.6113
1498659225839 -8.9027

Looking at HotGym demo and some other topic in the forum I decided to use this code:

Parameters p = Utils.getParameters();
        p = p.union(Utils.getEncoderParams());

    return Network.create("Network API Demo", p)
            .add(Network.createRegion("Region 1")
                    .add(Network.createLayer("Layer 2/3", p)
                            .add(Anomaly.create())
                            .add(new TemporalMemory())
                            .add(new SpatialPooler());

But honestly I can’t continue because I don’t understand which parameters I need and how I have to set them (is there a minimum essential set of standard parameters to use?) and moreover how to pass my data series to the network.

Hello guys,
playing with the code I have obtained an output for:
getAnomalyScore();
function.

I have used the network found in HTM test… It would be great if someone can explain me how to fit the network params to my dataset.

I noted that the output initially is composed by a lot of ‘1’ and decrease to 0 while computation move forward. As I read on this paper ‘1’ value means totally unpredicted and ‘0’ completely predicted right? I explained myself the behaviour due to the time needed by HTM to perform predictions on dataset, is this concept correct?

Can someone give me a feedback and hint about this simple implementation of anomaly detection? Is formally correct? How can I understand how to set network parameters?

In order to better clarify I attach used code:
driver class
util class

1 Like

Hi @Andrea_Giordano,

The way this has been working in the past is that if users have specific questions about HTM.Java and specific code handling issues - then I usually respond as best I as I can for those issues. Your questions about parameter handling are general, so they would get “better” attention if you ask them in the NuPIC category instead of here because I’m not as experienced with general NuPIC usage - and NuPIC and HTM.Java are functionally pretty much identical so any questions regarding setting of parameters and general functionality - those answers are directly translatable and applicable to HTM.Java. So I would try over there…

Parameter handling in general is pretty much a “dark art”. It helps to read the parameter explanations in the code to understand what they mean. The code documentation is pretty good - and it would be good to “sit” with it and try to understand as much as possible - what their affect is. Additionally for “deeper” discussions, you can always ask in the NuPIC forum and do searches of the forum for previous discussions because there have been a lot of them.

The network parameters for Anomaly Detection have been pretty much settled on, meaning that Numenta engineers arrived at the conclusion that a certain basic set of parameters works for over 80% of use cases roughly - so you can ask over at the NuPIC forum where those are kept. I think for your use case they should work?? (@rhyolight ??)

As far as this:

From what I can tell, your estimation is correct in that if your data repeats the same values over and over, eventually anomaly detection should go pretty much to “0”. You could try inserting random values at certain points to confirm that anomaly detection is taking place after this settling period has occurred? That might be informative?

Anyway, thanks for using HTM.Java and I’ll see you around the forums!

David

1 Like

thank you cogmission. I tried to insert a random value between two fully predicted value and effectively the anomaly score raise from 0 to 0.35. I expected a major raise due to the extreme value I put but I think it is ok.

1 Like

Moved from #htm-java to #nupic.