I am doing some basic anomaly detection. But some things are going wrong here.
So first of all: I’m loading the model params with params = getScalarMetricWithTimeOfDayAnomalyParams(data[predicted_filed_name])
So that should be right.
Then the mystery starts I creat a model with this model params
def create_model(model_params, predicted_field_name):
model = ModelFactory.create(model_params["modelConfig"])
model.enableInference({
"predictedField": predicted_field_name
})
model.enableInference(model_params["inferenceArgs"])
return model
Then I’m running the my code – quite similar to the prediction. But it is strange… I do not get predictions when running, even thou this is in the hot-gym example like this.
Exect this everything seems fine: I do get anomaly scores and a likelihood, calculated like this: anomaly_score = result.inferences["anomalyScore"] anomaly_likelihood_value = anomalyLikelihoodHelper.anomalyProbability(consumption, anomaly_score, timestamp=None)
The anomaly likelihood measures the degree of recent change in anomaly scores. So if the anomaly scores have been generally high over a longer time window and suddenly become low in a short recent window, that will spike the anomaly likelihood – same as if there scores were usually low and then jumped up.
To get predictions you need to use the ‘TemporalMultistep’ model-type. When you generate params with
you get a ‘TemporalAnomaly’ model – which has no classifier and thus yields no predictions – only anomaly scores.
I see, but when I am using TemporalMultistep I’m not getting anomalie scores… at least in my last problem this was the case… and as far as I know, in the hot gym example there is also TemporalAnomaly.
And besides this. @rhyolight you are saying in the anomaly tutorial that the model will still give us predictions when using TemporalAnomaly.
Did the code base change since that time?
haha okay … well yes in this tutorial you plot them both… but maybe the predictions where there before.
Anyway… Now I know that I wasn’t that stupid and everything works fine!! This is really nice to know
This seems very odd–wouldn’t it make more sense that a sudden decrease in anomaly score be correlated with a sudden increase in certainty that it’s not an anomaly, rather than the reverse?
Certainly a reasonable intuition, this is a big difference between anomaly score and likelihood. The anomaly score only measures how predicted the current input is at one time step, while the anomaly likelihood looks for shifts in the distribution of recent anomaly scores.
Consider the situation where the input signal switches from noisy to predictable. Here the anomaly scores would go from generally high to generally low, so the anomaly score distribution would shift and this would be an anomaly (according to the likelihood) even though the systems is getting more predictable.
This does make sense to me though, because if noise is the norm we don’t want to see more noise as anomalous. At that time we can expect unpredictability, and would thus be more surprised by a change to predictability than more noise.
And in the situation where the signal switches from predictable to noisy that would also cause a shift in anomaly score distribution and thus a jump in likelihood. So when the signal is more predictable the likelihood acts more like the anomaly score, and diverges in behavior under noise. So the likelihood works like a normalizer for noise in that way.
I creat a model with this model params
)
Thanks a lot!
… well yes in this tutorial you plot them both… but maybe the predictions where there before.