Anomaly Likelihood seems wrong

Hello =)

I am doing some basic anomaly detection. But some things are going wrong here.
So first of all: I’m loading the model params with
params = getScalarMetricWithTimeOfDayAnomalyParams(data[predicted_filed_name])

So that should be right.
Then the mystery starts :sweat_smile: I creat a model with this model params

def create_model(model_params, predicted_field_name):
    model = ModelFactory.create(model_params["modelConfig"])
        "predictedField": predicted_field_name
    return model

Then I’m running the my code – quite similar to the prediction. But it is strange… I do not get predictions when running, even thou this is in the hot-gym example like this.

prediction = result.inferences["multiStepBestPredictions"][1]

Exect this everything seems fine: I do get anomaly scores and a likelihood, calculated like this:
anomaly_score = result.inferences["anomalyScore"]
anomaly_likelihood_value = anomalyLikelihoodHelper.anomalyProbability(consumption, anomaly_score, timestamp=None)

But what is really strange is my plot:

The anomaly likelihood increases although the anomaly_score is 0. What is happening here?? :joy:

(ADD: on the top of the plot is the consumption data for which I wanna find the anomalies :blush:)

That doesn’t bother me too much. The only time I would pay attention to the likelihood value is when it is very high (over 0.999 at least).

The anomaly likelihood measures the degree of recent change in anomaly scores. So if the anomaly scores have been generally high over a longer time window and suddenly become low in a short recent window, that will spike the anomaly likelihood – same as if there scores were usually low and then jumped up.

To get predictions you need to use the ‘TemporalMultistep’ model-type. When you generate params with

you get a ‘TemporalAnomaly’ model – which has no classifier and thus yields no predictions – only anomaly scores.

1 Like

Thanks for your fast replies =)

I see, but when I am using TemporalMultistep I’m not getting anomalie scores… at least in my last problem this was the case… and as far as I know, in the hot gym example there is also TemporalAnomaly.

And besides this. @rhyolight you are saying in the anomaly tutorial that the model will still give us predictions when using TemporalAnomaly.
Did the code base change since that time?

ok cool =) goooood to know :yum: Thanks a lot!

I could be wrong! Sorry! I honestly thought I remembered plotting both predictions and anomaly score on the same plot in the past.

haha okay :grin: … well yes in this tutorial you plot them both… but maybe the predictions where there before.
Anyway… Now I know that I wasn’t that stupid and everything works fine!! This is really nice to know :yum:

Thanks a lot :blush:

This seems very odd–wouldn’t it make more sense that a sudden decrease in anomaly score be correlated with a sudden increase in certainty that it’s not an anomaly, rather than the reverse?

Certainly a reasonable intuition, this is a big difference between anomaly score and likelihood. The anomaly score only measures how predicted the current input is at one time step, while the anomaly likelihood looks for shifts in the distribution of recent anomaly scores.

Consider the situation where the input signal switches from noisy to predictable. Here the anomaly scores would go from generally high to generally low, so the anomaly score distribution would shift and this would be an anomaly (according to the likelihood) even though the systems is getting more predictable.

This does make sense to me though, because if noise is the norm we don’t want to see more noise as anomalous. At that time we can expect unpredictability, and would thus be more surprised by a change to predictability than more noise.

And in the situation where the signal switches from predictable to noisy that would also cause a shift in anomaly score distribution and thus a jump in likelihood. So when the signal is more predictable the likelihood acts more like the anomaly score, and diverges in behavior under noise. So the likelihood works like a normalizer for noise in that way.

1 Like

That’s a good explanation. Thanks!

1 Like