I’ve been experimenting on testing some new algorithms on the NAB. Like NuPIC, these algorithms can produce “raw” anomaly scores, but deriving a likelihood score based on the characteristics of each dataset (as the NuPIC detector does) improves the score significantly.
My suggestion for new algorithms (not integrated into the NAB detector namespace) is a new workflow:
- Generate a CSV file of
raw_score, store in
- Run a new option
python run.py -d <detector> --likelihood --score --normalizewhich fills in the
anomaly_scoreusing NuPIC’s AnomalyLikelihood class
- Scoring proceeds as before
This would serve to separate the contributions of the algorithms generating raw scores from the method used to threshold them. It might, if it changes the relative performance of competing algos, give appropriate attention to the idea of using the AnomalyLikelihood process to augment anomaly-detection systems.