# Time scale invariance of anomaly detection

**URL:** <https://discourse.numenta.org/t/time-scale-invariance-of-anomaly-detection/4279>\
**Category:** Engineering\
**Tags:** anomaly-detection, question\
**Created:** [July 30, 2018, 5:19pm UTC](https://discourse.numenta.org/t/time-scale-invariance-of-anomaly-detection/4279 "2018-07-30T17:19:09Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Oblynx](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/oblynx/32/4618_2.png) [@Oblynx](https://discourse.numenta.org/u/Oblynx)\
**Post date:** [July 30, 2018, 5:19pm UTC](https://discourse.numenta.org/t/time-scale-invariance-of-anomaly-detection/4279/1 "2018-07-30T17:19:09Z")

</div>

Let an HTM model do anomaly detection on a scalar timeseries `x(t)`, which doesn’t have very low frequencies, and which we can assume samples a physical phenomenon, like temperature. Let’s also interpolate `x` and make `y(t)= x(t/2)`. `y` therefore has _no extra information_ on the underlying phenomenon (no extra anomalies), but exhibits the same anomalous behaviour as `x`, albeit with many extra uninformative transitions.

Is it reasonable to expect that a sufficiently large HTM model should perform equivalently on predicting anomalies in `x` and in `y`? Is this a desirable property? And what would happen in practice?

I would call this property time scale invariance.

---

<div class="post-metadata">

**Author:** ![SimLeek](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/simleek/32/2298_2.png) [@SimLeek](https://discourse.numenta.org/u/SimLeek)\
**Post date:** [July 30, 2018, 9:11pm UTC](https://discourse.numenta.org/t/time-scale-invariance-of-anomaly-detection/4279/2 "2018-07-30T21:11:08Z")

</div>

The neurons in a temporal memory system make predictive connections to the active neurons in the next timestep. So, while the networks for x and y wouldn’t look the same, they’d make similar predictions. While y’s predictive connections would go from a to c, x’s should go from a to b to c.

However, y’s data stream may be more or less predictable than x’s, so having both could actually be a benefit. This benefit is kind of what we’re getting into in the recent numerical encoders based on biology.

---

<div class="post-metadata">

**Author:** ![Oblynx](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/oblynx/32/4618_2.png) [@Oblynx](https://discourse.numenta.org/u/Oblynx)\
**Post date:** [July 31, 2018, 8:15am UTC](https://discourse.numenta.org/t/time-scale-invariance-of-anomaly-detection/4279/3 "2018-07-31T08:15:54Z")

</div>

Thanks.

> [@SimLeek](#):
>
> y’s predictive connections would go from a to c, x’s should go from a to b to c.

You mean the opposite, `x: a->c`, `y: a->b->c`

I would actually expect an efficient learning algorithm to get maximum information out of x, and gain nothing from y.

> [@SimLeek](#):
>
> recent numerical encoders based on biology.

Can you point me to them?

---

<div class="post-metadata">

**Author:** ![SimLeek](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/simleek/32/2298_2.png) [@SimLeek](https://discourse.numenta.org/u/SimLeek)\
**Post date:** [July 31, 2018, 3:25pm UTC](https://discourse.numenta.org/t/time-scale-invariance-of-anomaly-detection/4279/4 "2018-07-31T15:25:27Z")

</div>

> [@Oblynx](#):
>
> You mean the opposite, `x: a->c` , `y: a->b->c`

Err, yup. My bad.

But actually, useful information can be found in both x and y. It’s like seeing the forest vs. the trees.

> [@Oblynx](#):
>
> Can you point me to them?

Sure thing. Look at the bottom [here].([Encoders created by the community](https://discourse.numenta.org/t/encoders-created-by-the-community/4251)) Also, look at the more recent HTM-school videos for grid cell stuff.

(I really need to wait until I’m off mobile for this stuff.)

---

<div class="post-metadata">

**Author:** ![Oblynx](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/oblynx/32/4618_2.png) [@Oblynx](https://discourse.numenta.org/u/Oblynx)\
**Post date:** [August 1, 2018, 9:07am UTC](https://discourse.numenta.org/t/time-scale-invariance-of-anomaly-detection/4279/5 "2018-08-01T09:07:15Z")

</div>

> [@SimLeek](#):
>
> But actually, useful information can be found in both x and y. It’s like seeing the forest vs. the trees.
> 
> ![](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/oblynx/40/1050_1.png) Oblynx:

Not really I believe. So as I assumed in the beginning, `x` is the discrete-time samples of a continuous-time natural phenomenon, let’s call it `p`. Then `y` is created by assuming a (eg polynomial) interpolation of `x` and drawing out “extra samples” magically. Of course, `y` has nothing new to tell us about the underlying physical phenomenon `p`, and no new anomalies; any anomaly should come directly from `p`.

If, for example, `p` has an unexpected sharp peak (anomalous) for a Δt \<\< Δt\_sampling, then most likely it will not be reflected at all in `x`, neither in `y`.

Seeing the forest vs the trees would imply sampling `y` directly from `p` with a higher sampling rate and focusing on a smaller time window.

---

<div class="post-metadata">

**Author:** ![SimLeek](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/simleek/32/2298_2.png) [@SimLeek](https://discourse.numenta.org/u/SimLeek)\
**Post date:** [August 2, 2018, 12:05am UTC](https://discourse.numenta.org/t/time-scale-invariance-of-anomaly-detection/4279/6 "2018-08-02T00:05:20Z")

</div>

> [@Oblynx](#):
>
> Then `y` is created by assuming a (eg polynomial) interpolation of `x` and drawing out “extra samples”

Ah, right. In that case, part of the temporal memory would be wasted on predicting the interpolation, and would be a little bit less useful in predicting x. I’m not sure exactly how it’d compare mathematically though.

Theoretically, it’d be interesting to go into the math. Practically though, I wouldn’t recommend interpolation with temporal memory.

Going back to your questions in your first post, I’d say that yes, a sufficiently large HTM model should perform approximately equivalently. I’d say the performance would be asymptotically equivalent when increasing size. While it’s a desirable quality to predict further in the future, I would say interpolating the data would be similar to adding noise, which HTM can handle, but other than for testing, it’s never really a good thing.

In practice, they should predict the same anomalies at the same times, if the time scales are accounted for correctly. If there’s a short spike in the data, the model with interpolation should predict interpolating back down from the spike very well.