# Recoding htm.core's hotgym.py (I'm making many errors)

**URL:** <https://discourse.numenta.org/t/recoding-htm-cores-hotgym-py-im-making-many-errors/7767>\
**Category:** NuPIC Community Fork\
**Tags:** sequence-memory, spatial-pooling, question\
**Created:** [July 30, 2020, 8:31pm UTC](https://discourse.numenta.org/t/recoding-htm-cores-hotgym-py-im-making-many-errors/7767 "2020-07-30T20:31:02Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![mcleverley](https://avatars.discourse-cdn.com/v4/letter/m/5f9b8f/32.png) [@mcleverley](https://discourse.numenta.org/u/mcleverley)\
**Post date:** [July 31, 2020, 7:34pm UTC](https://discourse.numenta.org/t/recoding-htm-cores-hotgym-py-im-making-many-errors/7767/3 "2020-07-31T19:34:27Z")

</div>

Hey Johan, thanks for the example, lovely stuff. It makes sense that we teach the predictor (which is a neural net used to convert the TM’s final-state SDR into a prediction, i think?) using `.learn()` with some data + a label at each iteration, and `.infer()` can be used to predict outputs from a given input state.

Everything I mention from here out is up on my [github](https://github.com/mcleverley/htm_nlp_predictive) as [‘alternate\_hotgym.ipynb’](https://github.com/mcleverley/htm_nlp_predictive/blob/master/alternate_hotgym.ipynb):

The `hotgym.py` code is a little different, it seems; they call `predictor.infer()` at each training iteration to store the prediction for use in anomaly detection (power consumption spikes etc) later on. This was giving me the aforementioned `"must call learn before infer"` error.

I was able to sidestep this error by adding another altered copy of the training loop immediately before the original loop, where I added a `predictor.learn()` right after the `tm.compute()` and ended it there:

```auto
predictor = Predictor(steps=[1,5], alpha=parameters['predictor']['sdrc_alpha'])
predictor_resolution = 1
inputs = []
anomaly = []
anomalyProb = []
predictions = {1: [], 5:[]}
for count, record in enumerate(records): # iterate through listified CSV
    dateString = datetime.datetime.strptime(record[0], '%m/%d/%y %H:%M')
    consumption = float(record[1])
    inputs.append(consumption)
    # encoders at work
    dateBits = dateEncoder.encode(dateString)
    consumptionBits = scalarEncoder.encode(consumption)
    # concatenate these bad boys for a composite x,y input
    encoding = SDR(encodingWidth).concatenate([consumptionBits, dateBits])
    enc_info.addData(encoding) # keep track in metrics
    # create an SDR for active columns, same dimensions as SP
    activeColumns = SDR(sp.getColumnDimensions())
    # hurl input into the pool
    sp.compute(encoding, True, activeColumns)
    tm_info.addData(tm.getActiveCells().flatten())
    # dunno why we add the TM active state before tm.compute(), but i don't know most things
    tm.compute(activeColumns, learn=True)
    # let's change things up and try to just let the predictor learn before making any predictions
    predictor.learn(count, tm.getActiveCells(), int(round(consumption/predictor_resolution)))

```

The second, original training loop is the same up to `tm.compute()`, where it replaces `predictor.learn()` with `predictor.infer()`; the original `.learn()` is right at the end:

```auto
for [...]:
    pdf = predictor.infer( tm.getActiveCells() )
    for n in (1,5):
        if pdf[n]:
            predictions[n].append( np.argmax( pdf[n] ) * predictor_resolution )
        else:
            predictions[n].append(float('nan'))
    
    anomalyLikelihood = anomaly_history.anomalyProbability( consumption, tm.anomaly )
    anomaly.append(tm.anomaly)
    anomalyProb.append(anomalyLikelihood)
    # original position of learn()
    predictor.learn(count, tm.getActiveCells(), int(consumption/predictor_resolution))

```

This works… well, maybe. I’ve attached the output graph.

 ![download](https://canada1.discourse-cdn.com/flex030/uploads/numenta/original/2X/4/4818b752e881225b0f13e3abe0b8e2b11bebddf3.png)  
Had a bothersome error where it was predicting all 0’s for anomalies, and then I realized I’d somehow deleted the `tm.compute()` from the 2nd original training loop.

There’s also some oddity with the prediction-shift loop:

```auto
# Shift the predictions so that they are aligned with the input they predict.
for n_steps, pred_list in predictions.items():
    for x in range(n_steps):
        pred_list.insert(0, float('nan'))
        pred_list.pop()

```

This inserts some 0’s and nans into the predictions and later crashes the accuracy\_score calculation with a divide by 0:

```auto
acc_lists = {1: [], 5: []} # i added this one for graphing
# calculate predictive accuracy, RMS
accuracy = {1: 0, 5: 0}
accuracy_samples = {1: 0, 5: 0}
for idx, inp in enumerate(inputs):
    for n in predictions: # for each [N]umber of timesteps ahead which was predicted
        val = predictions[n][idx]
        if not math.isnan(val):
            accuracy[n] += (inp - val) ** 2 # RMSE
            accuracy_samples[n] += 1
    for n in sorted(predictions):
# print("accuracy[n]: ",accuracy[n], " accuracy_samples[n]: ",accuracy_samples[n])
        accuracy[n] = (accuracy[n] / accuracy_samples[n]) **.5
# print("Predictive Error (root-mean-squared): ", n, "steps ahead:", accuracy[n])
# print()
        acc_lists[n].append(accuracy[n]) # add to lists
        
    print('Anomaly Mean: ', np.mean(anomaly))
    print('Anomaly Std: ', np.std(anomaly))

```

The div0 seems to happen near the end, with `accuracy[n] = (accuracy[n] / accuracy_samples[n]) **.5`.  
I tried skipping this loop entirely in the run, and it… kinda looks like it works. Dunno.

I’m not sure of the ramifications of my two workarounds; perhaps the added pre-training loop induces overfitting, perhaps the lack of prediction-shift means the predictions are misaligned and useless? Not sure.

If anyone’s got some feedback or ideas on how to improve this, please throw them my way. Looking through NuPIC’s docs, what with modelFactory and such, it feels like Keras to the htm.core’s PyTorch - though I certainly can’t complain about learning the more granular workings of HTM code.

---

_[View the full topic](https://discourse.numenta.org/t/recoding-htm-cores-hotgym-py-im-making-many-errors/7767)._