How much should I trust swarm results?

sheiser1 · February 20, 2019, 5:50pm

So I’m running swarms on mostly 5-10 column data sets with one as the ‘predicted field’. Each time it chooses adaptive scalar encoders, and this time not including the predicted field ‘device_temp_c’.

('_classifierInput', ' -- ', {'classifierOnly': True, 'name': '_classifierInput', 'clipInput': True, 'n': 521, 'fieldname': 'device_temp_c', 'w': 21, 'type': 'AdaptiveScalarEncoder'})
(u'libversion', ' -- ', {'name': 'libversion', 'clipInput': True, 'n': 449, 'fieldname': 'libversion', 'w': 21, 'type': 'AdaptiveScalarEncoder'})
(u'write_amp', ' -- ', {'name': 'write_amp', 'clipInput': True, 'n': 370, 'fieldname': 'write_amp', 'w': 21, 'type': 'AdaptiveScalarEncoder'})
(u'device_temp_c', ' -- ', None)

When I set the predicted field to another aging-related metric, the one field that the swarm picks up is ‘device_temp_c’. This is the one variable that has fluctuating, kind of periodic behavior, so it makes sense that the swarm found it useful. The encoder it picked was an Adaptive Scalar with the w = n-1, which seems like.a sign of trouble. I though it would’ve picked a Scalar or RDSE, which makes me suspect a problem somewhere in the encoding process.

 'sensorParams': {'encoders': {'_classifierInput': {'classifierOnly': True,
                                                    'clipInput': True,
                                                    'fieldname': 'media_wear_pct',
                                                    'n': 28,
                                                    'name': '_classifierInput',
                                                    'type': 'AdaptiveScalarEncoder',
                                                    'w': 21},
                               u'device_temp_c': {'clipInput': True,
                                                  'fieldname': 'device_temp_c',
                                                  'n': 22,
                                                  'name': 'device_temp_c',
                                                  'type': 'AdaptiveScalarEncoder',
                                                  'w': 21},
                               u'libversion': None,
                               u'media_wear_pct': None,
                               u'media_written_gb': None,
                               u'write_amp': None},

Does this seem troublesome?

rhyolight · February 20, 2019, 7:27pm

Swarming never tries the RDSE, because we wrote way after swarming code was considered legacy. You can safely change the AdaptiveScalar encoders to RDSE encoders.

sheiser1 · February 21, 2019, 2:11am

Ok great, thanks, so if the current params are:

 'sensorParams': {'encoders': {'_classifierInput': {'classifierOnly': True,
                                                    'clipInput': True,
                                                    'fieldname': 'device_temp_c',
                                                    'n': 164,
                                                    'name': '_classifierInput',
                                                    'type': 'AdaptiveScalarEncoder',
                                                    'w': 21},
                               u'device_temp_c': {'clipInput': True,
                                                  'fieldname': 'device_temp_c',
                                                  'n': 121,
                                                  'name': 'device_temp_c',
                                                  'type': 'AdaptiveScalarEncoder',
                                                  'w': 21},

Would you condone swapping out for ‘ScalarEncoder’ and using the same ‘minval’ and ‘maxval’ for each? When I did I got this result. For some reason all the predictions are shifted down from the real (‘Target’) values. Any intuition on why this may be? I also tried setting the ‘minval’ & ‘maxvals’ the same in the ‘_classifierInput’ as the input field ‘device_temp_c’.

Pred_v_TARGET%20--%20SCALAR Pred_v_TARGET – SCALAR.png

sheiser1 · February 21, 2019, 2:54am

Also when I used an RDSE the predictions were shifted way further away! For the RSDE I took the ‘numBuckets’ for the ‘resolution’ as ‘n’/‘w’ (as given for the AdaptiveScalarEncoder by the swarm). Any intuitions what may be wrong here?

Pred_v_TARGET%20--%20RDSE

rhyolight · February 21, 2019, 2:55pm

Paste your whole model params please.

sheiser1 · February 22, 2019, 6:59am

Ok, so here is the Params file using the swarm-generated params, and the plot of predicted field and the predictions of the Nupic model (all predictions were either 0 or 1):

MODEL_PARAMS = {'aggregationInfo': {'days': 0,
                     'fields': [],
                     'hours': 0,
                     'microseconds': 0,
                     'milliseconds': 0,
                     'minutes': 0,
                     'months': 0,
                     'seconds': 0,
                     'weeks': 0,
                     'years': 0},
 'model': 'HTMPrediction',
 'modelParams': {'anomalyParams': {u'anomalyCacheRecords': None,
                                   u'autoDetectThreshold': None,
                                   u'autoDetectWaitRecords': None},
                 'clParams': {'alpha': 0.0001,
                              'regionName': 'SDRClassifierRegion',
                              'steps': '1',
                              'verbosity': 0},
                 'inferenceType': 'TemporalMultiStep',
                 'sensorParams': {'encoders': {'_classifierInput': {'classifierOnly': True,
                                                                    'clipInput': True,
                                                                    'fieldname': 'media_wear_pct',
                                                                    'n': 28,
                                                                    'name': '_classifierInput',
                                                                    'type': 'AdaptiveScalarEncoder',
                                                                    'w': 21},
                                               u'device_temp_c': {'clipInput': True,
                                                                  'fieldname': 'device_temp_c',
                                                                  'n': 22,
                                                                  'name': 'device_temp_c',
                                                                  'type': 'AdaptiveScalarEncoder',
                                                                  'w': 21},
                                               u'libversion': None,
                                               u'media_wear_pct': None,
                                               u'write_amp': None},
                                  'sensorAutoReset': None,
                                  'verbosity': 0},
                 'spEnable': True,
                 'spParams': {'boostStrength': 0.0,
                              'columnCount': 2048,
                              'globalInhibition': 1,
                              'inputWidth': 0,
                              'numActiveColumnsPerInhArea': 40,
                              'potentialPct': 0.8,
                              'seed': 1956,
                              'spVerbosity': 0,
                              'spatialImp': 'cpp',
                              'synPermActiveInc': 0.05,
                              'synPermConnected': 0.1,
                              'synPermInactiveDec': 0.1},
                 'tmEnable': True,
                 'tmParams': {'activationThreshold': 12,
                              'cellsPerColumn': 32,
                              'columnCount': 2048,
                              'globalDecay': 0.0,
                              'initialPerm': 0.21,
                              'inputWidth': 2048,
                              'maxAge': 0,
                              'maxSegmentsPerCell': 128,
                              'maxSynapsesPerSegment': 32,
                              'minThreshold': 9,
                              'newSynapseCount': 20,
                              'outputType': 'normal',
                              'pamLength': 1,
                              'permanenceDec': 0.1,
                              'permanenceInc': 0.1,
                              'seed': 1960,
                              'temporalImp': 'cpp',
                              'verbosity': 0},
                 'trainSPNetOnlyIfRequested': False},
 'predictAheadTime': None,
 'version': 1}

Pred_v_TARGET%20--%20Scalar

Given this I tried changing the params to (changes in **):

MODEL_PARAMS = {'aggregationInfo': {'days': 0,
                     'fields': [],
                     'hours': 0,
                     'microseconds': 0,
                     'milliseconds': 0,
                     'minutes': 0,
                     'months': 0,
                     'seconds': 0,
                     'weeks': 0,
                     'years': 0},
 'model': 'HTMPrediction',
 'modelParams': {'anomalyParams': {u'anomalyCacheRecords': None,
                                   u'autoDetectThreshold': None,
                                   u'autoDetectWaitRecords': None},
                 'clParams': {'alpha': 0.0001,
                              'regionName': 'SDRClassifierRegion',
                              'steps': '1',
                              'verbosity': 0},
                 **'inferenceType': 'TemporalAnomaly',**
                 'sensorParams': {'encoders': {'_classifierInput': {'classifierOnly': True,
                                                                    'clipInput': True,
                                                                    'fieldname': 'media_wear_pct',
                                                                    **'n': 127,**
                                                                    **'minval': 324**
                                                                    **'maxval': 1272**
                                                                    'name': '_classifierInput',
                                                                    **'type': 'ScalarEncoder',**
                                                                    'w': 21},
                                               u'device_temp_c': {'clipInput': True,
                                                                  'fieldname': 'device_temp_c',
                                                                  **'minval': 234**
                                                                   **'maxval': 439** 
                                                                  **'n': 127,**
                                                                  'name': 'device_temp_c',
                                                                  **'type': 'ScalarEncoder'**,
                                                                  'w': 21},
                                               u'libversion': None,
                                               u'media_wear_pct': None,
                                               u'write_amp': None},
                                  'sensorAutoReset': None,
                                  'verbosity': 0},
                 'spEnable': True,
                 'spParams': {'boostStrength': 0.0,
                              'columnCount': 2048,
                              'globalInhibition': 1,
                              'inputWidth': 0,
                              'numActiveColumnsPerInhArea': 40,
                              'potentialPct': 0.8,
                              'seed': 1956,
                              'spVerbosity': 0,
                              'spatialImp': 'cpp',
                              'synPermActiveInc': 0.05,
                              'synPermConnected': 0.1,
                              'synPermInactiveDec': 0.1},
                 'tmEnable': True,
                 'tmParams': {'activationThreshold': 12,
                              'cellsPerColumn': 32,
                              'columnCount': 2048,
                              'globalDecay': 0.0,
                              'initialPerm': 0.21,
                              'inputWidth': 2048,
                              'maxAge': 0,
                              'maxSegmentsPerCell': 128,
                              'maxSynapsesPerSegment': 32,
                              'minThreshold': 9,
                              'newSynapseCount': 20,
                              'outputType': 'normal',
                              'pamLength': 1,
                              'permanenceDec': 0.1,
                              'permanenceInc': 0.1,
                              'seed': 1960,
                              'temporalImp': 'cpp',
                              'verbosity': 0},
                 'trainSPNetOnlyIfRequested': False},
 'predictAheadTime': None,
 'version': 1}

This resulted in the following plot:
Pred_v_TARGET%20--%20Scalar

This is an improvement, though it baffles me why the predictions are shifted so low. Any ideas how all the predictions could fall below either of the min/max ranges?

Also here are the predicted and target values on a second field. Sry the labels are messed up, the orange is the target and blue is the predicted value. Same basic phenomena.
Pred_v_TARGET%20--%20Scalar

rhyolight · February 22, 2019, 5:01pm

What is the exact delta between them? It looks something like 110? Find out and see if you can figure out why that delta? It seems to me like the classifier is using the wrong min/max, so double check the model params for the _classifierInput.

This looks like a resolution problem for the RDSE, and it also looks like a similar core issue as described above. Double check that the encoder parameters are the same for the input encoder and the classifier input. Not only do they seem to be off, they might be different encoders entirely!

Those params look bad to me, maybe I’m wrong, but the n is different between the _classifierInput encoder params and the device_temp_c encoder params. Change them to be the same and see what happens.

sheiser1 · February 22, 2019, 7:09pm

Ok I’m re-running the model now with the classifier params set the same as the input params for the predicted field. Here’s the most recent plot and swarm-generated params (with classifier params still different). I did convert these params to use ScalarEncoders instead of AdaptiveScalars, which required increasing the ‘n’ values to at least 6x the ‘w’ values, so the n’s for the non-classifier inputs (‘device_temp_c’ & ‘media_wear_pct’) went from 124 & 84 to 127 each.

I included the AnomScore and Likelihoods in the plot too. The Anomaly Likelihood is totally flat even though the data is over 10,000 rows, which has me confused too. I’ll see if that changes when I adjust the params.
Pred_v_TARGET%20--%20Scalar

----------------------------------------------------------------

MODEL_PARAMS = {'aggregationInfo': {'days': 0,
                     'fields': [],
                     'hours': 0,
                     'microseconds': 0,
                     'milliseconds': 0,
                     'minutes': 0,
                     'months': 0,
                     'seconds': 0,
                     'weeks': 0,
                     'years': 0},
 'model': 'HTMPrediction',
 'modelParams': {'anomalyParams': {u'anomalyCacheRecords': None,
                                   u'autoDetectThreshold': None,
                                   u'autoDetectWaitRecords': None},
                 'clParams': {'alpha': 0.012587500000000001,
                              'regionName': 'SDRClassifierRegion',
                              'steps': '1',
                              'verbosity': 0},
                 'inferenceType': 'TemporalMultiStep',
                 'sensorParams': {'encoders': {'_classifierInput': {'classifierOnly': True,
                                                                    'clipInput': True,
                                                                    'fieldname': 'media_wear_pct',
                                                                    'n': 459,
                                                                    'name': '_classifierInput',
                                                                    'type': 'AdaptiveScalarEncoder',
                                                                    'w': 21},
                                               u'device_temp_c': {'clipInput': True,
                                                                  'fieldname': 'device_temp_c',
                                                                  'n': 124,
                                                                  'name': 'device_temp_c',
                                                                  'type': 'AdaptiveScalarEncoder',
                                                                  'w': 21},
                                               u'drive_writes': None,
                                               u'host_written_gb': None,
                                               u'media_wear_pct': {'clipInput': True,
                                                                   'fieldname': 'media_wear_pct',
                                                                   'n': 84,
                                                                   'name': 'media_wear_pct',
                                                                   'type': 'AdaptiveScalarEncoder',
                                                                   'w': 21},
                                               u'media_written_gb': None,
                                               u'power_on_hours': None,
                                               u'write_amp': None},
                                  'sensorAutoReset': None,
                                  'verbosity': 0},
                 'spEnable': True,
                 'spParams': {'boostStrength': 0.0,
                              'columnCount': 2048,
                              'globalInhibition': 1,
                              'inputWidth': 0,
                              'numActiveColumnsPerInhArea': 40,
                              'potentialPct': 0.8,
                              'seed': 1956,
                              'spVerbosity': 0,
                              'spatialImp': 'cpp',
                              'synPermActiveInc': 0.05,
                              'synPermConnected': 0.1,
                              'synPermInactiveDec': 0.012762500000000001},
                 'tmEnable': True,
                 'tmParams': {'activationThreshold': 12,
                              'cellsPerColumn': 32,
                              'columnCount': 2048,
                              'globalDecay': 0.0,
                              'initialPerm': 0.21,
                              'inputWidth': 2048,
                              'maxAge': 0,
                              'maxSegmentsPerCell': 128,
                              'maxSynapsesPerSegment': 32,
                              'minThreshold': 10,
                              'newSynapseCount': 20,
                              'outputType': 'normal',
                              'pamLength': 2,
                              'permanenceDec': 0.1,
                              'permanenceInc': 0.1,
                              'seed': 1960,
                              'temporalImp': 'cpp',
                              'verbosity': 0},
                 'trainSPNetOnlyIfRequested': False},
 'predictAheadTime': None,
 'version': 1}

So now I’ll set the mix/max vals for the predicted field in the input equal to what they are in the classifier input (which has a much much higher n).

sheiser1 · February 22, 2019, 9:32pm

Ok so here is the new plot, with the input params (n/w/min/max) set equal to the corresponding classifier params for the pred field.
Pred_v_TARGET%20--%20Scalar

Here is the params file generate by the swarm. I passed these params into a function, which changed the encoder ‘type’ to ScalarEncoder, and added in the same min/max values (the 10th and 90th percentile values to constrain the range). So the ‘n’ for ‘media_wear_pct’ in the input was changed to 424, as it is in the _classifierInput.

MODEL_PARAMS = {'aggregationInfo': {'days': 0,
                     'fields': [],
                     'hours': 0,
                     'microseconds': 0,
                     'milliseconds': 0,
                     'minutes': 0,
                     'months': 0,
                     'seconds': 0,
                     'weeks': 0,
                     'years': 0},
 'model': 'HTMPrediction',
 'modelParams': {'anomalyParams': {u'anomalyCacheRecords': None,
                                   u'autoDetectThreshold': None,
                                   u'autoDetectWaitRecords': None},
                 'clParams': {'alpha': 0.005451785714285715,
                              'regionName': 'SDRClassifierRegion',
                              'steps': '1',
                              'verbosity': 0},
                 'inferenceType': 'TemporalMultiStep',
                 'sensorParams': {'encoders': {'_classifierInput': {'classifierOnly': True,
                                                                    'clipInput': True,
                                                                    'fieldname': 'media_wear_pct',
                                                                    'n': 424,
                                                                    'name': '_classifierInput',
                                                                    'type': 'AdaptiveScalarEncoder',
                                                                    'w': 21},
                                               u'device_temp_c': {'clipInput': True,
                                                                  'fieldname': 'device_temp_c',
                                                                  'n': 127,
                                                                  'name': 'device_temp_c',
                                                                  'type': 'AdaptiveScalarEncoder',
                                                                  'w': 21},
                                               u'drive_writes': None,
                                               u'host_written_gb': None,
                                               u'media_wear_pct': {'clipInput': True,
                                                                   'fieldname': 'media_wear_pct',
                                                                   'n': 307,
                                                                   'name': 'media_wear_pct',
                                                                   'type': 'AdaptiveScalarEncoder',
                                                                   'w': 21},
                                               u'media_written_gb': None,
                                               u'power_on_hours': None,
                                               u'write_amp': None},
                                  'sensorAutoReset': None,
                                  'verbosity': 0},
                 'spEnable': True,
                 'spParams': {'boostStrength': 0.0,
                              'columnCount': 2048,
                              'globalInhibition': 1,
                              'inputWidth': 0,
                              'numActiveColumnsPerInhArea': 40,
                              'potentialPct': 0.8,
                              'seed': 1956,
                              'spVerbosity': 0,
                              'spatialImp': 'cpp',
                              'synPermActiveInc': 0.05,
                              'synPermConnected': 0.1,
                              'synPermInactiveDec': 0.005641071428571429},
                 'tmEnable': True,
                 'tmParams': {'activationThreshold': 13,
                              'cellsPerColumn': 32,
                              'columnCount': 2048,
                              'globalDecay': 0.0,
                              'initialPerm': 0.21,
                              'inputWidth': 2048,
                              'maxAge': 0,
                              'maxSegmentsPerCell': 128,
                              'maxSynapsesPerSegment': 32,
                              'minThreshold': 11,
                              'newSynapseCount': 20,
                              'outputType': 'normal',
                              'pamLength': 1,
                              'permanenceDec': 0.1,
                              'permanenceInc': 0.1,
                              'seed': 1960,
                              'temporalImp': 'cpp',
                              'verbosity': 0},
                 'trainSPNetOnlyIfRequested': False},
 'predictAheadTime': None,
 'version': 1}

sheiser1 · February 22, 2019, 10:07pm

I also notice that the targets and predictions line up much better when I normalize both value sets before plotting:

Pred_v_TARGET%20--%20Scalar

rhyolight · February 22, 2019, 10:44pm

So is this more like what you expected? The issue was that predictions were coming out at a completely different resolution than input. So that looks like it is fixed when you ensure the input classifier params are the same as the encoder params right?

sheiser1 · February 22, 2019, 11:02pm

Well not exactly, cause the most recent plot without the gap is made from the same values as the prior plot with the gap. The only difference is before plotting I did:

Targets = normalize(Targets)
Preds = normalize(Preds)

So without normalization the gap remains, even with the equalized input/classifier param values. It’s not that I couldn’t take the current predictions and post-process them with normalization, though that it requires existing distributions of targets and predictions to normalize from - which seems to require a kind of batch processing in essence. Also I’d really like to figure out what’s causing this mysterious shift, so I can be confident in the robustness of my pipeline here.

rhyolight · February 22, 2019, 11:12pm

I’m confused about this plot because they are all on the same chart. Are Preds_1step and predField both plotted along the same Y axis in this chart?

sheiser1 · February 23, 2019, 12:02am

They are on the same axis yes, all the predictions fall in the range of 0-403 and all the predField values fall between 1,020 and 2,120. Since the Anomaly Score and Anomaly Likelihood values fall between 0 and 1 I scaled them to the predictions so they’d be basically visible.

rhyolight · February 23, 2019, 12:45am

That is crazy. I don’t get it.

So create a brand new model and feed it one data point. It should give you a prediction (if not just send another data point until you get one). Is the first prediction you get back outside the min/max of the input? If so, please again copy and paste the exact model parameters so we can figure this out.

sheiser1 · February 24, 2019, 11:09pm

for k,v in inclFIELD_Set_PARAMS_encoders.items():
    print(k,' -- ',v)

(u'media_wear_pct', ' -- ', None)
('_classifierInput', ' -- ', {'maxval': 2640.0, 'classifierOnly': True, 'name': '_classifierInput', 'minval': 1224.0, 'clipInput': True, 'n': 449, 'fieldname': 'media_wear_pct', 'w': 21, 'type': 'ScalarEncoder'})
(u'media_written_gb', ' -- ', None)
(u'device_temp_c', ' -- ', {'maxval': 940.0, 'name': 'device_temp_c', 'minval': 476.0, 'clipInput': True, 'n': 248, 'fieldname': 'device_temp_c', 'w': 21, 'type': 'ScalarEncoder'})
(u'power_on_hours', ' -- ', None)
(u'write_amp', ' -- ', None)
(u'host_written_gb', ' -- ', None)
(u'drive_writes', ' -- ', None)

In the console
model = ModelFactory.create(inclFIELD_Set_PARAMS)
predField = ‘meda_wear_pct’
model.enableInference({‘predictedField’: predField})
mod_inclFIELDS = [‘device_temp_c’,‘media_wear_pct’]

DF_NuPIC = pd.DataFrame(DF_input[mod_inclFIELDS].head(numRecords))
   
for index, row in DF_NuPIC.iterrows():
    modelInput = dict()
    for field in self.inclFIELDS:
        if field in inclFIELD_Set_PARAMS_encoders:
            modelInput[field] = row[field]
    break
 
row (pd.series) = 
media_wear_pct    2112.0
device_temp_c      725.0
Name: 0, dtype: float64

modelInput (dict) = 
{('media_wear_pct', ' -- ', 2112.0),
('device_temp_c', ' -- ', 725.0)}

resultObj = model.run(modelInput)
predFieldVal = row[predField]

bestPredictions = resultObj.inferences["multiStepBestPredictions"]
allPredictions = resultObj.inferences["multiStepPredictions"]
AnomScore = resultObj.inferences["anomalyScore"]
AnomLikl_obj = anomaly_likelihood.AnomalyLikelihood()
AnomLikl = AnomLikl_obj.anomalyProbability(modelInput[self.predField], AnomScore, index)
oneStep = bestPredictions[1]
try: oneStepConfidence = allPredictions[1][oneStep]
except: oneStepConfidence = 0
result = {'index':index, 'AScore':AnomScore, 'ALikl':AnomLikl, 'predFieldVal':predFieldVal, 'Pred_1': oneStep, 'Pred_1_conf':oneStepConfidence*100}

The 'result’s were accumulated in a list and I plotted all the ‘prevFieldVals’ vs the 'Pred_1’s. The model does seem to be learning well within its distribution, since the normalized plots of ‘predFieldVals’ and ‘Pred_1’ are very close.

When I ran the model on the first input in the console:

predFieldVal = row[predField]
predFieldVal
2112.0

bestPredictions = resultObj.inferences[“multiStepBestPredictions”]
allPredictions = resultObj.inferences[“multiStepPredictions”]
oneStep = bestPredictions[1]

oneStep
268.0

allPredictions
{1: {268.0: 1.0}}

sheiser1 · February 25, 2019, 3:25am

On timestep 2:

modelInput
{'media_wear_pct': 2112.0, 'device_temp_c': 740.0}

resultObj = model.run(modelInput)

predFieldVal = row[predField]
predFieldVal
2112.0

bestPredictions = resultObj.inferences["multiStepBestPredictions"]
allPredictions = resultObj.inferences["multiStepPredictions"]

allPredictions
{1: {268.0: 1.0000000000000027}}

rhyolight · February 25, 2019, 5:14pm

This looks fishy:

{
   "_classifierInput":{
      "maxval":2640.0,
      "minval":1224.0,
      "fieldname":"media_wear_pct",
   },
   "device_temp_c":{
      "maxval":940.0,
      "minval":476.0,
   }
}

Seems like _classifierInput.fieldname should be device_temp_c, and the min / max values should be the same. These are the model params that swarming returned?

sheiser1 · February 25, 2019, 11:14pm

Here are the parameters that swarming returned. I notice that it didn’t pick up the predicted field in the input encoders.

MODEL_PARAMS = {'aggregationInfo': {'days': 0,
                     'fields': [],
                     'hours': 0,
                     'microseconds': 0,
                     'milliseconds': 0,
                     'minutes': 0,
                     'months': 0,
                     'seconds': 0,
                     'weeks': 0,
                     'years': 0},
 'model': 'HTMPrediction',
 'modelParams': {'anomalyParams': {u'anomalyCacheRecords': None,
                                   u'autoDetectThreshold': None,
                                   u'autoDetectWaitRecords': None},
                 'clParams': {'alpha': 0.001050709364398733,
                              'regionName': 'SDRClassifierRegion',
                              'steps': '1',
                              'verbosity': 0},
                 'inferenceType': 'TemporalMultiStep',
                 'sensorParams': {'encoders': {'_classifierInput': {'classifierOnly': True,
                                                                    'clipInput': True,
                                                                    'fieldname': 'media_wear_pct',
                                                                    'n': 371,
                                                                    'name': '_classifierInput',
                                                                    'type': 'AdaptiveScalarEncoder',
                                                                    'w': 21},
                                               u'device_temp_c': {'clipInput': True,
                                                                  'fieldname': 'device_temp_c',
                                                                  'n': 283,
                                                                  'name': 'device_temp_c',
                                                                  'type': 'AdaptiveScalarEncoder',
                                                                  'w': 21},
                                               u'libversion': None,
                                               u'media_wear_pct': None,
                                               u'power_on_hours': None,
                                               u'write_amp': None},
                                  'sensorAutoReset': None,
                                  'verbosity': 0},
                 'spEnable': True,
                 'spParams': {'boostStrength': 0.0,
                              'columnCount': 2048,
                              'globalInhibition': 1,
                              'inputWidth': 0,
                              'numActiveColumnsPerInhArea': 40,
                              'potentialPct': 0.8,
                              'seed': 1956,
                              'spVerbosity': 0,
                              'spatialImp': 'cpp',
                              'synPermActiveInc': 0.05,
                              'synPermConnected': 0.1,
                              'synPermInactiveDec': 0.021600119406483983},
                 'tmEnable': True,
                 'tmParams': {'activationThreshold': 13,
                              'cellsPerColumn': 32,
                              'columnCount': 2048,
                              'globalDecay': 0.0,
                              'initialPerm': 0.21,
                              'inputWidth': 2048,
                              'maxAge': 0,
                              'maxSegmentsPerCell': 128,
                              'maxSynapsesPerSegment': 32,
                              'minThreshold': 10,
                              'newSynapseCount': 20,
                              'outputType': 'normal',
                              'pamLength': 1,
                              'permanenceDec': 0.1,
                              'permanenceInc': 0.1,
                              'seed': 1960,
                              'temporalImp': 'cpp',
                              'verbosity': 0},
                 'trainSPNetOnlyIfRequested': False},
 'predictAheadTime': None,
 'version': 1}

Before passing into the model I changed the modelParams[‘sensorParams’][‘encoders’] to:

(u'media_wear_pct', ' -- ', None)
('_classifierInput', ' -- ', {'maxval': 2472.0, 'classifierOnly': True, 'name': '_classifierInput', 'clipInput': True, 'minval': 666.0, 'n': 371, 'fieldname': 'media_wear_pct', 'w': 21, 'type': 'ScalarEncoder'})
(u'write_amp', ' -- ', None)
(u'libversion', ' -- ', None)
(u'device_temp_c', ' -- ', {'maxval': 922.0, 'name': 'device_temp_c', 'clipInput': True, 'minval': 480.0, 'n': 283, 'fieldname': 'device_temp_c', 'w': 21, 'type': 'ScalarEncoder'})
(u'power_on_hours', ' -- ', None)

The min/max values for each field are the 5th/95th percentile values for each field. Here’s the resulting plot:
Pred_v_TARGET

sheiser1 · February 25, 2019, 11:22pm

Here’s my main loop:

predField = 'media_wear_pct'
inclFIELDS = ['media_wear_pct', 'device_temp_c']
numRecords = 25000

    PARAMS_encoders = PARAMS['modelParams']['sensorParams']['encoders']
    model = ModelFactory.create(PARAMS)
    model.enableInference({'predictedField': predField})

    print('HTM Model Initiated')
    print(' --> inclFIELD_SET = {}'.format(inclFIELDS))
    print(' --> Pred Field = {}'.format(predField))
    print(' ')
    results = list()
    DF_NuPIC = pd.DataFrame(DF_input[inclFIELDS].head(numRecords))
    for index, row in DF_NuPIC.iterrows():
        modelInput = dict()
        for field in inclFIELDS:
            if field in PARAMS_encoders:
                modelInput[field] = row[field]

        resultObj = model.run(modelInput)
        predFieldVal = row[predField]
        bestPredictions = resultObj.inferences["multiStepBestPredictions"]
        allPredictions = resultObj.inferences["multiStepPredictions"]
        AnomScore = resultObj.inferences["anomalyScore"]
        AnomLikl_obj = anomaly_likelihood.AnomalyLikelihood()
        AnomLikl = AnomLikl_obj.anomalyProbability(modelInput[self.predField], AnomScore, index)
        oneStep = bestPredictions[1]
        try: oneStepConfidence = allPredictions[1][oneStep]
        except: oneStepConfidence = 0
        result = {'index':index, 'AScore':AnomScore, 'ALikl':AnomLikl, 'predFieldVal':predFieldVal, 'Pred_1': oneStep, 'Pred_1_conf':oneStepConfidence*100}

Topic		Replies	Views
Understanding NuPIC and troubleshooting to get the best results NuPIC	2	1455	July 20, 2016
Using various encoders in swarming NuPIC	2	379	July 6, 2017
Question on swarm results in hotgym example NuPIC	1	298	February 19, 2019
Bad results when adding date encoder Getting Started	14	1145	December 19, 2019
Scalar Encoder: specifying w and n NuPIC	2	1279	December 26, 2016

How much should I trust swarm results?

----------------------------------------------------------------

Related Topics