Asymmetric confidence when predicting a symmetric signal

Science Numenta Theory
28

So I observed from some debugging that the actualValues (and possibly probabilities), obtained as a consequence of running the code at http://nupic.docs.numenta.org/1.0.5/quick-start/network.html#getting-predictions (quoted below), do not produce correct values.

def getPredictionResults(network, clRegionName):
  """Helper function to extract results for all prediction steps."""


  classifierRegion = network.regions[clRegionName]
  actualValues = classifierRegion.getOutputData("actualValues")
  probabilities = classifierRegion.getOutputData("probabilities")
  steps = classifierRegion.getSelf().stepsList

  N = classifierRegion.getSelf().maxCategoryCount
  results = {step: {} for step in steps}
  for i in range(len(steps)):
    # stepProbabilities are probabilities for this prediction step only.
    stepProbabilities = probabilities[i * N:(i + 1) * N - 1]
    mostLikelyCategoryIdx = stepProbabilities.argmax()
    predictedValue = actualValues[mostLikelyCategoryIdx]
    predictionConfidence = stepProbabilities[mostLikelyCategoryIdx]
    results[steps[i]]["predictedValue"] = predictedValue
    results[steps[i]]["predictionConfidence"] = predictionConfidence
  return results

actualValues is supposed to be a map from bucket index to an estimate of the actual input value that encoder will see in that bucket and probabilities is supposed to be a map from bucket index to probability.

Let me show you the output actualValues and probabilities for first couple of values.

first input is 0.5 (corresponds to a bucket index of ‘89’)

image
image.jpg1813×378 178 KB

second input is 0.53270155 (corresponding to a bucket index of ‘94’)

image
image.jpg2914×423 254 KB

A more detailed view of actualValues shows that all the values till index 88 are 0.53270155 and 89th value is 0.5 and rest are 0.0. Similarly, probabilities are all 0.011111 till index 89 and rest are 0.0.

It seems that any bucket that is not yet seen is filled by the most recent value in actualValues. I am not sure about the exact calculations of probabilities but if the same kind of problem is going on with probabilities then it will cause problems with the “predictionConfidence”

1 Like