HTM.java network setup for making predictions - NuPIC edition

I created this topic on HTM.java forum and I was suggested by @cogmission to try this forum as well.
I am experimenting the prediction capabilities of HTM.java and I run into the following problem.

I am feeding three strings into HTM network in a particular order:

"Dark Side Of The Moon"
"Blowing In The Wind"
"The Piper Of The Gates Of Dawn"

After learning I got the following result:

Record Number: 985, input Value: Blowing In The Wind, anomaly Score: 0.0, predicted value: The Piper Of The Gates Of Dawn
Record Number: 986, input Value: The Piper Of The Gates Of Dawn, anomaly Score: 0.0, predicted value: The Piper Of The Gates Of Dawn
Record Number: 987, input Value: Dark Side Of The Moon, anomaly Score: 0.0, predicted value: The Piper Of The Gates Of Dawn
Record Number: 988, input Value: Blowing In The Wind, anomaly Score: 0.0, predicted value: The Piper Of The Gates Of Dawn
Record Number: 989, input Value: The Piper Of The Gates Of Dawn, anomaly Score: 0.0, predicted value: The Piper Of The Gates Of Dawn
Record Number: 990, input Value: Dark Side Of The Moon, anomaly Score: 1.0, predicted value: The Piper Of The Gates Of Dawn
Record Number: 991, input Value: Blowing In The Wind, anomaly Score: 0.0, predicted value: The Piper Of The Gates Of Dawn
Record Number: 992, input Value: The Piper Of The Gates Of Dawn, anomaly Score: 0.0, predicted value: The Piper Of The Gates Of Dawn
Record Number: 993, input Value: Dark Side Of The Moon, anomaly Score: 0.0, predicted value: The Piper Of The Gates Of Dawn
Record Number: 994, input Value: Blowing In The Wind, anomaly Score: 0.0, predicted value: The Piper Of The Gates Of Dawn
Record Number: 995, input Value: The Piper Of The Gates Of Dawn, anomaly Score: 0.0, predicted value: The Piper Of The Gates Of Dawn
Record Number: 996, input Value: Dark Side Of The Moon, anomaly Score: 0.0, predicted value: The Piper Of The Gates Of Dawn
Record Number: 997, input Value: Blowing In The Wind, anomaly Score: 0.0, predicted value: The Piper Of The Gates Of Dawn
Record Number: 998, input Value: The Piper Of The Gates Of Dawn, anomaly Score: 0.0, predicted value: The Piper Of The Gates Of Dawn
Record Number: 999, input Value: Dark Side Of The Moon, anomaly Score: 0.0, predicted value: The Piper Of The Gates Of Dawn

The anomaly score looks pretty good but the predicted value is the same.
I would expect to have a pretty accurate prediction after 900 cycles.

Here are the configuration parameters:

{
	Spatial: {
		learn:true
		inputDimensions:[64]
		potentialRadius:16
		potentialPct:0.5
		globalInhibition:false
		inhibitionRadius:0
		localAreaDensity:-1.0
		numActiveColumnsPerInhArea:10.0
		stimulusThreshold:0.0
		synPermInactiveDec:0.01
		synPermActiveInc:0.1
		synPermConnected:0.1
		synPermBelowStimulusInc:0.01
		synPermTrimThreshold:0.5
		minPctOverlapDutyCycles:0.001
		minPctActiveDutyCycles:0.001
		dutyCyclePeriod:1000
		maxBoost:10.0
		spVerbosity:0
	}
	Temporal: {
		columnDimensions:[2048]
		cellsPerColumn:32
		activationThreshold:13
		learningRadius:2048
		minThreshold:10
		maxNewSynapseCount:20
		initialPermanence:0.21
		connectedPermanence:0.5
		permanenceIncrement:0.1
		permanenceDecrement:0.1
		tmVerbosity:0
	}
	Other: {
		random:org.numenta.nupic.util.MersenneTwister@2ed2d9cb
		seed:42
		n:500
		w:21
		minVal:0.0
		maxVal:1000.0
		radius:21.0
		resolution:1.0
		periodic:false
		clipInput:false
		forced:false
		fieldName:UNSET
		fieldType:int
		encoderType:ScalarEncoder
		fieldEncodings:{alarm={clipInput=false, maxVal=0.0, fieldName=alarm, minVal=0.0, periodic=false, forced=false, w=21, radius=0.0, resolution=0.0, fieldType=string, n=500, encoderType=SDRCategoryEncoder}}
		hasClassifiers:false
	}
}

Any idea what is wrong with my setup?

Thank you for your time,
Pavel Guler

2 Likes

Hi @pguler. I edited your post for code formatting.

It looks like you are using a ScalarEncoder, but shouldnā€™t you be using a CategoryEncoder?

HTM.java is using the encoder that is set in the parameter: fieldEncodings.
In the javadoc this parameter is documented like this: /** Designates holder for the Multi Encoding Map */

So the encoder is SDRCategoryEncoder as specified:
fieldEncodings:{alarm={clipInput=false, maxVal=0.0, fieldName=alarm, minVal=0.0, periodic=false, forced=false, w=21, radius=0.0, resolution=0.0, fieldType=string, n=500, encoderType=SDRCategoryEncoder}}

It looks like this parameter is required otherwise an exception is thrown.

On the other hand, in all the examples I worked with, apparently, this field was set and used so I just changed the existing examples.

If ScalarEncoder was used it would not accept a string input. I have made this mistake before :slight_smile:

1 Like

Just to make sure that I am using the right encoder I set the parameters manually:

	Other: {
		random:org.numenta.nupic.util.MersenneTwister@2ed2d9cb
		seed:42
		n:500
		w:21
		minVal:0.0
		maxVal:0.0
		radius:0.0
		resolution:0.0
		periodic:false
		clipInput:false
		forced:false
		fieldName:alarm
		fieldType:string
		encoderType:SDRCategoryEncoder
		fieldEncodings:{alarm={clipInput=false, maxVal=0.0, fieldName=alarm, minVal=0.0, periodic=false, forced=false, w=21, radius=0.0, resolution=0.0, fieldType=string, n=500, encoderType=SDRCategoryEncoder}}
		hasClassifiers:false
	}

ā€¦ and the same result.

@pguler @rhyolight,

I have to do a check and make sure that that method outputs the true state of the Network. It has changed recently, and unfortunately there may be a disconnect between what that method uses to ā€œgatherā€ state and what the state (fields and their values) really isā€¦

EDIT: There are two levels in the map holding encoder state and the parent level (which has no effect), has values - the key fieldEncodings: is the ā€œrealā€ holder of encoder valuesā€¦ I need to clean this up so that the parent level is not outputted by this method, for example:

An issue has been created for this, here

Looking at Spatial parametersā€¦

	Spatial: {
		learn:true
		inputDimensions:[64]
		potentialRadius:16
		potentialPct:0.5
		globalInhibition:false
		inhibitionRadius:0
		localAreaDensity:-1.0
		numActiveColumnsPerInhArea:10.0
		stimulusThreshold:0.0
		synPermInactiveDec:0.01
		synPermActiveInc:0.1
		synPermConnected:0.1
		synPermBelowStimulusInc:0.01
		synPermTrimThreshold:0.5
		minPctOverlapDutyCycles:0.001
		minPctActiveDutyCycles:0.001
		dutyCyclePeriod:1000
		maxBoost:10.0
		spVerbosity:0
	}

I noticed the following parameters

globalInhibition:false
inhibitionRadius:0
localAreaDensity:-1.0
numActiveColumnsPerInhArea:10.0

Not sure how they are related and whether the default values are alright?
Shouldnā€™t inhibitionRadius have a value different from zero for example?

Any suggestion for these parameters?

Hi @pguler,

You are correct, the way this output needs to be looked at. inhibitionRadius may be internally calculated and not captured in the field used to display this output, though all the internal settings are correct and have the values they need.

Let me be clear again. Internally, everything is functioning fine (as confirmed by a very extensive line by line vetting process and comparison with NuPICā€™s Python and C++ output) - there just needs to be a ā€œonce-overā€ to make sure the fields used to display this output are updated with the correct values.

EDIT: I added an issue here to handle this.

I have a few comments on your parametersā€¦

  • globalInhibition should be true.
  • potentialRadius should be the same size as your input encoding array (although I believe it is ignored if global inhibition is on).
  • for an SP with 2048 columns, numActiveColumnsPerInhArea should be 2% of that, which is 40
  • play around with the maxBoost setting, I would try 1.0 (no boosting) and 5.0 to see if that changes anything

One quick correction: potentialRadius is not ignored if global inhibition is on. But I think HTM.java allows you to set it to -1, which is a nice feature. :slight_smile: https://github.com/numenta/nupic/issues/3368#issuecomment-253936692

1 Like