Spatial Pooling Parameter Descriptions

inputDimensions

  A sequence representing the dimensions of the input vector. Format is
  (height, width, depth, ...), where each value represents the size of the
  dimension.  For a topology of one dimension with 100 inputs use 100, or
  (100,). For a two dimensional topology of 10x5 use (10,5).

Values depend on dimensions of encodings.

columnDimensions

  A sequence representing the dimensions of the columns in the region.
  Format is (height, width, depth, ...), where each value represents the
  size of the dimension.  For a topology of one dimension with 2000 columns
  use 2000, or (2000,). For a three dimensional topology of 32x64x16 use
  (32, 64, 16).
  • sane default: [2048]
  • min: ?
  • max: ?

potentialRadius

  This parameter determines the extent of the input that each column can
  potentially be connected to.  This can be thought of as the input bits
  that are visible to each column, or a 'receptiveField' of the field of
  vision. A large enough value will result in 'global coverage', meaning
  that each column can potentially be connected to every input bit. This
  parameter defines a square (or hyper
  square) area: a column will have a max square potential pool with sides of
  length 2 * potentialRadius + 1.
  • sane default: 16
  • min: ?
  • max: ?

potentialPct (“potential percent”)

  The percent of the inputs, within a column's potential radius, that a
  column can be connected to.  If set to 1, the column will be connected
  to every input within its potential radius. This parameter is used to
  give each column a unique potential pool when a large potentialRadius
  causes overlap between the columns. At initialization time we choose
  ((2*potentialRadius + 1)^(# inputDimensions) * potentialPct) input bits
  to comprise the column's potential pool.
  • sane default: 0.85
  • min: 0.0
  • max: 1.0

globalInhibition

  If true, then during inhibition phase the winning columns are selected
  as the most active columns from the region as a whole. Otherwise, the
  winning columns are selected with respect to their local neighborhoods.
  Using global inhibition boosts performance x60.
  • sane default: true

localAreaDensity

  The desired density of active columns within a local inhibition area
  (the size of which is set by the internally calculated inhibitionRadius,
  which is in turn determined from the average size of the connected
  potential pools of all columns). The inhibition logic will insure that
  at most N columns remain ON within a local inhibition area, where
  N = localAreaDensity * (total number of columns in inhibition area).
  • sane default: -1.0
  • min: ?
  • max: ?
  • linked to numActiveColumnsPerInhArea

numActiveColumnsPerInhArea (“number of active columns per inhibition area”)

  An alternate way to control the density of the active columns. If
  numActiveColumnsPerInhArea is specified then localAreaDensity must be
  less than 0, and vice versa.  When using numActiveColumnsPerInhArea, the
  inhibition logic will insure that at most 'numActiveColumnsPerInhArea'
  columns remain ON within a local inhibition area (the size of which is
  set by the internally calculated inhibitionRadius, which is in turn
  determined from the average size of the connected receptive fields of all
  columns). When using this method, as columns learn and grow their
  effective receptive fields, the inhibitionRadius will grow, and hence the
  net density of the active columns will *decrease*. This is in contrast to
  the localAreaDensity method, which keeps the density of active columns
  the same regardless of the size of their receptive fields.
  • sane default: 10.0
  • min: ?
  • max: ?
  • linked to localAreaDensity

stimulusThreshold

  This is a number specifying the minimum number of synapses that must be
  on in order for a columns to turn ON. The purpose of this is to prevent
  noise input from activating columns. Specified as a percent of a fully
  grown synapse.
  • sane default: 0
  • min: 0
  • max: ?

synPermInactiveDec (“synapse permanence inactive decrement”)

  The amount by which an inactive synapse is decremented in each round.
  Specified as a percent of a fully grown synapse.
  • sane default: 0.008
  • min: 0.0
  • max: ?

synPermActiveInc (“synapse permanence active increment”)

  The amount by which an active synapse is incremented in each round.
  Specified as a percent of a fully grown synapse.
  • sane default: 0.05
  • min: 0.0
  • max: ?

synPermConnected (“synapse permanence connected”)

  The default connected threshold. Any synapse whose permanence value is
  above the connected threshold is a "connected synapse", meaning it can
  contribute to the cell's firing.
  • sane default: 0.10
  • min: 0.0
  • max: ?

minPctOverlapDutyCycle (“minimum percent overlap duty cycle”)

  A number between 0 and 1.0, used to set a floor on how often a column
  should have at least stimulusThreshold active inputs. Periodically, each
  column looks at the overlap duty cycle of all other columns within its
  inhibition radius and sets its own internal minimal acceptable duty cycle
  to: minPctDutyCycleBeforeInh * max(other columns' duty cycles).  On each
  iteration, any column whose overlap duty cycle falls below this computed
  value will  get all of its permanence values boosted up by
  synPermActiveInc. Raising all permanences in response to a sub-par duty
  cycle before  inhibition allows a cell to search for new inputs when
  either its previously learned inputs are no longer ever active, or when
  the vast majority of them have been "hijacked" by other columns.
  • sane default: 0.001
  • min: 0.0
  • max: 1.0

minPctActiveDutyCycle (“minimum percent active duty cycle”)

 A number between 0 and 1.0, used to set a floor on how often a column
 should be activate.  Periodically, each column looks at the activity duty
 cycle of all other columns within its inhibition radius and sets its own
 internal minimal acceptable duty cycle to: minPctDutyCycleAfterInh *
 max(other columns' duty cycles).  On each iteration, any column whose duty
 cycle after inhibition falls below this computed value will get its
 internal boost factor increased.
  • sane default: 0.001
  • min: 0.0
  • max: 1.0

dutyCyclePeriod

  The period used to calculate duty cycles. Higher values make it take
  longer to respond to changes in boost or synPerConnectedCell. Shorter
  values make it more unstable and likely to oscillate.
  • sane default: 1000
  • min: ?
  • max: ?

maxBoost

  The maximum overlap boost factor. Each column's overlap gets multiplied
  by a boost factor before it gets considered for inhibition.  The actual
  boost factor for a column is number between 1.0 and maxBoost. A boost
  factor of 1.0 is used if the duty cycle is >= minOverlapDutyCycle,
  maxBoost is used if the duty cycle is 0, and any duty cycle in between is
  linearly extrapolated from these 2 endpoints.
  • sane default: 1.0
  • min: 0.0
  • max: ?
8 Likes

I just created this as a wiki so others can update it and add notes about these parameters as they work with them.

In particular, I need help identifying some of the minimum and maximum values for some parameters.

Nice. Thanks. @rhyolight

Thanks for this.Can we expect one for the temporal pooler as well? The descriptions on the Github Wiki seem to be incomplete.

I’m still trying to figure out if this is the best place for this document. When I get to doing the TM in HTM School, I’ll probably do something similar to expose those params as well.

As far as I understood, the potential radius means the number of input bits that the each column can connect to. In other words it determines the size of the potential pool of each column right?
Now if my input is of length 15 bits and I give the potential radius of the Spatial Pooler having 4 columns as 15 only, then each column is supposed to connect to all the input bits, however when I run the getConnectedSynapses function I can see that each column is cannot see all the bits in the input. I think I missed some point in my understanding, so can you please elaborate? Thank You. :slight_smile:

I think you’re using too small of numbers. Experiments this small have trouble working. Make 2048 columns and an input space of like 100-400 bits.

Also, potentialRadius should only come into play if globalInhibition is on (off by default).

Hello @rhyolight @David_Keeney @sheiser1 ,

I hope you will answer my questions below. Thank you to everyone.

While creating a potential pool for each mini column in SP, if I use GLOBAL INHIBITION = TRUE, I select random values based on a percentage on the entire input. Does INHIBITION RADIUS include all the rest of its neighbors here?

If I don’t use GLOBAL INHIBITION, how to choose a random area selection and size for mini columns? Is INHIBITION RADIUS done here for the selected local area? (Euclidean Distance used in some papers)

Can you tell me this concept of inhibition radius? How is the initial value calculated? I found the function “averageReceptiveFieldSize ()” but how does it relate to Mini columns?

Then, am I free to use numActiveColumnsPerInhArea or localareadensity for kthscore’s function?

Right, the top 40 columns in overlap score inhibit the other 2008. Any column can inhibit any other regardless of topology.

With local inhibition each column completes only with those in its neighborhood.

The neighborhood for each column (centerColumn) comes from numpy.math.topology():

inhibitionRadius is a float, and columnDimensions is a 2D array like (64,64)

Here’s the repo for reference:

2 Likes

Thank you very much for your valuable information.
I will examine the codes.

When I give the numActiveColumnsPerInhArea parameter, I cannot get a fixed cluster size at the output. What should I do to get this a real sparse (0.02) value?

Should I use Localareadensity?

I m trying write the sp codes myself.

1 Like

Yes localAreaDensity, is the alternative to numActiveColumnsPerInhArea so one must always be 0.

It seems localAreaDensity can range from 0 to 0.5:

1 Like

Hello again, @sheiser1

Should the number of active mini columns be the same for each input before proceeding to the “Learning” phase in SP?

Because I couldn’t fix the number of active mini columns.

In the neighborhood function, all elements except “column index” were sorted as desc for reach minLocalActivity value.

I give the code sample I wrote below.

miniColumnsCount=1024
globalInhibiton=True

arrMiniColumn = []
arrPotantialSynaps = []
arrPotantialSynapsValue = []
for index, row in enumerate(arrInputs):
arrMiniColumn.append(index)
arrTempPotSynaps = []
arrTempPotSynapsValue = []
for j, rowPerm in enumerate(row[3]):
if (rowPerm >= connectedPerm):
arrTempPotSynaps.append(arrInputs[index][1][j])
arrTempPotSynapsValue.append(arrInputs[index][2][j])
arrPotantialSynaps.append(arrTempPotSynaps)
arrPotantialSynapsValue.append(arrTempPotSynapsValue)

def averageReceptiveFieldSize(globalInhibiton,arrPotantialSynaps,miniColumnsCount):
value=miniColumnsCount
if(globalInhibiton==False):
sumvalue = 0
for index, row in enumerate(arrPotantialSynaps):
sumvalue = sumvalue + len(row)
value= math.floor(sumvalue / len(arrPotantialSynaps))

return value

inhibitionRadius = averageReceptiveFieldSize(globalInhition,arrPotantialSynaps,miniColumnsCount)
numActiveColumnsPerInhArea = numActiveColumnsPerInhArea

By “each input” do you mean each new value at each time step? I’m having a hard time reading your code there but yes, the number of active mini columns should be the same.

1 Like

Yes, I meant it as new value.Assuming there is no learning. every new iteration.
“kth function” output produces minimal overlap. Doesn’t this value dynamically divide the output in the inhibition step? I cannot reach a fixed number of active column elements.

Do you mean that you’re not getting the same number of activated columns from the SP every time? If so it should be the top 2% of column in overlap score – which may call for tie-break if some are equal.

Yes.yes I need same number of activeted columns. Ä°s this required right before learning

Is the “local area density” parameter used for this result instead of “numActiveColumnsPerInhArea”?

Yes, it is required before learning. Having a variable sparsity (variable number of active mini-columns) will be seen as noise (or failure) impacting temporal patterns learned by the network.

1 Like

Now can we say that? @AMZ @sheiser1
I don’t need to use the “numActiveColumnsPerInhArea” parameter if there is no “learning”. because when I use “numActiveColumnsPerInhArea”, a dynamic number of active mini columns are generated. I need to use localareadensity for learning.

I suspect the code was written for either to work, learning or not.

If set between 0 and 0.5, localAreaDensity is set to density, which is passed into the _inhibitColumnsLocal function.

In this function localAreaDensity determines: the number of “the columns in a local neighborhood are allowed to be active”:

1 Like