Boostfunction location, and other questions

Engineering
1

A few questions:

  1. I could not find the ‘boostfunction’ in the nupic source, where is it? (I used a text-search program looking for ‘boostfunction’)
  2. if I present patterns for the program to learn, do I have to present them many times, in many different orders? For instance, if I want it to learn to represent 100 numbers (between 1 and 100), would I just present all 100 numbers once? Or fifty times? Or more?
  3. If I skipped some numbers, would it be able to generalize to those numbers?
  4. if a column has synapses attached to indices 5,15,20, (for a one dimensional topology), can I say that the “natural center” of the column is the median (in this case, 15) of the indices?
2

New spatial patterns will be uniquely represented by spatial pooling right away, but as it seems the same spatial pattern again and again will better represent it. So you don’t need to show it all the possible spatial patterns for it to learn sequences of those patterns.

If you have a temporal pattern you want it to learn, usually it can learn simple sequences after seeing them only a few times.

I’m not sure, I think it depends on the spatial data representation and the sequences.

Hm, perhaps. But what are you trying to get out of this?

3

Matt: the reason I asked about ‘natural center’ is that the PDF I was reading was saying I should initialize permanences to a slightly larger value if they are near the ‘natural center’ of a column. At least the original PDF (which was outdated) said to do this. I have updated most of the code now with the most recent PDF, but maybe this is now incorrect, I have to take a look.

I got into this project mainly because I was lazy. To run nupic on windows, I have to download cMake and read the manual for it. Rather than do that, I thought that since the algorithm for the spatial pooler does not take much code, I’ll write it in vb.net, and then test out my idea.

The PDF I’m using has pseudocode, but it is not all that detailed about some aspects of the algorithm, and so I’m sure I’m diverging from nupic.
The idea I wanted to test was this. Given a spatial pattern (in the spatial pooler, and maybe the temporal pooler), if I want to know what value it corresponds to, rather than use a CLA, I could trace the connections backwards - from the pooler to the encoder layer. For instance, if column 35 is active, I could see what inputs from the encoder feed into column 35’s dendrites. Some of them would be incorrect (assuming column 35 responds to more than one pattern), but I could winnow them out, because those inputs would project to columns that were inactive. (Its more complicated, but that’s the basic idea). This idea might also be usable for a temporal pooler.
If it worked I would have to test it with noise, and see how it generalizes to patterns it has never seen, and so forth.

4

Haha, that doesn’t sound lazy to me. Certainly writing the algorithms yourself is the best way to learn how HTM works. But then you’ll have to implement the TM algorithm to do any temporal prediction work.

Your idea sounds like you want to implement temporal pooling yourself. NuPIC’s TM algorithm does not do temporal pooling, so I can’t help much there.

5

I wrote the spatial pooler in VB.net, but I need to know if it is working right.
First I trained it by showing it all numbers between 1 and 134.
Then I tried feeding in some numbers to see what the SDRs would look like, and I pasted results below. I am using 134 buckets for values that range from 0 to 134, and I am using global inhibition with at most 40 columns on at any time. The stats below tell you for instance that when I fed in ‘7’ and it got encoded and then produced an SDR, that resulting SDR had 32 active columns, and 30 of those were in common with the SDR that I got when I fed in an input of ‘6’. But you can also see that values of ‘4’ and ‘5’ gave identical SDRS. So is my program working? If not, is there a way to know when it is?
Here is the data:

input of 7 gives: 32 active columns: 
2008,2010,2012,2013,2014,2015,2017,2019,2021,2022,2024,2025,2027,2028,2029,2030,2031,2032,2034,2035,2036,2037,2038,2039,2040,2041,2042,2043,2044,2045,2046,2047
There are 30 values in common with the prior SDR (below this one in the textbox)

input of 6 gives: 32 active columns: 
2008,2010,2012,2013,2014,2017,2018,2019,2021,2022,2024,2025,2026,2027,2028,2029,2030,2031,2032,2035,2036,2037,2038,2039,2040,2041,2042,2043,2044,2045,2046,2047
There are 32 values in common with the prior SDR (below this one in the textbox)


input of 5 gives: 32 active columns: 
2008,2010,2012,2013,2014,2017,2018,2019,2021,2022,2024,2025,2026,2027,2028,2029,2030,2031,2032,2035,2036,2037,2038,2039,2040,2041,2042,2043,2044,2045,2046,2047
There are 32 values in common with the prior SDR (below this one in the textbox)


input of 4 gives: 32 active columns: 
2008,2010,2012,2013,2014,2017,2018,2019,2021,2022,2024,2025,2026,2027,2028,2029,2030,2031,2032,2035,2036,2037,2038,2039,2040,2041,2042,2043,2044,2045,2046,2047
There are 31 values in common with the prior SDR (below this one in the textbox)

input of 3 gives: 31 active columns: 
2008,2010,2012,2013,2014,2017,2018,2019,2021,2022,2024,2025,2026,2027,2028,2029,2030,2031,2032,2035,2036,2037,2038,2039,2040,2041,2042,2043,2044,2045,2046
6

This means there will be no overlap between any numbers, so no numbers will be semantically similar to any other numbers. Are you sure you are creating proper encodings for these numeric values to use as SP input?

Given that you are unique buckets for each number value, the SP should be giving each number a unique representation as well. Something seems wrong here.

7

I must misunderstand what ‘buckets’ means. My encoder looks like this:

  Public Shared Sub createglobalEncoderPattern(ByVal encodethis As Integer, ByRef encodedinput() As Boolean)
        Dim i As Integer
        Dim range As Double
        Dim startat As Integer

        range = ClassConstants.MaxPattern - ClassConstants.MinPattern
        startat = Math.Floor(ClassConstants.buckets * (encodethis - ClassConstants.MinPattern) / range)
        cleartheinputpattern(encodedinput)
        For i = startat To startat + ClassConstants.bitsPerPattern - 1
            encodedinput(i) = True
        Next
    End Sub

So it is saying that (for example) ''4" might be represented as:
00011111110
and “5” might be represented as:
0000 1111111
So the representations overlap.
My understanding of ‘bucket’ was that if you put consecutive numbers in the same bucket, then they cannot be distinguished at all.

8

This encoding example in the BAMI book says you can use as many buckets as values:

Here is an example of encoding the outside temperature for a location where the temperature varies between
0 and 100.
1. This minVal is 0 (degrees F) and the maxVal is 100 (degrees F).
2. The range is 100.
3. We choose to split the range into 100 buckets
9

Yes, that sounds about right.

Yes, you can do that, but that means each value will get its own bucket, and no other values will be represented in that bucket.

10

I found out one major problem with my code recreation of the spatial pooler, which is that the vb.net “random” function gives the same value repeatedly when it is called in a tight loop. So I fixed all instances, and most SDRS are different, but there still are a few that are the same. Remember I’m using a scalar encoder, so I might have two patterns that differ only the position of one bit, and those patterns activate the same columns in a few cases. I’m using 2048 columns, and their synapses are randomly initialized to connect to inputs. I could see how that might cause a problem, because there might be some bits in the encoder that simply never attach to a column, not even via a ‘potential’ synapse, and given that my input patterns only differ by the values of 2 bits, (for instance 01110 vs 00111), the algorithm would not be able to distinguish two closely related patterns such as the encodings for 5 and 6.
I did use a potential Percent of 80% so that synapses to a particular column are made only from a contiguous 80% of the encoder bits.
So possibly I would get better behavior if I reduced the percent, and increased the number of columns?
Or should every encoded value be distinguishable given the parameters that I am using?

On another topic, suppose I have 10 distinct input values, but only 2 buckets. What would this mean in practice? It would seem to me, that the first 5 distinct values are always assigned to the first bucket. Since the Spatial pooler just sees buckets, it would treat all 5 values that land in that bucket the same way, and would activate the same set of columns so matter which of those 5 had been encoded. If so, what is the value of that for a spatial encoder? Why would I want to represent the number ‘2’ exactly the same as the number ‘4’, but then suddenly represent the number ‘6’ very differently? After all, ‘6’ is as close to ‘4’ as ‘2’ is?

I’ll post my code in a separate reply below, in case you are interested.

11

How large are your encodings? I would suggest you make them a few hundred bits long.

12

My Spatial Pooler Code, minus the forms that let the user set parameters. (there are a few redundant routines in here, but its still beta)

C:\progfolder\MakeSDR\MakeSDR\ClassConstants.vb
Imports System.Text
Public Class ClassConstants
    Private Shared _previousListOfAskIndices As New List(Of Integer)
    Private Shared _encoderType As EnumEncoderType

    Public Shared CopyOfColumns() As spColumn = Nothing

    Public Shared columnCount As Integer = 2048
    Public Shared numActiveColumnsPerinhArea As Integer = 40 ' dependent on inhibition radius
    Public Shared maxBoost As Double = 1 ' for simple scenarios
    Public Shared PotentialFraction As Double = 0.8 ' dependent on how many on-bits are in a input, and how many synapses are originally connected
    Public Shared connectedPerm As Double = 0.3
    Public Shared synPermActiveInc As Double = 0.03 ' data dependent
    Public Shared synPermInactiveDec As Double = 0.015 ' data dependent
    Public Shared excludeList As List(Of Double)
    Public Shared encoderDimensions As Integer = 1
    Public Shared encodedinput() As Boolean
    '   Public Shared reconstructedInput() As Boolean = Nothing
    Public Shared stimulusThreshold As Integer = 0
    Public Shared buckets As Integer = 30
    Public Shared boostStrength As Double = 1
    Public Shared MaxPattern As Double = 30
    Public Shared MinPattern As Double = 0
    Public Shared bitsPerPattern As Integer = 21
    Public Shared sp As SpatialPoolClass

    ' calculated values:
    Public Shared inhibitionRadius As Integer
    Private Shared _encoderlength As Integer = 0



    Public Enum EnumEncoderType
        Other = 0
        GlobalEncode = 1
        localEncode = 2
    End Enum
    Public Shared Property pPreviousListOfAskIndices As List(Of Integer)
        Get
            Return _previousListOfAskIndices
        End Get
        Set(value As List(Of Integer))
            _previousListOfAskIndices = value
        End Set
    End Property
    Public Shared Property pEncoderType As EnumEncoderType
        Get
            Return _encoderType
        End Get
        Set(value As EnumEncoderType)
            _encoderType = value
        End Set
    End Property
    Public Shared Property pEncoderLength As Integer
        Get
            Return _encoderlength
        End Get
        Set(value As Integer)
            _encoderlength = value
            inhibitionRadius = value
            ReDim encodedinput(_encoderlength)
        End Set
    End Property
    Public Shared Function CalculateEncoderLength() As Integer
        Dim n As Integer

        n = ClassConstants.buckets + ClassConstants.bitsPerPattern - 1
        Return n
    End Function
    Public Shared Sub createglobalEncoderPattern(ByVal encodethis As Integer, ByRef encodedinput() As Boolean)
        Dim i As Integer
        Dim range As Double
        Dim startat As Integer

        range = ClassConstants.MaxPattern - ClassConstants.MinPattern
        startat = Math.Floor(ClassConstants.buckets * (encodethis - ClassConstants.MinPattern) / range)

        cleartheinputpattern(encodedinput)

        For i = startat To startat + ClassConstants.bitsPerPattern - 1
            encodedinput(i) = True
        Next

    End Sub

    Public Shared Sub cleartheinputpattern(ByRef encodedinput() As Boolean)
        Dim i As Integer
        For i = 0 To encodedinput.Length - 1
            encodedinput(i) = False
        Next
    End Sub
    Public Shared Sub DisplayEncodedInput()
        Dim sb As New StringBuilder("")
        For i = 0 To encodedinput.Length - 1
            If i > 0 Then
                sb.Append(",")
            End If
            If encodedinput(i) Then
                sb.Append("1")
            Else
                sb.Append("0")
            End If
        Next
        MessageBox.Show(sb.ToString)
    End Sub

    Public Shared Function median(ByVal thelist As List(Of Integer)) As Double
        thelist.Sort()
        Dim thecount As Double = thelist.Count
        thecount = thecount / 2
        If thecount = 1 Then
            Return thelist(0)
        End If

        If isEven(thecount) Then
            Return (thelist(thecount) + thelist(thecount - 1)) / 2
        Else
            Return thelist(Math.Floor(thecount))
        End If
    End Function
    Public Shared Function isEven(ByVal theCount As Double) As Boolean
        If theCount / 2 = Math.Floor(theCount / 2) Then
            Return True
        Else
            Return False
        End If
    End Function
End Class




C:\progfolder\MakeSDR\MakeSDR\SpatialPoolClass.vb
Public Class SpatialPoolClass

    Public columns() As spColumn
    Public activeColumnIndices As List(Of Integer)
    Public PotentialAreaMagnitude As Integer
    Public Sub New()
        ' every dendrite should have synapses connected to the input encoder.   Their permanences should be set.

        Dim HowManySynapses As Integer
        Dim i As Integer
        Dim j As Integer
        Dim r As New Random

        If ClassConstants.pEncoderType = ClassConstants.EnumEncoderType.GlobalEncode Then
            PotentialAreaMagnitude = Math.Round(ClassConstants.PotentialFraction * ClassConstants.pEncoderLength)
            HowManySynapses = ClassConstants.bitsPerPattern * 0.5
            ReDim columns(ClassConstants.columnCount - 1)

            For i = 0 To ClassConstants.columnCount - 1
                Dim c As New spColumn(i, r)
                For j = 0 To HowManySynapses
                    c.addNewSynapse(r, PotentialAreaMagnitude)
                Next
                c.initPermanences(r)
                c.pDendrite.Sort()
                columns(i) = c
            Next

        Else
            Throw New Exception("Spatial Pooler 'Initialize' routine has not been implemented for non-scalar input-encodings")
        End If


    End Sub
    Sub ComputeOverlapWithInput()
        Dim c As spColumn
        Dim s As Synapse
        Dim numones As Integer

        For Each c In columns
            numones = 0
            For Each s In c.pDendrite
                If s.permanence >= ClassConstants.connectedPerm Then
                    If s.active Then
                        numones = numones + 1
                    End If
                End If
            Next

            c.pOverlap = numones * c.pBoost

        Next
    End Sub


    Sub UpdateSynapsePermanenceAndMore()
        Dim index As Integer
        Dim c As spColumn
        Dim s As Synapse
        Dim ov As Double
        Dim activeMovingAverageNeighbors As Double
        Dim sumDuty As Double = 0

        For Each index In activeColumnIndices
            c = columns(index)
            For Each s In c.pDendrite
                If s.active Then
                    s.permanence = s.permanence + ClassConstants.synPermActiveInc
                    If s.permanence > 1 Then
                        s.permanence = 1
                    End If
                Else
                    s.permanence = s.permanence - ClassConstants.synPermInactiveDec
                    If s.permanence < 0 Then
                        s.permanence = 0
                    End If
                End If
            Next
        Next

        If ClassConstants.pEncoderType = ClassConstants.EnumEncoderType.GlobalEncode Then
            sumDuty = GetSumMovingAverages()
        End If
        For Each c In columns
            c.pActiveMovingAverage = c.updateActiveMovingAverage()
            If ClassConstants.pEncoderType = ClassConstants.EnumEncoderType.GlobalEncode Then
                activeMovingAverageNeighbors = (sumDuty - c.pActiveMovingAverage) / (columns.Length - 1)
            Else
                activeMovingAverageNeighbors = getNeighborsActiveMovingAverage(c.pColindex)
            End If

            Dim MinMovingAverage As Double = 0.01 * maxMovingAverage(getNeighbors(c.pNeighborIndices))

            c.pBoost = c.boostFunction(activeMovingAverageNeighbors)
            ov = c.UpdateOverlapMovingAverage()
            If ov < MinMovingAverage Then
                c.increasePermanences(0.1 * ClassConstants.connectedPerm)
            End If
        Next
        If ClassConstants.pEncoderType = ClassConstants.EnumEncoderType.GlobalEncode Then
            ClassConstants.inhibitionRadius = ClassConstants.pEncoderLength
        Else
            ClassConstants.inhibitionRadius = averageReceptiveFieldSize()
        End If

    End Sub
    Function getNeighborsActiveMovingAverage(ByVal cColindex As Integer) As Double
        Dim i As Integer
        Dim sum = 0
        For i = 0 To columns.Count - 1
            If i = cColindex Then
                Continue For
            End If
            sum = sum + columns(i).pActiveMovingAverage
        Next
        Return sum / (columns.Length - 1)
    End Function
    Function GetSumMovingAverages() As Double
        Dim i As Integer
        Dim sum = 0
        For i = 0 To columns.Count - 1

            sum = sum + columns(i).pActiveMovingAverage
        Next
        Return sum
    End Function
    Function getNeighbors(ByRef neighborindices As List(Of Integer)) As List(Of spColumn)
        Dim nc As New List(Of spColumn)
        Dim i As Integer
        For i = 0 To neighborindices.Count - 1
            nc.Add(columns(i))
        Next
        Return nc
    End Function
    Public Sub ComputeOnce(ByVal train As Boolean)
        ComputeOverlapWithInput()
        '  MessageBox.Show(classPrint.printOverlaps(15))
        ComputeWinningColumns()
        If train Then
            UpdateSynapsePermanenceAndMore()
        End If
    End Sub


    Function kthScore(ByRef cols As List(Of spColumn), ByVal k As Integer, ByVal colindex As Integer) As Double
        ' return k'th highest overlap value.  
        ' this assumes that overlap was calculated earlier for each column in the list
        ' it assumes k starts at 1
        ' it assumes columns are sorted by activity in increasing order
        Dim i As Integer
        Dim j As Integer

        If ClassConstants.pEncoderType = ClassConstants.EnumEncoderType.GlobalEncode Then

            If cols.Count = 0 Then
                Return 0
            End If
            i = cols.Count - k
            If i < 0 Then
                Return 0
            End If
            If colindex > -1 Then
                For j = i To cols.Count - 1
                    If cols(j).pColindex = colindex Then
                        i = i - 1
                        Exit For
                    End If
                Next
            End If
        Else
            Throw New Exception("kth score not implemented for non global case")
        End If

        Return cols(i).pOverlap
    End Function


    Public Function getMinimumOverlapOfActiveCols() As Integer
        Dim collist As List(Of spColumn)
        collist = columns.ToList
        collist.Sort()
        Dim li As List(Of Integer)
        Dim colindex As Integer
        li = GetActiveColumnsUsingkthScoreGlobal(collist)
        If li.Count = 0 Then
            Throw New Exception("no active columns in getMinimumOverlapOfActiveCols")
        End If
        colindex = li(0)
        Return columns(colindex).pOverlap
    End Function
    Public Function GetActiveColumnsUsingkthScoreGlobal(collist As List(Of spColumn)) As List(Of Integer)
        Dim activelist As New List(Of Integer)
        Dim startI As Integer
        Dim i As Integer
        If collist.Count = 0 Then
            Return activelist
        End If
        startI = collist.Count - ClassConstants.numActiveColumnsPerinhArea
        If startI < 0 Then
            startI = 0
        End If
        For i = startI To collist.Count - 1
            If collist(i).pOverlap = 0 Then
                Continue For
            End If
            activelist.Add(i)
        Next
        Return activelist
    End Function
    Sub ComputeWinningColumns()
        ' assuming global inhibition area.
        Dim c As spColumn
        Dim minLocalActivity As Double
        Dim collist As List(Of spColumn)
        Dim index As Integer
        Dim locallist As New List(Of Integer)

        activeColumnIndices = New List(Of Integer)
        If ClassConstants.pEncoderType = ClassConstants.EnumEncoderType.GlobalEncode Then
            collist = columns.ToList
            collist.Sort()
            locallist = GetActiveColumnsUsingkthScoreGlobal(collist)
            ' the above gets the top 10 columns (in overlap)

            For Each c In columns
                c.pActive = False
            Next
            For Each index In locallist
                c = columns(index)

                If c.pOverlap > ClassConstants.stimulusThreshold And c.pOverlap >= minLocalActivity Then
                    c.pActive = True
                    activeColumnIndices.Add(c.pColindex)
                End If
            Next
        Else
            Throw New Exception("'ComputeWinningColumns' is not implemented for non-global case.")
        End If
    End Sub


    Public Function averageReceptiveFieldSize() As Double
        Dim c As spColumn
        Dim sum As Double

        For Each c In columns
            sum = sum + c.pReceptiveFieldSize
        Next
        Return (sum / columns.Count)
    End Function
    Function maxMovingAverage(ByVal cols As List(Of spColumn)) As Double
        Dim i As Integer

        Dim maxdc As Double = 0
        For i = 0 To cols.Count - 1
            If maxdc < cols(i).pMovingAverageOfActivity Then
                maxdc = cols(i).pMovingAverageOfActivity
            End If
        Next
        Return maxdc
    End Function
End Class

C:\progfolder\MakeSDR\MakeSDR\spColumn.vb
Imports MakeSDR
Imports System.Runtime.Serialization.Formatters.Binary
Imports System.IO
Imports System.Text
<Serializable()>
Public Class spColumn
    Implements IComparable(Of spColumn)

    Private _boost As Double = 1 ' to start
    Private _overlap As Integer
    Private _movingAverageOfActivity As Double
    Private _movingAverageOfOverlap As Double
    Private _active As Boolean
    Private _activeCbuf As New CircularBufferBoolean(1000)
    Private _overlapCbuf As New CircularBufferInteger(1000)
    Private _dendrite As New List(Of Synapse)
    Private _colindex As Integer
    Private _activeMovingAverage As Double
    Private PotentialAreaStartIndex As Integer = 0
    '   Private _naturalCenter As Integer



    Public Sub New(ByVal colindex As Integer, ByRef theRandom As Random)


        ' Dim bitspercol As Double

        PotentialAreaStartIndex = theRandom.Next(0, ClassConstants.pEncoderLength)
        _colindex = colindex

        ' comment out for now:
        'If ClassConstants.pEncoderType = ClassConstants.EnumEncoderType.GlobalEncode Then
        '    
        '    ' If ClassConstants.columnCount = ClassConstants.pEncoderLength Then
        '    '     _naturalCenter = colindex
        '    'Else
        '    ' bitspercol = ClassConstants.pEncoderLength / ClassConstants.columnCount
        '    '      _naturalCenter = bitspercol * colindex + bitspercol / 2.0 ' wait, not have PotentialAreaStartIndex and Area
        '    'End If

        'Else
        '    Throw New Exception("have not implemented 'natural center' for non-global case")
        'End If

    End Sub

    Public Sub addNewSynapse(ByRef r As Random, ByVal PotentialAreaMagnitude As Integer)
        Dim se As New Synapse
        Dim sii As Integer

        With se

            Do While True
                sii = r.Next(PotentialAreaStartIndex, PotentialAreaMagnitude + PotentialAreaStartIndex)
                If sii >= ClassConstants.pEncoderLength Then
                    sii = sii - ClassConstants.pEncoderLength
                End If
                If Not DendriteAlreadyContainsInput(sii) Then
                    Exit Do
                End If
            Loop
            .sourceInputIndex = sii
        End With
        _dendrite.Add(se)
    End Sub
    Public Function DendriteAlreadyContainsInput(ByVal sourceindex As Integer) As Boolean
        Dim i As Integer
        For i = 0 To pDendrite.Count - 1
            If pDendrite(i).sourceInputIndex = sourceindex Then
                Return True
            End If
        Next
        Return False
    End Function


    Public Function printDendrite() As String
        Dim sb As New StringBuilder("")
        Dim i As Integer
        For i = 0 To _dendrite.Count - 1

            sb.AppendLine(_dendrite(i).printself)
        Next
        Return sb.ToString
    End Function
    Public Sub initPermanences(ByRef r As Random)
        ' _naturalCenter = getNaturalCenter()
        Dim i As Integer
        Dim nd As Double

        '  Dim distance As Double
        For i = 0 To _dendrite.Count - 1
            With _dendrite(i)
                ' distance = Math.Abs(.sourceInputIndex - _naturalCenter) / (ClassConstants.columnCount - 1)

                nd = r.NextDouble ' double >= 0, but less than 1
                nd = nd / 5  ' 0 to .2
                If nd > ClassConstants.connectedPerm Then
                    nd = ClassConstants.connectedPerm / 2
                End If
                'distance = Math.Ceiling(distance * 10)
                'If distance > 10 Then
                '    distance = 10
                'End If

                .permanence = ClassConstants.connectedPerm - 0.1 + nd ' + 0.02 / distance
            End With
        Next
    End Sub


    Public ReadOnly Property pReceptiveFieldSize As Integer
        Get
            Dim i As Integer
            Dim sum As Integer = 0
            For i = 0 To _dendrite.Count - 1
                If _dendrite(i).permanence >= ClassConstants.connectedPerm Then
                    sum = sum + 1
                End If
            Next
            Return sum
        End Get
    End Property

    Public Sub increasePermanences(ByVal scalefactor As Double)
        Dim i As Integer
        For i = 0 To _dendrite.Count - 1
            With _dendrite(i)
                .permanence = .permanence * (1 + scalefactor)
            End With
        Next
    End Sub
    Public Function boostFunction(ByVal ActiveMovingAverageNeighbors) As Double
        ' if activity of column equals activity of neighbors, return 1
        ' if activity of column is greater than activity of neighbors, return a number less than 1
        ' otherwise, return a number greater than 1. (at most it could be 'e' which is about 2.7183))
        If ClassConstants.pEncoderType <> ClassConstants.EnumEncoderType.GlobalEncode Then
            Throw New Exception("boostFunction is not implemented for columns that do not have global inputs")
        End If
        Dim diff As Double
        Dim expdiff As Double

        diff = pMovingAverageOfActivity - ActiveMovingAverageNeighbors
        expdiff = Math.Exp(diff * ClassConstants.boostStrength * -1)

        Return expdiff
    End Function
    Public Property pBoost As Double
        Get
            Return _boost
        End Get
        Set(value As Double)
            _boost = value
        End Set
    End Property
    Public Function updateActiveMovingAverage() As Double
        ' some routine before this must have set .pactive, or this will not work
        AddActiveValue()
        Return pMovingAverageOfActivity
    End Function

    Public Function UpdateOverlapMovingAverage() As Double
        ' some routine before this must have set .poverlap, or this will not work
        AddOverlap()
        Return pMovingAverageOfOverlap
    End Function

    Sub New()

    End Sub

    Public Function DeepCopyMe() As spColumn

        Dim m As MemoryStream = New MemoryStream()
        Dim b As BinaryFormatter = New BinaryFormatter()
        b.Serialize(m, Me)
        m.Position = 0
        Return CType(b.Deserialize(m), spColumn)
    End Function
    Public ReadOnly Property pDendrite As List(Of Synapse)
        Get
            Return _dendrite
        End Get
    End Property
    Public ReadOnly Property pColindex As Integer
        Get
            Return _colindex
        End Get
    End Property
    Public ReadOnly Property pNeighborIndices As List(Of Integer)
        Get
            Dim neighborindices As New List(Of Integer)
            If ClassConstants.pEncoderType = ClassConstants.EnumEncoderType.GlobalEncode Then
                Dim newlist As New List(Of Integer)
                Dim i As Integer
                For i = 0 To ClassConstants.columnCount - 1
                    If i = pColindex Then
                        Continue For
                    End If
                    newlist.Add(i)
                Next
                neighborindices = newlist
                Return neighborindices
            Else
                Throw New Exception("pNeighborIndices property has been coded only for scalar encodings (global)")
            End If

        End Get

    End Property
    Public Property pOverlap As Integer
        Get
            Return _overlap
        End Get
        Set(value As Integer)
            _overlap = value
        End Set
    End Property
    Public Property pActive As Boolean
        Get
            Return _active
        End Get
        Set(value As Boolean)
            _active = value
        End Set
    End Property
    Public ReadOnly Property pMovingAverageOfActivity As Double
        Get
            _movingAverageOfActivity = _activeCbuf.MovingAverage
            Return _movingAverageOfActivity
        End Get

    End Property
    Public ReadOnly Property pMovingAverageOfOverlap As Double
        Get
            _movingAverageOfOverlap = _overlapCbuf.MovingAverage(ClassConstants.stimulusThreshold)
            Return _movingAverageOfOverlap
        End Get

    End Property
    Public Sub AddActiveValue()
        _activeCbuf.Addbuf(pActive)
    End Sub
    Public Sub AddOverlap()
        _overlapCbuf.Addbuf(pOverlap)
    End Sub
    Public Function CompareTo(other As spColumn) As Integer Implements IComparable(Of spColumn).CompareTo
        If _overlap = other.pOverlap Then
            Return 0
        Else
            If _overlap < other.pOverlap Then
                Return -1
            Else
                Return 1
            End If
        End If

    End Function


    Public Property pActiveMovingAverage As Double
        Get
            Return _activeMovingAverage
        End Get
        Set(value As Double)
            _activeMovingAverage = value
        End Set
    End Property
    Private Function getNaturalCenter() As Integer
        Dim i As Integer
        Dim listindices As New List(Of Integer)
        For i = 0 To pDendrite.Count - 1
            listindices.Add(pDendrite(i).sourceInputIndex)
        Next
        Return ClassConstants.median(listindices)
    End Function
    Public Function showColumnFields() As String
        Dim sb As New StringBuilder("")

        sb.AppendLine("column # " & _colindex)
        sb.AppendLine("boost is " & Math.Round(_boost, 2))
        sb.AppendLine("overlap is " & _overlap)
        sb.AppendLine("movingAverageOfActivity is " & Math.Round(_movingAverageOfActivity, 2))
        sb.AppendLine("movingAverageOfOverlap is " & Math.Round(_movingAverageOfOverlap, 2))
        sb.AppendLine("_active is " & _active)
        sb.AppendLine("_activeCbuf is " & _activeCbuf.pMostRecentValues)
        sb.AppendLine("_overlapCbuf is " & _overlapCbuf.pMostRecentValues)
        sb.AppendLine("dendrite follows: ")
        sb.AppendLine(printDendrite())
        sb.AppendLine("PotentialAreaStartIndex is " & PotentialAreaStartIndex)

        Return sb.ToString
    End Function
End Class


C:\progfolder\MakeSDR\MakeSDR\Synapse.vb
<Serializable()>
Public Class Synapse
    Implements IComparable(Of Synapse)

    Public permanence As Double
    Public sourceInputIndex As Integer

    Public Function active() As Boolean
        If ClassConstants.encodedinput(sourceInputIndex) Then
            Return True
        Else
            Return False
        End If
    End Function
    Public Function printself() As String
        Return ("input[" & sourceInputIndex & "] - permanence is " & Math.Round(permanence, 1))
    End Function
    Public Function CompareTo(other As Synapse) As Integer Implements IComparable(Of Synapse).CompareTo
        If sourceInputIndex = other.sourceInputIndex Then
            Return 0
        Else
            If sourceInputIndex < other.sourceInputIndex Then
                Return -1
            Else
                Return 1
            End If
        End If

    End Function
End Class
13

I honestly don’t have time to review your code. But I’m going to move this topic from #nupic into #htm-hackers. Perhaps someone there will help you out.