I used the pdf that numenta supplies which explains the pseudocode of the spatial pooler to see if I could write a quick version in vb.net. In the process, I ran into a few ambiguities. One of them was with the following line:
inhibitionRadius = averageReceptiveFieldSize()
Lets assume I am using a scalar encoder to encode some numbers, and then I pass the output of the scalar encoder to the spatial pooler. I would think that there is no topology in this specific case, so that any particular column’s neighbors would encompass all columns (except itself) in the spatial pooler. I would also think that the inhibition would be global too - the inhibition radius would take in all the columns. Is that true?
I’ll paste one class from my program to give a flavor of what I’m doing. Nothing has been tested yet, but if it works (I am using to test an idea), I could post all the code here. Here is one class:
Public Class SpatialPoolClass Private columns() As spColumn Private activeColumnIndices As List(Of Integer) Sub ComputeOverlapWithInput() Dim c As spColumn Dim s As Synapse Dim numones As Integer For Each c In columns numones = 0 For Each s In c.pDendrite If s.permanence < ClassConstants.connectedPerm Then Continue For End If If ClassConstants.encodedinput(s.sourceInputIndex) Then numones = numones + 1 End If Next If numones < ClassConstants.stimulusThreshold Then c.pOverlap = 0 Else c.pOverlap = numones * c.pBoost End If Next End Sub Sub ComputeWinningColumns() ' assuming global inhibition area. Dim c As spColumn Dim minLocalActivity As Double Dim collist As List(Of spColumn) collist = columns.ToList activeColumnIndices = New List(Of Integer) For Each c In columns minLocalActivity = kthScore(collist, ClassConstants.numActiveColumnsPerinhArea, c.pColindex) If c.pOverlap > 0 And c.pOverlap >= minLocalActivity Then activeColumnIndices.Add(c.pColindex) End If Next End Sub Sub UpdateSynapsePermanenceAndMore() Dim index As Integer Dim c As spColumn Dim s As Synapse For Each index In activeColumnIndices c = columns(index) For Each s In c.pDendrite If s.active Then s.permanence = s.permanence + ClassConstants.synPermActiveInc If s.permanence > 1 Then s.permanence = 1 End If Else s.permanence = s.permanence - ClassConstants.synPermInactiveDec If s.permanence < 0 Then s.permanence = 0 End If End If Next Next Dim ov As Double For Each c In columns c.pMinDutyCycle = 0.01 * maxDutyCycle(c.pNeighbors) c.pActiveDutyCycle = c.updateActiveDutyCycle() c.pBoost = c.adjustBoost ov = c.UpdateOverlapDutyCycle() If ov < c.pMinDutyCycle Then c.increasePermanences(0.1 * ClassConstants.connectedPerm) End If Next ClassConstants.inhibitionRadius = averageReceptiveFieldSize() End Sub Sub ComputeOnce() ComputeOverlapWithInput() ComputeWinningColumns() UpdateSynapsePermanenceAndMore() End Sub Sub Initialize(ByRef inputVector() As Boolean) ' every dendrite should have synapses connected to the input encoder. Their permanences should be set. Dim HowManySynapses As Integer HowManySynapses = ClassConstants.bitsPerPattern * 0.5 ReDim columns(ClassConstants.columnCount - 1) Dim i As Integer Dim j As Integer For i = 0 To ClassConstants.columnCount - 1 Dim c As New spColumn(i) For j = 0 To HowManySynapses c.addNewSynapse() Next columns(i) = c Next End Sub Function kthScore(ByRef cols As List(Of spColumn), ByVal k As Integer, ByVal colindex As Integer) As Double ' return k'th highest overlap value. Sideeffect is to sort the columns. ' this assumes that overlap was calculated earlier for each column in the list ' it assumes k starts at 1 Dim i As Integer Dim j As Integer cols.Sort() If cols.Count = 0 Then Return 0 End If i = cols.Count - k If i < 0 Then Return 0 End If If colindex > -1 Then For j = i To cols.Count If cols(j).pColindex = colindex Then i = i - 1 Exit For End If Next End If Return cols(i).pOverlap End Function Public Function averageReceptiveFieldSize() As Double Dim c As spColumn Dim sum As Double For Each c In columns sum = sum + c.preceptivefieldsize Next Return (sum / columns.Count) End Function Function maxDutyCycle(ByVal cols As List(Of spColumn)) As Double Dim i As Integer Dim maxdc As Double = 0 For i = 0 To cols.Count - 1 If maxdc < cols(i).pMovingAverageOfDutyCycle Then maxdc = cols(i).pMovingAverageOfDutyCycle End If Next Return maxdc End Function End Class