Simple Change for Implementing Focus in the HTM

I am proposing the addition of real variables for the prediction output, instead of a boolean flag. This uses a fair amount of extra memory in scaling systems, but only a digit or two of precision will be necessary (proof needed). My justification is: such a variable (for the strength of the prediction) would allow the HTM to focus on areas which are predicting very strongly, or when something unexpected happens.

To actually focus, it is as simple as boosting columns that are predicting strongly, for the next time step alone. Columns which are expected strongly to do something (if any columns are) will be given preference for the spatial pooler by boosting their overlaps. This has the effect of preferring “features” that the columns are looking for when the columns expect those features to be active. If the system has been successfully predicting things, it eventually pays attention only to regions that it is “interested” in.

This also has the added benefit of aiding the process of predictions from predictions (PP). Weights for predictive cells can be assigned based on (a) overlap of the active segment and (b) the levels of predictions required before a “primary” prediction is found - a primary prediction being a cell segment that is connecting to currently active cells, not currently predicting cells. (There should be better ways than crawling along segments to determine roughly how long until a cell actually thinks it will be active, by the way.) Using the weight of the prediction, you can tell if the cell will be active really soon, or a several or more time steps ahead.

This is my proposal: The addition of real numbers (say from 0.0 to 1.0) assigned to predicting cells, instead of a boolean flag.


1 Like