Object modeling pre- and post- 9/21 Hawkins talks

In September, Jeff gave two talks and led a discussion on object modeling. In my view these talks are of very great significance. Yet, on this forum I can find no discussion of the important points made in these talks (unless I didn’t look in the right place – I am new to the forum).

The column pre-9/21 is described in three papers: columns paper, columns+ paper, frameworks paper

Three processing layers are defined in the columns paper – grid cells, sensory layer output layer, although the focus in the columns paper is on the sensory and output layers. A simulation model was implemented and the paper gives simulation results for synthetic data. The simulations demonstrate a model that works.

The columns+ paper focuses on grid cells and the sensory layer. A simulation model was implemented and the paper gives simulation results for synthetic data.

The frameworks paper proposes displacement cells which compute the displacement between two locations. The operation of displacement cells is not demonstrated via simulations.

Key features of the column model as they appeared pre-9/21:

  1. Objects are defined as a set of feature/location pairs.

  2. The grid is implemented as a number of modules, which are based on a rhombus-like grids of different scales and orientations.

  3. Displacements are determined via computation using pairs of locations as inputs.

  4. The output layer can identify an object (columns paper) and the sensory and grid layers can converge to a known location (columns+ paper). Collectively – both an object and the location within the object can be identified.

After the 9/21 talks:

  1. Objects are defined as a set of sub-object/displacement pairs (a directed graph), not feature/location pairs.

  2. A multi-module grid is wrong – among other reasons: it is too complicated. Grid cells are still present, but their role is not clear. Are they some form of “scratchpad”?

  3. Displacements are maintained directly rather than being computed based on locations.

  4. The output layer as previously considered is no longer part of the model.

In my view, all the proposed changes put forward in the 9/21 talks appear to be significant improvements. Conversely, many key parts of the original column model are no longer considered viable.

To summarize: before the 9/21 talks there was a published column model that could be demonstrated via working simulations. After the 9/21 talks, there is no published, working column model.

It has been several months since the 9/21 talks – has anyone successfully constructed and demonstrated a new column model that contains all the proposed improvements?

4 Likes

Hi @jes. Welcome to the forum.

Indeed, I liked those presentations a lot too, and maybe it’s a shame that we didn’t discuss them on the forum. (Personally I’ve talked about some elements off-forum with other people).

But you have to take into account that these are highly speculative talks. @jhawkins regularly reminds us that all the research meetings are very work-in-progress. So I wouldn’t discount the so-called pre-9/21 key features.

vs

A feature and a sub-object, as far as I understood throughout the many papers, was always the same thing. If you consider that the neocortex has to make sense of the world without knowing in which sensory environment it is put (visual / sonar / tactile / abstract / …), then what is a feature in one sensory modality is very different than a feature in another. And according to Numenta (based on findings by Mountcastle) the neocortex and its sub-elements should deal with those in the same way. This is a type of hierarchy that is fundamentally different from the hierarchy that is typically offered in neurology.

Also, a location is always a displacement from a reference point. In earlier presentations there have been discussions about allocentric vs egocentric references, and different types of references in different layers.

Grid cells have always been mysterious. In earlier presentations @jhawkins and @mrcslws in particular were never satisfied with their working theories.

I don’t know for sure, but during this paper review serious hints were dropped that Numenta is about to present new work. Check out timestamps 39:06 and 1:00:26.

Exciting times ahead. :-).

3 Likes