Connecting Hinton's capsules to Numenta research

This morning, Marcus is planning on discussing capsules on the whiteboard, connecting them to our work.

Here are 3 Hinton capsules papers and 1 talk.

1_uItEGzY1I9NK6hl1u4hPYg

Watch live at 10AM PDT on https://www.twitch.tv/rhyolight_. I will post the video here afterwards as always.

3 Likes
2 Likes

Nice to see this addressed by you guys. You didn’t mention what Hinton calls doing “inverse graphics”, it’s in the video link I think. Hinton seems to have the idea of objects being represented in some archetypical form by incorporating the spatial prior into the network architecture. The 4x4 dimensionality of the matrices are for sure inspired by the affine transformation matrices of 3D space, though it’s never motivated as such anywhere in the paper. I suppose it’s a more constrained computation than what you are proposing is possible with the grid cell like stuff in the minicolumns. Anyway, there are many similarities, and I think it’s exciting to see this kind of bridging between your approaches and fields, with none less than Hinton on the other end.

4 Likes

Just came out: https://arxiv.org/abs/1906.06818
“An object can be seen as a geometrically organized set of interrelated parts. A system that makes explicit use of these geometric relationships to recognize objects should be naturally robust to changes in viewpoint, because the intrinsic geometric relationships are viewpoint-invariant. We describe an unsupervised version of capsule networks, in which a neural encoder, which looks at all of the parts, is used to infer the presence and poses of object capsules. The encoder is trained by backpropagating through a decoder, which predicts the pose of each already discovered part using a mixture of pose predictions. The parts are discovered directly from an image, in a similar manner, by using a neural encoder, which infers parts and their affine transformations. The corresponding decoder models each image pixel as a mixture of predictions made by affine-transformed parts. We learn object- and their part-capsules on unlabeled data, and then cluster the vectors of presences of object capsules. When told the names of these clusters, we achieve state-of-the-art results for unsupervised classification on SVHN (55%) and near state-of-the-art on MNIST (98.5%).”

3 Likes