Hierarchy & Invariance

[gmirey]

@sebjwallace

The supervision is not coming from any external teacher, it is simply coming from the region above.

@sebjwallace

This kind of reminds me of back-propagation in ANNs except that the cortex is the teacher/supervisor, not another human. It is just mapping. Given a set of different inputs you want a single output

I’m feeling there are lots of collisions between our ideas currently.
Even if not "yes, but"ing at the exact same things you and me, it seems we end up with the same intuition.

@gmirey

I was thinking about that very subject quite recently.
One way to have that teacher effect taken into account, for our online-learning models, is:
[…]

• we’d need to replace the ‘error to expected output’+backprop of classic SL as used by DL, with something else
• this something else is simply the presentation of a pre-formed SDR (=‘expected output’), somewhere into the scheme
  […]
• since this is an ideal, an abstraction, the ‘somewhere’ is likely an higher area