Distributed Representations (1984) by Hinton

Ultimate representations are definitely distributed in the brain, the only way to localize them is in local memory, separated from processing. Which neurons simply don’t have, but we don’t need to emulate this handicap. Encapsulation in the brain is probably similar to localization in your mapping:

only I think this mapping may be many-to-many, not just one map to another map. But hierarchy is a separate issue. I didn’t go through the whole thread, but it seems to focus on lower areas: V1-to-V2. Organization doesn’t have to be hierarchical everywhere, hierarchical mappings interspersed with and on top of modality-specific lateral mappings would make the whole cortex hierarchical.

Well, how do you think this crazy quilting emerged? It’s not really about “spreading of representation” so much as spreading search: learning that formed those quilted mappings, I don’t mean learning in the last hour, but also during early development, if not evolution. Mappings that spread far from input areas must’ve been learned by long-range search, hence hierarchical spreading. The problem is that search must locally restricted at any given step of this expansion, else you get combinatorial explosion. But most of this macro-structural learning is over by end of adolescence, what we see in the brain is fossilized results.

I think convergence here also mediates divergence: broad concepts contain multiple levels of detail, which would also be transferred from lower to higher areas. I don’t think there is any dispute that higher areas, especially dlPFC and inferior parietal cortex, represent such broad concepts. And they can only be discovered by search across accumulated experience, even if this search is continuously pruned or localised. Going further, I think default mode network is hierarchically higher, in generality of it’s contents, than task-positive network. But it’s pretty hard to study.

3 Likes