Edelman’s Neural Darwinism came up in this thread because @DanML responded to my arguments - that a failure to recognize contradictory, even chaotic, structure is what has been holding us back in AI - by mentioning similar themes in György Buzsáki’s work:
I looked up György Buzsáki and said his work reminded me of Gerald Edelman’s Neural Darwinism.
@Bitking then recommended Calvin’s book on another variant of Neural Darwinism, with many of the same themes of expanding, growing, changing, structure, even chaos, that had been the basis of things he had been working on for HTM.
So basically Neural Darwinism came up in this thread because I am arguing that contradictory structure may be what we are missing in our attempts to code “meaning” in cognition. And that lead to some links to similar ideas, notably growing or contradictory, even chaotic, representation, in Neural Darwinism.
I agree with the themes of growing structure, and especially chaos, in Neural Darwinism. But I disagree with the specific mechanism it proposes for finding new structure.
I’m arguing in this thread that we don’t need to have the random variation followed by evolutionary selection of Neural Darwinism. Rather I think existing “learning” methods, specifically the cause and effect generalization of transformers and Large Language Models, is already telling us the structuring mechanism we need. The only thing which is holding us, and transformers, back, is that we still assume the structure to be found is static and can be learned.
I’m arguing that the blow out in the number of parameters “learned” by transformers is actually indicating that they are generating, growing, contradictory, and even chaotic, structure already. So the only mistake is that we are trying to “learn” all of it at once. Rather we should be focusing on its chaotic generator character, and generating the structure which is appropriate to each new prompt/context, at run time.
And I’m suggesting we can do that, by seeking cause-effect prediction maximizing structure in “prompt” specific resonances in a network of sequences, initially language sequences. This in contrast to transformers, which look for these cause-effect prediction maximizing structures, by trying to follow prediction entropy gradients to static minima, and then only selecting between potentially contradictory structures learned, using a prompt, at run time.