Just sharing some interesting transformer based research that ties in intelligence to complexity.
The research states that “intelligence arises from the ability to predict complexity and that creating intelligence may require only exposure to complexity…” Doesn’t matter if the complexity emerges from some arbitrary rules with no real world bearings as long as it generates a complex system. “… Our findings reveal that rules with higher complexity lead to models exhibiting greater intelligence, as demonstrated by their performance on reasoning … tasks. … We conjecture that intelligence arises from the ability to predict complexity and that creating intelligence may require only exposure to complexity.”
The complexity of the rules has a sweet spot: “… uniform and periodic systems, and often also highly chaotic systems, resulted in poorer downstream performance, highlighting a sweet spot of complexity conducive to intelligence.”
The transformer trained on any complex systems must’ve learned reusable mini processes that could be used to predict any other complex systems. That’s my take on it anyways.