Why do we need AGI?

No, I think you misunderstand. Yes, indeed, if you have an algorithm that can be executed by one TM (including the TM itself) you can implement that algorithm in another TM.

But if you have a TM of unknown design executing an unknown algorithm you cannot emulate that by observing its behaviour alone. In general you cannot derive the TM from what it does, so you cannot emulate it. HTM and other research are attempts to study and unravel the internals of the TM. There is no shortcut.

Anything that humans can do reliably—better than chance—could also be done reliably by a hypothetical brain simulation, even if that simulation did not have access to the unpredictable chaos and randomness of my actual real-world brain. Right?

No argument there. But chaos means you cannot get the same outcome as some other brain would get, given what might seem to be the same starting conditions. An AGI will read that paper and learn from it, but not learn exactly what you would learn. Brains live on the edge of chaos: never the same thing twice, just near enough that it works (kind of).

As an aside, this is my argument against predestiny: chaos forbids it. One electron misplaced is enough…:blush:

1 Like