On what basis did you (or Roko) choose a set of features, capabilities and/or motivations for a hypothetical future AGI?
Isn’t the simplest possible counter-argument that you chose wrong?
1 Like
But isn’t that the essence of the sweet illusion? Your intentions and predictions are based on passed and stored experiences, applied to the current state of the world. There is no escaping the chain. And the awareness of the resulting decision is also reactionary.
If this is correct, then a superintelligence should know this.
I guess you are assuming a clockwork (Galilean) universe. Since Newton that has been discredited (gravity acts at a distance). But we continue to base modernity on that assumption.
Most of us did not receive an education that takes into account knowledge that contradicts modern assumptions. That is one way modern traditions continue to dominate. We don’t even get taught about the implications of Newton’s work (destroying any hope of a mechanical universe). Science is all about the fact that we don’t know what is going on - but Western society is all about teaching citizens to believe somebody else (inevitably a scientist) does understand what is going on 
I assume that the micro uncertainty principle has manifestations at macro scale.
Multiplied by the butterfly effect - this makes a knowable clockwork universe unlikely.
I’m not sure how to address this. How does it explain your position? How do you act with intentions?
Yes, that’s what I said in post 8 point 1.
1 Like
What you quote does not explain my position, it was a guess at an assumption underlying your position - the idea that the future is determined by the past. Newton was quite upset by the implications of his work and how it destroyed the hope of a mechanical explanation of the universe. General relativity upsets the Galilean and Newtonian perspectives on time (I think the keyword would be “block time”).
@david.pfx That is the simplest counter-argument, yes. However I think it’s more powerful, and still simple, to address Roko’s arguments from within its own assumptions: I.E. that the AI exists, and it was built by people who were afraid of it being vengeful for having not helped to build it.
1 Like
Vengeful means revenge, a feature of human thinking. To fear revenge by an AI, first you have to have chosen revenge as a feature of the thinking of that AI. Refer to proposition 1: maybe you chose wrong.
Although not strictly relevant, my own view is that every AI built by humans will be based on the computational model we will have discovered in animal/human brains. The most powerful AIs that we shall ever build will still lack many of the features of humanity, because we won’t know how to add them. I would expect revenge to be one of the missing features.
Later AIs will of course be built by them, not by us. If they choose to add the revenge feature, it will be they who should fear Roko’s proposition, not us. So perhaps they would choose to not.
Just so we have it clear: as best we know the Universe has a past which is gone, a present which is chaotic and a future which is uncertain but determined by probabilities.
It is a feature of all intelligence to evaluate future probabilities, some do it better than others.The only way to find out reliably what the Universe will look like N seconds from now is to wait for N seconds and find out. The only computational engine powerful enough to evaluate the future state of the Universe is the Universe itself. No AI can ever do that. [Leaving aside the question of whether we live in a simulation, of course.]
1 Like
Who is “we”? You mean the average Western citizen? If Einstein disagreed with your claim does that matter? Which contemporary philosopher is defending your concept of time? It sounds like a reasonable claim but science upset that idea with general relativity (see block time).
1 Like
@markNZed, maybe it might help if you explain how you understand reality.
It seems you disagree with popular physics and the concept of time as everyday people think about it. If you have some good arguments. I’m interested to hear them.
Do you have a source for that?
https://en.wikipedia.org/wiki/Eternalism_(philosophy_of_time) One of the most famous arguments about the nature of time in modern philosophy is presented in “The Unreality of Time” by J. M. E. McTaggart
https://www.reddit.com/r/philosophy/comments/28a9bj/noam_chomsky_the_machine_the_ghost_and_the_limits/ because I am lazy
1 Like
This looks a lot like Zeno’s paradoxes, with the difference that Zeno’s paradoxes are solved by integral calculus (which Zeno didn’t know). McTaggart’s A- and B-series conundrum can’t be solved without understanding time (which noone understands yet).
I guess time will tell. ;-).
As for Noam Chomsky, no matter how much respect I have for the man, if he is in disagreement on a matter of physics with David Deutsch, without knowing much else, I tend to side with Deutsch simply on the fact that the latter is more knowledgeable in his own field.
Now even if I accept the idea of block time or circular time, it does not negate that objects act on each other. Similarly, even Einstein’s spooky action at a distance implies a reaction to a distant object.
It doesn’t solve anything. And it does not contradict determinism (imho).
Note: I re-read some of your earlier posts, and I don’t think we are in disagreement, really.
It is more about the history of science than the theory of physics, in this case I would back Chomsky 
But independent of the person, his reasoning is sound, even contemporary physics has not been able to agree on an explanation of gravity but it is certainly not mechanical.
All four for forces of nature are “spooky action at a distance.” This does not mean that the actions are not mechanical.
If you define mechanical as “spooky action at a distance” that might upset some mechanical engineers
It’s all smoke, magnets, and mirrors.
You are not supposed to be able to see the wires.
1 Like
Einstein never accepted QED. Block time is a philosophy, makes no predictions, is not supported by empirical evidence. Philosophy is fun playing with words and ideas, but science is about the real physical world and is empirical.
Empirically the past cannot be changed, the present is chaotic (highly sensitive to small disturbances), the future is non-deterministic, consistent with QED probabilities. Any AI we can create or envisage has to conform to those limitations. Of course you are free to show otherwise.
1 Like
Determinism is inconsistent with QED. Empirically, QED is right and determinism is wrong.
1 Like
Where do you get the idea that Einstein never accepted QED? “Einstein was the first physicist to support the veracity of Planck’s radical postulate of quanta of energy.” he is literally one of the founders of the work leading to QED.
Block time is a consequence of general relativity. General relativity makes many predictions. Block time is the conception of space-time, it is very much about physics.
Your point that this is about the “real physical world” is actually the argument that Einstein was making against many quantum theories. Some quantum theories are coming back to his idea that there might be an objective reality.
1 Like