Google DeepMind claims they're close to achieving human-level AI

Science
142

I think it’s conscious if it can experience time skips. When I went under general anesthesia, it was like a time skip. My memory of the moments up to “I wonder how long the anesthetic will take to hit my brain” was strong when I woke up, and it felt like little time passed.

You can put a mouse under general anesthesia. Maybe also a lizard, I don’t know. But a nematode, I really doubt. And a rock? Nope. Yet rocks still have “memories”. A rock “remembers” being scraped by another rock a million years ago. It leaves an imprint. But the rock doesn’t know what happened.

Even under general anesthesia, you still have neural responses to stimuli. Anesthesia tends to disrupt the connections between the brain. So maybe like the neural response doesn’t get past primary sensory cortex, all the way to the parts for abstract thought.

Here’s basically what they say:

It’s possible to not be aware of stimuli, and still have a neural response. The difference seems to be bursting in L5, although that’s not totally proven and it’s probably more complicated.

The bursting triggers detection via a few subcortical structures. Those synapses are required for detection. (This is pretty cool, they silenced specific synapses.) One possibility is that the burst signal is required for thalamocortical cells in higher order nuclei to fire, propagating the sensory response to parts of the brain for more abstract perception etc.

The studies mainly use this kind of experiment: There’s a very weak stimulus, does the animal notice it? Imagine trying to notice a very weak stimulus. Even if you fail, your brain is still having a neural response.

2 Likes
143

I meant sensory modalities: vision, sound, etc. Qualia people love to talk about “the meaning of red”, to a scientist colors are sub-modalities of vision. Subjective? Anything in the brain is “subjective” to a clueless user, objective to a scientist.

“Subjective” explanation is an oxymoron, to explain anything you have to treat it as an object.

1 Like
144

You’re probably right. We have skin and eyes and nerves and stuff and then ‘magic happens’. :roll_eyes:

1 Like
145

Brainchip Inc… any thoughts on their position ?

146

You’ve completely misunderstood my point.

If there is a model of a real person, then we don’t even have to discuss if that model is “human-level” because we can just ask him general questions we can ask from real human-being, then we get appropriate answers.

I can agree though with “human-like” comment above.

147

Sorry, too long ago. But to try to explain: a ‘model’ is not a fully detailed simulation (which is actually impossible). A ‘model’ is a construct setting out the workings of some small part of the real world. A ‘model’ of a ball you want to catch has an observed position, velocity, maybe colour and spin, but no internal detail, no history, just enough to catch it.

We have a thing called ‘theory of mind’ whereby we construct mental models of how other people and animals behave in the past, present and future. We use them to make predictions and choose our actions, but we can’t ask them questions. They’re not real.

148

Whoever created the human mind and its capabilities to cause behavior created a model of the human mind based on something or carved by evolution. It’s real, obviously, we can ask this human mind questions, perhaps millions of joint and disjoint questions like we normally ask a person in his/her entire life. Now if we ask these same questions to an AI model presented in the original topic, its answers would not satisfy us, perhaps 50 questions would suffice to show it’s not human-level. We would also feel something odd or wrong.

AGI is hard to manufacture, therefore the function to prove AGI is likely hard to manufacture as well. This function is I believe the natural human mind, obviously and luckily we have this mind.

149

Nobody has or will create a ‘model of the human mind’, ever. The strongest argument is that it is chaotic, like the weather. You have to run the real thing to find out what it’s going to do. The only model of the human mind is a human mind.

AGI is quite different. It is a piece of software/hardware that demonstrates ‘intelligence’, in the same way that an animal or a bird is intelligent. It can learn to drive a car or fly a plane or read/write a document or solve problems/puzzles better than any of us, but it is not a human mind, not even close.

150

The difference is, we have a mathematical fit of what improvements (i.e scale and architectural) is required to convert those 50 questions it can answer well, to a hundred, then a thousand, then eventually to the same limits as that of a human.

I agree - but a better question is, is the model of a human mind the ultimate model? couldn’t it be simply littered with unused vestigial parts and processes? in that sense, what stops us from using mathematical objects to approximate the same models to any arbitrary degree?

1 Like
151

Sounds contradicting to me, the last statement is my point here. We are still not talking sensibly to the “human-level” AI.

We are limited by what we know only about AGI. I can only temporarily agree to this. When true AGI is reached, I’m pretty sure this assumption will change, and even the meaning of AGI will change. In the best case, we will face the problem of identifying which one is the human mind and the artificial mind. Again we are limited by what we only know about AGI for now.

1 Like
152

Are you saying that we have a known scalability function that tells us how much a model needs to improve to reach some level?

I disagree. There’s no such thing as AGI, it doesn’t exist (yet). So how can we say that it’s different from a human mind if no one has achieved it (yet)?

It’s like stating the following;

My aim is to design f(x,y,z) so that its result is very close to g(x,y,z)'s, at the same time I believe that nobody will model g(x,y,z) because it’s chaotic. I also think that by training, I can shape f(x,y,z) to return almost the same result as g(x,y,z) without necessarily modeling g(x,y,z) and by testing it with my test data set.

Going back to the “human-level” AI topic, obviously we can use our minds to simply test the AI if it really has “human-level” intelligence. Obviously we can’t even converse with it casually.

1 Like
153

I make no claim of ‘ultimate’, nor would I. For AGI we have the model of animal intelligence, with the prospect of doing the same stuff but faster and better. That means an AGI that constructs ‘mental models’ of aspects of reality, predicts past and future, learns from experience, understands time and place.

Simple example: recently Tesla cars killed 2 motorcyclists because a motorbike up close on a dark road looked too much like a car in the distance. An animal/human/AGI would understand the problem and learn a way to distinguish but current AI cannot. Tesla has to obtain a bunch of new photos and use them to update the ANN, and maybe it will still run over the odd motorcycle.

AGI is nothing to do with the human mind, it’s about modelling reality, choosing strategies and learning from experience.

1 Like
154

Yep, we do. Check out the Chinchilla paper - the current bottleneck is data, though technically those scaling laws are incomplete and quite narrow. So Chinchilla laws are usually treated on the pessimistic, “worst-case” side. There are even some more famous scaling papers (Kaplan et al.) which you can check out with a background.

Indeed - since we are approximating the brain, you’re right that it won’t be like the brain - only behave similarly to the brain (the beaten-to-death paper on how brain and NN activations are linearly correlatable).

And yes, talking to AI is definitely a nice benchmark - but the DL field is more rigorous, and thus has a more extensive suite of benchmarks to test capabilities. If you think it can do x task, then you can provide it actually more complex examples it has never seen and check if it can actually do x task. Look at the Myriad of benchmarks which total in thousands of tasks: A famous one by Google, BIG-Bench

So this is quite annoying that its literally brought up the 4th time again.

  • Tesla is NOT some leader of AI - their research is little to negligible. The current giants are Google and Meta which hold large percentages of DL research.
  • Tesla’s models are NOT at scale. They are tiny, few hundred million parameter networks. AlexNet, the first “Deep” Convolutional Neural Network had 22M parameters in 2012.

Models which are not at scale enjoy none of the emergent scaling heavier (the hypothesis that certain abilities emerge at certain scales - we thought it was a sudden spike in some capability, turns out if you measure stuff properly, you can still obtain a smooth curve of task performance vs. scale and predict what models will unlock what abilities at what scale).

A famous example I’ve quoted multiple times is PaLM where emergent abilities at scale allowed explaining jokes (a simple task, yet no system is able to do that) complex deductive and inductive reasoning (like the example of inferring a katana through complex multi-step reasoning steps) and in general being more efficient in learning new tasks - it was able to outperform Codex, a specialized model on ALL programming tasks with 50x less data. Scale provides sample efficiency, which is very common in natural organisms.

Tesla models enjoy NONE of these advantages becauase they’re simply too small. So comparing Tesla is redundant.

Tesla went about it the wrong way - they were using off-the-shelf Ryzen CPUs which are terrible at DL inference. But they actually realized that wayy back, and thus why they’re now focusing on DOJO. They aren’t fools, but it did take them some time to catch up on what they needed to do - which is better, because other companies haven’t even touched expanding hardware costs.

Inferencing in a safe manner, having backup chips, not drawing too much power to drain Tesla batteries, resistance to inteference and adversarial attacks, compliance with outdated ADAS laws slows you all down.

So really, Tesla is a really bad example to cite why we won’t achieve AGI. They are a self-driving company, not a research institution.

3 Likes
155

This reminds me of McCoy “Dixie Flatline” Pauly in William Gibson’s Neuromancer. (In the story McCoy is dead, but his memories have been stored in a ROM so that he can be consulted). I never quite understood the difference between Dixie and Wintermute, other than that ROM constructs were not allowed to store what they learn.

1 Like
156

I partially disagree. We humans are the best/ultimate testers of AGI. Saying that it doesn’t have to do with the human mind is a cogbitive dissonance.

The mind/brain is probably one of the most complicated/complex function in this known universe. Mimicking it’s “intelligent” abilities requires a similar function, a function that might even be the brain/mind. What are the odds of a disjoint function from the human mind of all functions in these universe that can mimick the same human mind? Almost 0? But maybe more if AGI is met.

1 Like
157

I see what you’re saying. But Chinchilla is very limited, in the sense that it can get obsolete with new architectures or models.

1 Like
158

Sorry if I’m repeating old info, but the Tesla example stands. With current AI there is no direct feedback from ‘experience’ to ‘learning’ other than through high cost external ‘training’. With AGI a car that ran over one motorcyclist would tell all the other cars about the problem, and they would all learn from the experience, and watch out for motorcyclists masquerading as distant cars. AFAICT we have nothing like that, not even close.

1 Like
159

Actually, we do, but it is at a very simplistic stage. A component of the ‘Smart Highway’ is inter-vehicle communication and some of it has been implemented and at an extremely simple level there is the traffic data (check out Waze). So the mechanism for what you describe is there, but the level of detail and how it is used; i.e., learning for behavior modification, is certainly not there yet.

Then there’s the problem of the ‘bad example’, like when mom told you not to hang out with a certain group that had questionable moral principles. So your Kia starts thinking it can hang out with the Porsches and get away with their behavior.

160

We have plenty of sensors, low level tech, to match or beat most organics. That is not the issue. And bad example? Tell me a good one!

The point I make here is that AGI has nothing to do with the human mind, inner voice, consciousness, moral code. I make very specific claims about the kind of things AGI could do, which nobody seems willing to either accept, or challenge. What do you think it is?

An AGI car would be like an Uber without the driver. It would take jobs, collect passengers or goods, deliver them to an address, refuel itself, answer questions like weather and sports scores, etc. It learns the territory, short cuts, bottle necks. It learns its regular customers. It is the ultimate public transport!

1 Like
161

What makes you say that updating networks is always costly? this and this, which are some among the hundreds of such techniques (with very strong results, might I add) which allow low-cost updates.

The fundamental idea behind that is transfer learning - so any updates you make to the network will require less and less data and compute. If that doesn’t make sense, look at the brain (which is definitely a field you guys know more about than me). Young kids have high neuroplasticity to adapt to the environment they’re thrown in. When you get older (say, around ~25) you aren’t as plastic as your 10 year old self - but you can still learn to build upon the fundamentals of knowledge and explore more complicated problems than you’re 10 year old self every could.

That’s a crude analogy, but in essence, if you already have a pre-trained model, then applying updates to it takes less and less data and compute because the model is already at a stage where it knows a lot of stuff an how it meshes together.

The drawback is that at some point, you will exhaust the model where no more updates are allowed unless you overwrite some other information. For Large enough models, that’s shrinks less and less with scale because you simply have a lot of space to overwrite stuff or even benefit from it (positive transfer). But for tiny models, its better to insert those cases in the training dataset and re-train. You can still update it (via fine-tuning) but you would have to be quite careful and monitor accuracy to ensure no forgetting takes place.

1 Like