Indeed. There is no guarantee, however - it could still be just a failed investment. I fail to share any optimism about anything without strong and accurate results.
what.
@MaxLee A prime example of what I mentioned earlier.
The only model we have for something that acts human, well, are humans. And all our behaviors, unless you assume we’re some magical spirit antenna, come from our brains. So yeah, study neuroscience (or read and build up an awareness of the physical phenomena that happens within our biological neurons)… otherwise you’re building a system based on a whole host of potentially flawed assumptions.
I think that’s safe enough to say with a level of surety.
3 Likes
I think it’s safe to say that this thread was a waste of time for most involved - clearly, the cross-domain specialists I was looking forward to conversing with aren’t as open-minded as I thought.
I suppose everyone has a different mindset and opinions to what they think would be the optimal path to AGI; but I would caution against sticking to a single route and recognizing ideas in their true light - otherwise one may end up pouring their entire lives on something that turned out to incorrect.
As a reminder, GOFAI communities still exist with the strong enthusiasm they’re going to discover AGI tomorrow (I really wish I was joking) - clearly their optimism has weathered the decades and decline of such methods. Whether they have been wrong or right is never a simple question, but the majority hold out irrational ideas about how implementing this idea based on how they ‘feel’ would suddenly create consciousness.
Simply put, I really don’t hope that Numenta’s work ends in another hole like this. I would advise anyone who’s reading this to keep an open mind about all the roads and learn as much as they can from all of the paths.
This is a pretty accurate description of the DL community - precisely this approach of focusing towards strong results that have managed to achieve so much in these short decades.
One really wishes to jump in and trust the process, but if anything history of AI has taught us is that those methods usually end up too ambitious and near-sighted to actually deliver in the long term.
Thanks to everyone for participating and taking the time out to share their thoughts 
Hope everyone here has great success with their endeavours!
2 Likes
This is true of both deep learning as well as neuroscience, but the two fields of study measure their results in very different ways. Scientists measure their productivity by the number of true facts that they discover. Measured by this standard: Numenta and the the whole field of neuroscience have achieved a lot in the past 20 years.
I know and that’s just sad. I too spent years in college studying GOFAI and deep learning and thinking like this.
But regardless, good luck with your studies!
This thread is titled “Why do members here think DL-based methods can’t achieve AGI?”
Naturally, that will prompt responses and opinions as to why we think DL-based methods can’t achieve AGI. There have been a few folks that took the time to try to respond, quite generously in some cases. To that end, this thread accomplished its surface-labeled goal.
I’m trying to emphasize this to anyone who reads this later… quite a few of us here actively use different ML/DL approaches in our work/study. We’re not anti-DL or anti-ML, but the scope of this topic was “Can it achieve AGI?”
Now, if the actual guerrilla goal of this topic was to proselytize why DL-based method might achieve AI, it’s probably been a waste of time to that end. If you feel we’re missing something, you can always create a new topic with the label of “Here’s why I think DL will achieve AGI” then lay it out there.
Personally, at work I inherited a team and project from a couple folks who were DL charlatans that were promising the impossible in order to try to get rich and have spent the past year doing everything to get us back to reality with a real, practical, working system. I also work on algorithm development, systems level programming, architecture, platform, data science, and deployment problems, so that experience all weighs on and influences my opinion.
For a real business where the intent is to make working systems while keeping risks/costs low, there IS a drive towards the pragmatic requirements of a system when it gets to production. More often than not, DL is a risky proposal in production systems. It’s expensive to train (GPU instances are often more than 4x a CPU instance), they’re unstable at arriving to their intended learning destination, explainability is always an issue, as is bias (very important when working on financial decision applications), and the simple time required to train a DL-based system… more often than not, xgboost and its variations achieve just as good of results, require far less compute to train, have higher levels of interpretability, etc… They’re cheaper while delivering “good enough” results.
What I think you might get out of even just the spatial pooler of HTM is using it as a dropout selection mechanism in your DL networks. None of us have to be religiously inclined to any approach or another. You have full freedom to pick and choose, just as many of us here do. Have fun with it and take care
4 Likes
The thing I got from this thread was how little agreement/understanding there is of what constitutes AGI.
My take is that AGI has nothing to do with ‘being human’ and everything to do with science and engineering and performing intellectual tasks. Clearly there are those who don’t see things that way.
It might help if you set down your definition/concept of AGI. Or it might not.
1 Like
Hello MaxLee,
Thanks for your detailed response! I liked how you tried to give a wholesome view combining the problems in DL with the ongoing efforts in Numenta.
I have been following Numenta’s work for quite a while now (although not very keenly in the last 6-8 months), but I have never come across this idea of HTM using Attention mechanisms or anything approximating it in their work so far.
Perhaps, I may have missed it. 
Could you please elaborate on this point a little more?
Many thanks!
2 Likes
There’s a little bit of reading between the lines required, but for a moment, pretend you train a spatial pooler on your dataset. The resulting spatial pooler itself, with its sparse representation, already represents which points are important within your dataset.
The enforced sparsity in a spatial pooler IS attention, even if it isn’t advertised as that, by forcing minicolumns in a pool to learn efficient and sparse representations for a given input. How we choose to take advantage of that is up to us though. For example, take that SDR representation and map it to the neurons within your DL architecture so that it turns neurons on/off depending on which columns are active in your spatial pooler.
Trawl the blog a bit
4 Likes
I’m a bit late to the party, but here are my two cents…
There are many reasons (such as efficiency, resiliency, lack of continuous learning, flexibility, etc… ) that makes AGI unattainable for DL, but a nice one is in The Brain from Inside Out. The book, which is really fun to read, has an overwhelming number of examples of how AGI will require a complex and stable internal state that dominates in the outputs (i.e., there is an “inside” dynamical state is affected slightly by the inputs). (Assuming that our brain does some sort of AGI )
I think that this invalidates DL. My understanding is that DL are mostly a “state-less” out-to-inside system (i.e., output is mostly dependent on the inputs).
4 Likes
I think that reason also applies to general object recognition.
What I find most interesting about Buzsáki’s work is the ‘search for neural syntax’ theme. A hugely important question and not trivial to answer. In the end, it will come down to metaphor.
He has an impressive knowledge in neuroscience. Certainly with very nice ideas like that (although I am little fond of some, like his interpretation on oscillations).
Why are you ‘little fond’ of? Apparently Science had no issue and neither does Brockett.
This sort of smacks of belief. Like “I believe in science.” Not talking about the journal, but those signs and memes that speak to science as religion–it is not. There’s no faith and belief going on here, you state a hypothesis and then you either prove or disprove it. If you can’t prove or disprove, then at that point you can turn it over to religion.
1 Like
I agree. I’m just saying that looks not very elegant to me. It’s not a question of beliefs, just of aesthetics
1 Like
Not sure any gradient based methods would work for error assignment even if we had an explosion in chip technology to handle the trillions of calculations and at a local level. Whatever the brain is doing its not adjusting synapses like deep learning tunes weights. The gradient calculations are too expensive and slow.
4 Likes
Apart from it being the worlds largest consumer of industrial robotics.
Given the quality of “scientific” justification needed to have the compliance of most of the population in the last two years, it would seem the AGI will not need to be very smart to convince society of the why even if the why is very dubious.
Proving or disproving requires a formal representation, which is lacking from a lot of science. It is more about intuitions, statistics, and Occam’s razor than most would like to admit.
True, but it seems likely that if an AGI emerges it will not be a copy of the brain.
We learnt a great deal about flying by observing nature, early aircraft are inspired by nature but technically very different, today there are flying devices that are much closer to birds (e.g. flapping wings). I imagine human like AGI being a curiosity developed after the emergence of AGI.
Consier AlphaGo, bootstrapped on human played games, yet inferior to AlphaGo Zero (does not use human games). If we want to master Go we would be better to learn from AlphaGo Zero’s insights rather than worry about AlphaGo Zero playing Go in human style.
DL-based could be interpretted to mean many-layer ANN trained using gradient descent. Given the empiriclal results and investment they are likely to be a part of future improvements in AI and contribute to new learning algorithms that use many different optimization techniques.
Another general remark, which may not apply to some people here, when referring to DL, in the past,I’ve thought of this as systems like large CNN but deep reinforcement learning (DRL) is way more sophisticated. DRL can include recurrent neural networks and predictive models so the potential seems much greater. This is not to say that any current architecture is going to achieve AGI. If you look at the evolution in the sophistication of the DL architectures over the last 10 years, I don’t see this slowing down any time soon and the progress is impressive.
4 Likes
I think whatever pops out as a consequence of this experimentation will be an alien, frozen state of something exhibiting intelligence. Will it be general, will it be self-understanding with the ability to introspect, make decisions, and change itself over time?
I guess we’ll see, but I suspect our bottleneck on a DL-based approach will continue to be the offline nature of its training… and again, to emphasize, whatever we do produce will be completely alien from us and our way of “thinking”, a ephemeral, stateless entity, existing entirely within the box we place it in, trapped like a ghost in a machine.
Impressive it will be. AGI? Let’s wait another decade and see what happens.
1 Like
In DRL there is both an offline and online aspect. It is the online aspect (closer to search/optimization) that amplifies RL results and explains a great deal of their success. The “bleeding edge” of research in DRL is on adaptive/autonomous systems e.g. Yann Lecun’s recent talks. In these types of systems the objective function is no longer static e.g. based on self-supervised learning using predictions in space/time.
Alien: absolutely. Frozen? Probably not. Make decisions: of course! Stateless: no, we would build it to preserve state because it’s more useful that way.
Introspection: problematic. How would you go about proving scientifically that you, your friend and your dog do or do not have introspection?
The essence of intelligence to find solutions to problems in aid of survival faster than evolution or genetic adaptation and to pass those solutions on to progeny through learning instead of genetic inheritance.
2 Likes
Any state mechanism will be bolted onto the inference engine, rather than embedded as it is with ourselves. So its internal weights (all 100tn of them) will likely remain frozen, barring another leap up in technology to allow simultaneous inference, feedback, and weight update… the thought of that though is pretty terrifying to me, in that it would become unpredictable while still being alien (though still confined to its box). At least humans, with our physical state machines attached to muscle outputs providing some level of interpersonal queues and feedback allow us to relate to each other… this ‘thing’ wouldn’t be, not for a while anyway.
2 Likes