A response to "Building machines that think and learn like humans"

help-wanted
papers
#1

I have been watching the videos coming out of the MIT AGI course. I was commenting on Josh Tenenbaum’s video on twitter when Jack Clark (journalist at OpenAI) wondered why we were not more a part of the conversation. Another comment suggested we write a response to the 2017 paper Building machines that learn and think like humans:

I have now read Building machines that learn and think like humans and the responses from other parties included in the paper. I’m going to take the suggestion and write a response from Numenta to this article.

Help wanted

I want our response to be well-informed, but our focus at Numenta is not to read all the latest AI research papers. We focus on specific neuroscience papers. I know many of you have ties to other areas in machine learning, and if anyone knows any major progress being made in the areas below, please post a link to a paper or article below so I can add it.

There are several areas highlighted in the article where AI needs to focus. I’ve listed them below along with any new papers or articles I’ve found published after the original paper above. Please help me add more to this list.

3 Likes

#2

In a separate post on language vs conscious, I related some cases where a human is raised past the language plastic period without learning a human language; there is a deficit in symbolic learning and reasoning. This strongly suggests that symbolic reasoning is a byproduct of learning to name things in language.
It would also point to language as an elaboration of somatosensory learning.
I don’t see this in any of the linked papers, in fact, the symbolic reasoning seems to be an expected inherent behavior.

The plastic period for language:
https://www.ncbi.nlm.nih.gov/books/NBK11007/

2 Likes

#3

I have thought a lot about this, and I’ve decided not to write this response.

I work in this field because I want to understand what makes us intelligent. It’s one of those frontiers of science where real progress is being made in our lifetimes. And when I write “this field” I mean brain science, not AI. (When people ask me what I do, I usually tell them “I study brains and make YouTube videos”.)

I’ve been studying non-biological (Bayesian) techniques and how they relate to HTM, and finally realized that this journey has already been taken by many people before me, and they either come out philosophically in one camp or the other:

  • biology
  • maths

The biology camp says:

Mathematical theories might tell us what a process is doing, but it cannot tell us how it does it. We must understand the low-level neuronal communication techniques and circuits before we will understand.

The maths camp says:

Without proofs, all biology explorations are shots in the dark. There’s no need to understand the cellular level details if we can approximate processes and functions in a generic, provable way.

If you’re a member of this forum, you’re probably in the biology camp. But there are many more people in the maths camp.

Because “AGI” is an unsolved problem and there is no proof that either approach will be fruitful, we must establish a belief that one way is better or more likely to produce the desired goal than the other. And we all know how hard beliefs are to change in people.

Anyway, that’s the long-winded reason I decided not to write this thing. I don’t think it is going to change anyone’s mind about their belief in what approach will result in AGI.

purity

1 Like

“Prediction” from the first principles
HTM vs. bayesian inference (network), predictive coding
#4
          they either come out

philosophically in one camp or the other:

  • biology
  • maths

It’s a false dichotomy. Kepler says “T2
~ R3
” <- astronomy or math?

0 Likes

#5

I’m not sure that’s the best analogy. A better one IMO (biased obviously) might be how someone teaches himself to become a mechanic by tinkering with cars. He could meticulously describe every working part of the car in complex mathematical formulas. Or he could study the car from a systems perspective and observe how each part interacts with the system to make it work. Sure, he may need to pull out his calculator from time to time, but it isn’t the first tool he reaches for.

0 Likes

#6

We’re scientists here not tinkerers.

1 Like

#7

I suppose that explains why I spend most of my time in the HTM Hackers category :grin:

0 Likes

#8

Scientists or tinkerers?

Theorist!

I’m a paper junkie: I collect them like a crazy cat lady and stray kitties. I read them like trashy serial novels, one after another; I skim the abstracts and group like ones so I can work a topic at a time.

I have been doing this for decades.
Today I’m on neural bundle connectograms - tomorrow it may be the meta-analysis of learning systems.
Sometimes I take a break and read a sci-fi or history book but my true love is all things thinking.

The engine of research keeps spewing out this river of papers and i’m loving it.

Sometimes I sling a bit of code to test drive what I am reading but the computer hardware we have available is a little toy compared to what the brain is doing. I have faith that Moore’s curve is relentless and will grow these machines to fit the task before I die. The trend line is good - my first machine in the late 1970’s was a DIY wire-wrapped SC/MP with 256 bytes of memory. My current daily driver is an off-lease quad-core workstation with 12 Gb of memory and a powerful GPU and terabytes of rotating memory. (under $400 on Amazon!) I can simulate a simplified model of perhaps a square mm of cortex with it.

My hope is that the trend lines (paper consumption and CPU power) will converge at some point - I can see all the details of how to “do it” and have enough computing horsepower pull it off.

1 Like

#9

I would point out that I recently implemented the grid cell module without cracking a calculator. The concept is simple and painfully self-evident – I don’t require a mathematical proof to model it in software. Granted, making that claim is probably what lands me squarely in the category of tinkerer and not scientist. :stuck_out_tongue:

2 Likes

#10

Besides the fact that Numenta will get better value for your time and I will get lesson fifteen that much sooner you deserve props for the having the courage to change your mind in public.

0 Likes

#11

it’s gonna be awhile no matter what

1 Like