Hoping some people in the community can help me out. I’m finding HTM theory is a very strong and persistent interest of mine, but I’m struggling with my complete lack of biology knowledge. E.g. someone will casually mention some part of the brain and I end up down the wikipedia rabbit hole for hours just so that I can understand the next paragraph.
I know there are a lot of people here who seem to successfully bridge the Computer Science<->Neuroscience camps, but how did you do it?
I’m more than comfortable with the former, with a degree plus 10 years in and around software dev. But for the latter, is it worth the time to go back to university? (I’d keep my day job and go part time)
My local one has:
a 3 year Biomedical Science degree, seems very broad so you’d have to study the entire body.
a 3 year Psychological Science degree, which focuses on the brain but could have less of a mechanical focus and more about behaviours.
a 4 year Medical Engineering degree, which is a odd mix of electronic/mechanical engineering and physiology. Again it’s whole body stuff, I assume they build cool stuff to fix people.
Alternatively, can you gradually accumulate enough knowledge about the brain’s biology that you don’t need all the general chemistry/biology background? If so, how long does the interim feeling of scientific inadequacy last?
In terms of my specific goals, first and foremost I’d like to fully comprehend the biologically inspired models of intelligence as they are developed. Possibly even contribute something original!
As the theories mature, I’m sure there will be a big push to start building intelligent systems that use these principles, I’d like to be prepared for the emergence of that market.
I also think that having a unifying theory for how the brain works will change the way we approach things like mental health treatment, addictions, even just optimising how a regular brain learns concepts - particularly once we have better brain instrumentation and can build systems that respond to a particular individual’s biology. This is another area I’d love to work in someday.
In short, I don’t need the formal qualifications for the purpose of landing a fancy research job, but if formal study is the best way to round out my knowledge then I’d happily pursue it.
To me it depends on the applications you envision for the systems you build & use, like personalized treatment as you mentioned. I have no neuroscience expertise myself besides HTM theory, but it seems that if you want to build more brain-like computing systems with greater AGI potential then you’re already in the right place,
The ‘sensory-motor model of the world’ that the cortex builds allows us new and strange things from 0. With tricky tasks we struggle at first, but then get the hang of it and eventually reach mastery if desired. If the machine that does this uses common mechanisms across all people and most tasks, and these mechanisms are carried out by a population of ‘cortical column’ modules, why doesn’t all of Neuroscience and AI pursue this with the focus that Numenta is? It seems partly because the two fields haven’t been truly synthesized. Efforts from the AI side like ANN’s and now DNN’s for instance are clearly interested in Neurosciece, but apparently not enough to build their models like actual brains. Efforts from the Neuroscience side (at least according to Jeff in On Intelligence) produced a giant heap of data without clear structures that would enable much real engineering action. As a result its 2018 and there’s still no common robot that I can robustly train to clean up my room or do my laundry. Anyone else disappointed? The sea of ‘deep’ connectionist-style models that pervade AI now seem based on notions of neural structures rather than the structures themselves, using math to fill the gaps. Does anyone really think the cortex is doing gradient descent? This is where Numenta seems quite different.
All that said I’m not trying to dissuade you from getting another degree, it just seems that depending what you want to build there may be much more direct ways to some very powerful applications.
I have been studying both computers/electronics and neuroscience since the 70’s. In my day job, I design and program embedded systems and feel that I have achieved mastery in this area.
Like you - I still end up running back to reference material every time I get deep into the neurological end of things.
If I had any advise at all it is to pay more attention to the older brain structures (the lizard brain) as these are still running things in the human brain. The relative simplicity and regularity of the cortex is a seductive siren call that has drawn most researchers attention but without understanding the interplay with the older brain structures none of this will ever get us to a complete understanding of how the whole thing works.
I am currently working through a course on Coursera through the University of Washington that is entitled “Computational Neuroscience” and it has been hugely enlightening. I share much of the same sentiment as you. I’ve spent the past 5 years in college digging deep into mathematics and computer science in a BS/MS program but I’m ready to become a better biologist. I’d be willing to bet a vast majority of people working in AI share this imbalance of expertise just as a result of how and where AI research has been conducted for decades.
I’m going to be spending some time in industry for awhile paying off my debt but I’m looking into computational neuroscience PhD labs and programs. I think that is the way to go to be able to confidently build and augment computational models in theoretical neuroscience such as HTM.
By the way, @rhyolight, that coloring book, what an awesome idea! I’m going to be ordering a copy for myself and sharing it with my colleagues.
I learned a lot of my basic neural anatomy doing cat dissection demo setups for the biology class in high school under a stereo microscope. Best study hall ever.
I don’t know that I could do much with coloring as in my mind it’s almost all pinkish white and gray with the occasional splash of yellow.
I would focus on something very specific, like a particular layer/sublayer and just one class of pyramidal cell. You can narrow it down further by brain region, species, and things like plasticity, dendritic compartment, connectivity, and so on. This might not be good for thinking about AI, but it’ll get you out of the initial confusion more quickly since you’ll know the terminology for the topic soon. It’s hard to connect neuroscience with AI if you’re bogged down with what neuroscience writers assume the reader know about the exact topic.
I have only taken intro level chemistry and a couple biology courses. Since I don’t know exactly what is going on chemically, it takes a while for me to understand what a particular ion channel does and similar things, and even then I can’t make many connections between those facts. I also can’t evaluate technical methods very well.
Besides those problems, I usually feel like I know what an article is saying and can come to some conclusions. At least right now, I think my time is better spent ready articles and writing summaries instead of spending that time in a lab and mostly at the single-cell scale or anatomy scale. Neuroscience is usually ambiguous, so you can’t build AI by understanding everything. You can come at it from whatever angles seem best and enjoyable. You should probably enjoy whatever learning process you choose as much as possible, since lateral thinking and absorbing facts with ambiguous meaning is hard otherwise. It might take a decade before you start creating solid ideas. Object recognition by narrowing down objects based on features at locations is a one sentence thought, and SDRs and sequence memory could be thought of with a single sentence before expanding on that initial idea. But those thoughts take a lot of time to arrive at. So I would expect to have interesting thoughts which get you excited but end up going nowhere 99% of the time, so enjoy the process. If that means getting a degree, go for it, otherwise don’t rush it.
When I started reading wikipedia articles, it was definitely a rabbit hole like you mentioned. If someone mentions part of the brain, I wouldn’t focus on the exact brain region. There are a lot of details about each brain region, but really you’re just dealing with cortex (with small differences between things like prefrontal cortex and what/where pathways), thalamus of two projection types, and dozens of subcortical structures which aren’t understood in many generalizable ways and so can be put into some general buckets until you decide to dive in. I don’t mean to say that subcortical structures don’t matter, but don’t waste time learning the abbreviations and such for each structure you encounter. Unless you plan on learning about an area in detail, don’t worry about the details of a specific part of the brain.
That’s a good point, plus all scientists rely on abstractions in order to be productive (see xkcd: Purity).
I think I’ll spend at least the next few months or so studying macro brain structure and high level roles (especially as it involves colouring-in books). After that point I’ll evaluate whether or not a lower level chemistry and biology foundation would help.
For anyone on the lookout for good resources in neuroscience: this series of videos called “Brains Explained” on Youtube includes several easy-to-digest lecture style videos on various brain topics. I find they go into satisfying detail, though. The video on the neocortex is particularly useful and includes details beyond what is available in Jeff’s book.
