# \#deep-learning

**URL:** https://discourse.numenta.org/tag/deep-learning/189.md

[Latest](https://discourse.numenta.org/latest.md) · [Categories](https://discourse.numenta.org/categories.md) · [Tags](https://discourse.numenta.org/tags.md)

---

## [Yan LeCun on GI vs. current DL](https://discourse.numenta.org/t/yan-lecun-on-gi-vs-current-dl/10636)

<div class="topic-metadata">

**Author:** [@bkaz](https://discourse.numenta.org/u/bkaz)\
**Replies:** 32\
**Last updated:** [June 16, 2023, 3:03pm UTC](https://discourse.numenta.org/t/yan-lecun-on-gi-vs-current-dl/10636 "2023-06-16T15:03:28Z")

</div>

His recent lecture: https://drive.google.com/file/d/1BU5bV3X5w65DwSMapKcsr0ZvrMRU\_Nbi/view Recommendations: Abandon generative models in favor joint-embedding architectures Abandon Auto-Regressive generation Abandon p…

---

## [Replicated some of "Going beyond the point neuron"](https://discourse.numenta.org/t/replicated-some-of-going-beyond-the-point-neuron/9759)

<div class="topic-metadata">

**Author:** [@bainro](https://discourse.numenta.org/u/bainro)\
**Replies:** 5\
**Last updated:** [May 28, 2022, 9:11pm UTC](https://discourse.numenta.org/t/replicated-some-of-going-beyond-the-point-neuron/9759 "2022-05-28T21:11:47Z")

</div>

I replicated some of the results from “Going beyond the point neuron model” paper. I also tried extending ADNs to CNNs, and tested them on SplitCIFAR100 (similar to the continual learning dataset from the SI paper). You …

---

## [Numenta Research on 3 Visual Stream & Deep Predictive Learning](https://discourse.numenta.org/t/numenta-research-on-3-visual-stream-deep-predictive-learning/7057)

<div class="topic-metadata">

**Author:** [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Replies:** 3\
**Last updated:** [January 27, 2020, 3:13pm UTC](https://discourse.numenta.org/t/numenta-research-on-3-visual-stream-deep-predictive-learning/7057 "2020-01-27T15:13:14Z")

</div>

Where we try to make sense of https://discourse.numenta.org/t/deep-predictive-learning-a-comprehensive-model-of-three-visual-streams/3076. Wed, Jan 22 at 10:15AM PST.

---

## [Variational Inference provides a taxonomy for training deep networks](https://discourse.numenta.org/t/variational-inference-provides-a-taxonomy-for-training-deep-networks/6718)

<div class="topic-metadata">

**Author:** [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Replies:** 2\
**Last updated:** [October 23, 2019, 4:50pm UTC](https://discourse.numenta.org/t/variational-inference-provides-a-taxonomy-for-training-deep-networks/6718 "2019-10-23T16:50:55Z")

</div>

Tomorrow, Friday Oct 18, 10:15AM Pacific. Hosted by @mrcslws. The referenced paper.

---

## [Second order optimization methods for SGD convergence?](https://discourse.numenta.org/t/second-order-optimization-methods-for-sgd-convergence/5488)

<div class="topic-metadata">

**Author:** [@Jarek\_Duda](https://discourse.numenta.org/u/Jarek_Duda)\
**Replies:** 0\
**Last updated:** [February 21, 2019, 8:19am UTC](https://discourse.numenta.org/t/second-order-optimization-methods-for-sgd-convergence/5488 "2019-02-21T08:19:30Z")

</div>

It seems that, especially for deep learning, there are dominating very simple methods for optimizing SGD convergence like ADAM - nice overview: http://ruder.io/optimizing-gradient-descent/ They trace only single directi…

---

## [Deep learning meets sparse distributed representations: The Kanerva Machine](https://discourse.numenta.org/t/deep-learning-meets-sparse-distributed-representations-the-kanerva-machine/3743)

<div class="topic-metadata">

**Author:** [@jakebruce](https://discourse.numenta.org/u/jakebruce)\
**Replies:** 7\
**Last updated:** [March 23, 2019, 1:22am UTC](https://discourse.numenta.org/t/deep-learning-meets-sparse-distributed-representations-the-kanerva-machine/3743 "2019-03-23T01:22:01Z")

</div>

“We present an end-to-end trained memory system that quickly adapts to new data and generates samples like them. Inspired by Kanerva’s sparse distributed memory, it has a robust distributed reading and writing mechanism.…

---

## [OpenAI’s massive new NLP model](https://discourse.numenta.org/t/openai-s-massive-new-nlp-model/5449)

<div class="topic-metadata">

**Author:** [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Replies:** 13\
**Last updated:** [February 17, 2019, 12:04pm UTC](https://discourse.numenta.org/t/openai-s-massive-new-nlp-model/5449 "2019-02-17T12:04:11Z")

</div>

This came out yesterday. I’m surprised no one is talking about it yet. It is really impressive. Basically you create an NLP DL network that is really really big, then you show it the entire internet. Now by handing it…