# Geoffrey Hinton explains sparse coding and drop-out

**URL:** <https://discourse.numenta.org/t/geoffrey-hinton-explains-sparse-coding-and-drop-out/2336>\
**Category:** Lounge\
**Created:** [May 20, 2017, 2:17am UTC](https://discourse.numenta.org/t/geoffrey-hinton-explains-sparse-coding-and-drop-out/2336 "2017-05-20T02:17:35Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sean\_O\_Connor](https://avatars.discourse-cdn.com/v4/letter/s/34f0e0/32.png) [@Sean\_O\_Connor](https://discourse.numenta.org/u/Sean_O_Connor)\
**Post date:** [May 20, 2017, 2:17am UTC](https://discourse.numenta.org/t/geoffrey-hinton-explains-sparse-coding-and-drop-out/2336/1 "2017-05-20T02:17:35Z")

</div>

I though the comparison between the effects of drop-out and sparse coding given in the video was very interesting. Namely sparse coding is not really being used for feature discrimination, rather it is used to force ensemble learning and generalization by breaking up unhelpful dependencies:  
[https://youtu.be/05MpycrocRI](https://youtu.be/05MpycrocRI)

---

<div class="post-metadata">

**Author:** ![Sean\_O\_Connor](https://avatars.discourse-cdn.com/v4/letter/s/34f0e0/32.png) [@Sean\_O\_Connor](https://discourse.numenta.org/u/Sean_O_Connor)\
**Post date:** [May 20, 2017, 2:53am UTC](https://discourse.numenta.org/t/geoffrey-hinton-explains-sparse-coding-and-drop-out/2336/2 "2017-05-20T02:53:26Z")

</div>

With sparse coding you can never be sure a particular feature dimension (neuron) will be active when classifying an input. Hence the system cannot over fit by building up a dependency on that one dimension. I’m sure there are more subtle aspects as well.