# Adversarial examples and deep neural networks

**URL:** <https://discourse.numenta.org/t/adversarial-examples-and-deep-neural-networks/3838>\
**Category:** Lounge\
**Created:** [April 27, 2018, 2:00pm UTC](https://discourse.numenta.org/t/adversarial-examples-and-deep-neural-networks/3838 "2018-04-27T14:00:45Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sean\_O\_Connor](https://avatars.discourse-cdn.com/v4/letter/s/34f0e0/32.png) [@Sean\_O\_Connor](https://discourse.numenta.org/u/Sean_O_Connor)\
**Post date:** [April 27, 2018, 2:00pm UTC](https://discourse.numenta.org/t/adversarial-examples-and-deep-neural-networks/3838/1 "2018-04-27T14:00:45Z")

</div>

I’ve been watching a very intelligent youtube video about adversarial examples and deep neural networks.  
[https://youtu.be/CIfsB\_EYsVI](https://youtu.be/CIfsB_EYsVI)  
Obviously the mammalian brain processes information in a rather different way, I guess by doing extensive unsupervised feature learning first before attempting to classify.  
Also I have been doing some html5 practice and have some WHT information that I may add to over the next few days. I can’t say it works in any browser other than pale moon 'cos I didn’t use jquery.  
[http://md2020.eu5.org/wht1.html](http://md2020.eu5.org/wht1.html)

---

<div class="post-metadata">

**Author:** ![Sean\_O\_Connor](https://avatars.discourse-cdn.com/v4/letter/s/34f0e0/32.png) [@Sean\_O\_Connor](https://discourse.numenta.org/u/Sean_O_Connor)\
**Post date:** [April 28, 2018, 1:59am UTC](https://discourse.numenta.org/t/adversarial-examples-and-deep-neural-networks/3838/2 "2018-04-28T01:59:19Z")

</div>

Also Uber have an interesting article on measuring the complexity of problems posed to deep neural networks:

[https://eng.uber.com/intrinsic-dimension/](https://eng.uber.com/intrinsic-dimension/)

Uber have brought on board some very creative and capable talent recently.

I think the “fastfood” algorithm is more complicated than necessary just to avoid having to cite a hobbyist. I would say google themselves use the faster version and leave the competition with a slow thing.

---

<div class="post-metadata">

**Author:** ![keghn\_feem](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/keghn_feem/32/3743_2.png) [@keghn\_feem](https://discourse.numenta.org/u/keghn_feem)\
**Post date:** [April 29, 2018, 1:30am UTC](https://discourse.numenta.org/t/adversarial-examples-and-deep-neural-networks/3838/3 "2018-04-29T01:30:45Z")

</div>

Interesting. I missed that one.

Understanding Generative Adversarial Networks:

> **[Understanding Generative Adversarial Networks – Towards Data Science](https://towardsdatascience.com/understanding-generative-adversarial-networks-4dafc963f2ef?gi=66579619e05a)**
>
> If you see the above image and it does not make much sense, this article is written for you.

---

<div class="post-metadata">

**Author:** ![Sean\_O\_Connor](https://avatars.discourse-cdn.com/v4/letter/s/34f0e0/32.png) [@Sean\_O\_Connor](https://discourse.numenta.org/u/Sean_O_Connor)\
**Post date:** [April 30, 2018, 6:08am UTC](https://discourse.numenta.org/t/adversarial-examples-and-deep-neural-networks/3838/4 "2018-04-30T06:08:57Z")

</div>

In the Uber paper they use dimension increasing random projections:  
[https://arxiv.org/abs/1804.08838](https://arxiv.org/abs/1804.08838)

If you are willing to use evolution you could try Fourier synthesis for the dimension increase. That would give locally repeating structure that might work well with natural images that also have local structure.  
You could use the FFT. That would force you to use a lot of parameters though , rather than compactly representing the pattern.

You could use sums of a.sin(x.t+b) but then you are faced with tons of calls to the time consuming sin() function.

Instead you might try using this quadrature oscillator algorithm:

[https://groups.google.com/forum/#!topic/comp.dsp/GFAzbora6bE](https://groups.google.com/forum/#!topic/comp.dsp/GFAzbora6bE)

You get more than you need because you get a sin and a cos output at the same time, I’m sure you can set the phase initially so you only need one of sin() or cos(), unless you want to draw circles in 2d or something like that.

Anyway the Uber paper shows you can achieve very high levels of compression for deep neural networks.

---

<div class="post-metadata">

**Author:** ![Sean\_O\_Connor](https://avatars.discourse-cdn.com/v4/letter/s/34f0e0/32.png) [@Sean\_O\_Connor](https://discourse.numenta.org/u/Sean_O_Connor)\
**Post date:** [April 30, 2018, 10:37am UTC](https://discourse.numenta.org/t/adversarial-examples-and-deep-neural-networks/3838/5 "2018-04-30T10:37:16Z")

</div>

I think with a dimension increasing random random projections you can achieve relatively more precision is some of the dimensions at the expensive of others.  
The “at the expensive of others” would explain why they are not able to completely equal training on the full set of weights. I’m sure there are a few ways around that problem, like doing a second correction round.  
I see I mentioned it before:  
[https://randomprojectionai.blogspot.com/2018/02/neural-network-weight-sharing-using.html](https://randomprojectionai.blogspot.com/2018/02/neural-network-weight-sharing-using.html)  
However I don’t wish to take any of Uber’s thunder away.