What is noise?


#1

I am struggling with concept of noise. I understand the definition of noise, i.e. cloudy/altered/partial input data point. What I do not quite understand is how this definition applies to actual data.
For example, if I am analyzing stock trading data, and I receive data in a stream, how would I know that data started becoming noisy at any point? Wouldn’t HTM treat this new data as “training” data and adjust connections and all other parameters to learn this new data? For non-biological agents, how can input become noisy? Perhaps I am missing something obvious here.
Another example could be converting to SDRs a stream of images coming from retina. If my body develops cataracts, I start seeing cloudy vision, generated SDRs will become different when compared to SDRs generated for the same picture if it weren’t cloudy. Is this an example of how my neocortex learned to see and is now tolerating noisy input? But is it adjusting its connections to “better” know how to deal with this data?

Thank you!


#2

Neural cells are never completely quiet or completely committed to signalling some value: It is a stochastic signal with populations firing at random or not firing when you might expect them too. The signal in question is mixed in with this noise.

In these recordings of nerve cells firing you can hear the randomness in the signalling:


In a more visual form:

The same could be said of the weights learned: some theory may predict that all the cells in a certain population should have their synaptic connections strengthened between this population and that - in practice the cells are increased by some rule but it is more of a strong tendency than a hard and fast rule.

The different between some mathematical ideal and the messy wetware implementation could be considered noise. This is one of the huge advantages of signals being carried by large populations of cells - individual deviations from some stochastic mean (noise) are reduced by the square root of the number of observations.

BTW: this is one of the things that give me hope that we will not have to slavishly copy all of the neurons and synapses of the brain - our models can be made with better parts with less noise.