Brains@Bay Meetup: Sparsity in Neural Networks - Aug 19, 2019

If you are in the Bay Area, RSVP here:

If you can’t make it, watch the live-stream here:

2 Likes

This is going to be an interesting meeting. We’ll have RAIN Neuromorphic CEO Gordon Wilson talking about some chips that really got me excited last week. Also @subutai will present on “Sparsity in the neocortex”.

If you are in the Bay Area, please RSVP, and I’ll see you there.

If you can’t make it, I will be live-streaming the meetup.

2 Likes

Here are a couple of papers if you want to read up on their tech:

Pay close attention to how they use memristor-like mechanisms in coated nano-wires to create “synapses” between neurons (electrodes). This enables sparse connectivity and super fast matrix multiplications.

2 Likes

This looks incredible indeed.

Did they actually build a (prototype) chip, or is this still theory?

Will they talk about microcommands how these memristors are read and written?

Many thanks to RAIN Neuromorphics for this presentation!

I think this is their slide deck for a previous presentation:

This event is live now.

1 Like

This was great. Thanks so much!

1 Like

Big crowd compared to when I was there a couple of months ago! Were these mostly Numenta community members?

1 Like

No they were new folks. :smile:

1 Like

Could it be the free pizza?

@subutai - Seriously - in the opening video sequence (time index 30 minutes) there is a twinkling pattern of firing dots.
How far apart are they in relation to the size of an SDR (dendrite span) in the cortex.
How many of those dots are needed in a 40 um section to form an SDR in the next map?
The neocortex is not fully connected - an SDR can’t exist outside of a single dendrite.

For the cells I see firing I don’t see the activity that we call brain waves. Why is this not visible in this visualization? In a different presentation with an array of probes, the wave action was clearly visible as a traveling wave pattern. (time index 4:00)

The time sparsity really can’t be useful if it is much outside of 2 alpha cycles ( about 200 ms). This is the discrete window for judging simultaneous events.

The attendee question about fire together/wire together is key to spike timing networks and this window is very well defined. For example this this article:
“When it comes to laying down memories, the relative timing of spikes seems to be as
important as the rate of firing. In particular, the synchronized firing of spikes in the cortex is
important for increasing the strengths of synapses—an important process in forming longterm memories. A synapse is said to be strengthened when the firing of a neuron on one side
of a synapse leads the neuron on the other side of the synapse to register a stronger response.
In 1997 Henry Markram and Bert Sakmann, then at the Max Plank Institute for Medical
Research in Heidelberg, discovered a strengthening process known as spike-timingdependent plasticity, in which an input at a synapse is delivered at a frequency in the gamma
range and is consistently followed within 10 milliseconds by a spike from the neuron on the
other side of the synapse, a pattern that leads to enhanced firing by the neuron receiving the
stimulation. Conversely, if the neuron on the other side fires within 10 milliseconds before
the first one, the strength of the synapse between the cells decreases.”