I wrote up some details of context driven learning in as simple a way as I could:
https://archive.org/details/triad-of-context-driven-learning
The triad is:
- Context
- Conditional computing
- Learning
At some stage I also mentioned that you can have a decision matrix view of ReLU neural networks.
The decision matrix D is a diagonal matrix with 1 where the corresponding ReLU x>=0 decision is true and 0 otherwise. So y=ReLU(Wx) becomes y=DWx and D does row selection on W and in fact does column selection on the weights in the next layer.
This is in fact parameter selection driven by D viewed as context.
So you have layer-wise context generation, followed by parameter selection, followed by backpropagation learning.
There is nothing in the Triad of Context Driven Learning note that limits you to layer wise context generation.
It seems quite valid to generate all context directly from the input to the neural network, should that empirically turn out to be a reasonable thing to do.
What is really being said here?
That current deep neural networks are context driven factorization of composite linear mappings.
That is a downgrade from the conception most people have. Still and all, reasonably powerful.
And taking such a mechanical view may allow better and more varied design choices.