If you think a context dependent chain of linear mappings is a good approach then you have a lot of opportunity to include context signals often seen in biological systems:
Principle: Separate Data from Control
The data pathway carries the quantities to be transformed. The context control pathway carries persistent state that selects, gates, or modulates the computation performed on that data.
Rather than encoding all contextual information within the feature representation, context is communicated through a dedicated control pathway that determines which local computation is executed. This separates what is being computed (the data) from how it is computed (the context).
Examples of context include behavioral state, temporal state, internal clocks, spatial or navigational state, physiological state, safety state, and task or mission state. These variables evolve on their own time scales and need not be rediscovered from instantaneous sensory input.
- Finite automata → behavioral context.
- Internal clocks → temporal context.
- SLAM or place cells → spatial context.
- Energy level → physiological context.
- Atlas/LSH bits → geometric computational context.
- User goals → task context.