I suppose you could have one large controller network (CN) deciding what to write into and read from a single vast associative memory (AM.) The CN filtering (sensory) input information and deciding what to write where in the AM etc.
However intuitively I think it would be better to have lots of little CNs linked to lots of little AMs. Each AM could specialize in remembering a small number of specifics, such as a sequence item in a task to be done and completed or not completed yet. Maybe you would have to look into the mathematics of tessellation for some inspiration about how to arrange the CN and AM components. https://en.wikipedia.org/wiki/Tessellation