You are coming at this like a programmer and not thinking of this with a wide enough view of the possibilities. The brain is all about memory but it does not have a space for memory to be recalled, processed and so forth like you relate - it all happens in the same place and at the same time with the same parts.
In this thread on memory I related that the process of perception is actually an act of active recall of prior memory fragments to match the current perception:
What is added to the existing memory is what is different, the delta of perception - the surprise in the perception.