A Bloom filter is a compressive (vector,boolean) hash table that can tell you if you have seen something before. There are compressive (vector,float) analog like hash tables (AHT) that can tell you if you have previously seen a vector and how alike it is to the original.
The human nervous system is very high dimensional input, low dimensional output. There is a huge divide between the two. Current deep neural nets more or less operate in the full dimensional space, perhaps unnecessarily. It makes it very difficult to obtain smart behaviors out of them, other than say “just” object recognition.
The training algorithm is being overwhelmed with dimensions.
The idea I have at the moment is to use a moderate number of <vector,float> analog hash tables with a small but very smart neural network controller.
It receives input values from all the AHT’s and independently decides when a particular AHT learns something new. That kinda solves how to integrate memory into a neural net and at the same time the memory is being used efficiently due to compression. Obviously you would not use back propagation to train such a net. The net is small - you can use a powerful evolutionary algorithm instead.
That is a simple idea but it is true that researchers have not thought up fully rational, coherent ways of integrating dynamic memory into deep networks yet. By putting memory in front of the network the system might also learn how to exploit that to do part of its calculations via lookup.