I have created a knowledgebase/graph neural network architecture (GIAANN prototype) which can be used to predict the next token in a sequence. It trains a set of columns for every new noun encountered in a textual corpus, and feature neurons for every contextual word (non-noun) directly surrounding each noun. It supports distinctions in dendrite proximity (SANI; sequentially activated neuronal inputs), and is based on neural assembly and cortical column theory.
It is currently citing “Hawkins, J. et al. (2011). Hierarchical Temporal Memory (HTM) Whitepaper (Version 0.2.1). Numenta”, although they may have more recent preferred citations.
Originally posted on Column/assembly graph neural network - Research and Theory - Thousand Brains Project