Good questions from cezar_t, and your direction is interesting. I have been working on a related problem from a different angle (hypothesis-centered lifelong learning with neuromodulatory control, SQLite substrate, no GPU). A few notes that might help, especially on concept formation and scale.
How concepts show up
In pure associative memory, “a new concept appears” often means a new stable attractor or a dense enough cluster of associations. That can work for binding and recall, but it does not answer when something is worth promoting onto a global symbolic map.
What we found in practice: you need an explicit intermediate layer between raw experience and committed knowledge. In our system that layer is context hypotheses. They are cheap, revisable, and not yet facts or graph nodes. Promotion only happens after repeated evidence, consolidation (including sleep or replay-style passes), and a critic or gate. Most hypotheses never graduate. That is intentional.
So the “brain” does not invent a concept because a stream looks novel. It accumulates candidates under uncertainty, lets them compete under consolidation pressure, and only then allows a small commit. Without that, continuous learning fills the store with blends and composites that feel rich but do not stay usable as discrete concepts.
Blending separate inputs into composite associations is a real strength of your setup. For language-level concepts you still need a selection story: what survives, what merges, what gets discarded, and under what budget. Capacity alone does not do that.
Scaling toward something like GPT-2 level prediction
I would not bet on “one associative memory, just bigger vectors and more active bits” as the main scaling path. One component storing millions of associations is fine for episodic or binding memory. Predictive language use needs competing pathways, staged memory (working → episodic → longer-term), and a way to keep the write surface narrow so noise does not become structure.
Rough intuition from our side:
• Vector width and sparse active bits set binding capacity and noise tolerance.
• They do not set abstraction quality.
• Abstraction quality tracks how hard graduation is, how consolidation is scheduled, and whether the system has an external complexity source (corpus, interaction, structured input). Empty self-play on a tiny seed does not bootstrap arbitrary capability.
For text prediction specifically, you probably want:
1. Stable high-level codes from something HTM-like or equivalent (your SHR path).
2. Associative memory for binding and fast exact-ish retrieval.
3. A symbolic or quasi-symbolic layer that only accepts highly selected structure (lexicon, relations, constructions), not every composite the memory can form.
4. Multiple memory components with different timescales, not one monolith.
On central vs thousand-brains: distributed competing units help with robustness and specialization. You still want a thin global commit path, or the “concepts” never become shareable across units. We keep productive fact and relation writes locked until the candidate flow is proven stable. That is painful early on and very useful later.
Where your unique properties still matter
Constant-time exact neighbor behavior, subset pattern matching, continuous learning without heavy index rebuilds, and superposition traversal are real advantages over bog-standard ANN plus inverted indices. I would keep those as the memory substrate, and put concept formation in a gated pipeline above them rather than expecting the memory itself to decide what is a concept.