I’ve written a proof of concept variation on semantic folding to explore the possibility of a universal encoder (after a discussion with @jordan.kay on this thread). My implementation is intended to be a bit more general purpose than cortical.io implementation, though.
Their implementation, as I understand it, requires a complete data set ahead of time for pre-training (each snipit from Wikipedia is known ahead of time and given a specific coordinate on the semantic map). My implementation instead is trained on the fly, modifying on its encodings of a particular input over time, using the concept of eligibility traces from RL to establish semantics from other nearby inputs.
The system assumes causality is an important element of establishing semantics (the more often two inputs are encountered near to each other, the more overlap they will have in their encoding). So far I have only used it for generating word SDRs, though… next step is to give it a non-language problem to solve, such as the hot gym data.