Sure, LLMs are not real AI, but they accomplish a lot of what we wanted to achieve with AI, so let’s get started using them.
NOTE: You don’t have to be tied to a cloud service; it is now within the grasp of any small engineering team to bring in a resident AI research assistant that becomes utterly indispensable.
Note this is all off-the-shelf software and the tech to assembly these into a functioning package is a single modest GPU card.
I think it is just a matter of time until a linux package installs a functional system with no fuss or muss.
3 Likes
I was aware of bitcoin very early on. I did not have the IQ to get in on the act. A smart person would just realise.
I’m sure likewise with the above information.
Off-topic. An important event in the history of artificial neural networks was the discovery that convolution made neural network image processing possible. Neural networks + convolution.
I wonder if associative memory + ?? could out perform deep neural networks? Where ?? is just some simple additional thing. Or perhaps you hardly even would need ??, just have a better understanding.
Unfortunately there is absolutely zero money in addressing that question, unlike bitcoin, leveraging current neural networks, etc.
You just end up on loser street looking into such questions.
2 Likes
Vector databases is what is already used as arbitrary large associative memory. How effective they are it’s a different story, but that also depends on what else is using them, how they are used and the quality of embeddings used for storage
1 Like
Such databases need efficient search queries/algorithms in order for then to be useful. These queries are usually aligned to significant features of the dataset such that the variation of a single (or few) parameters will produce a desired change in the sampled result.
Therefore, I would suggest that the piece that is missing from @SeanOConnor 's post is a means for discovering meaningful structure in the observed data and also in the interplay between the agent’s action space and the affordances (exploitable parameters) of their environment.
1 Like
That’s called clustering
1 Like
Clustering is one way to find groups of similar samples in a larger population of data. But finding similarities and differences in static data samples may not be sufficient to extract the desired features from the data.
For example, an agent may need to learn about the behavior of objects in the world under different environmental conditions including manipulation by the agent itself. To accomplish this, the agent would also need to learn a dynamical model of the world and the objects in it before it could begun to form meaningful groupings of objects (e.g. objects that open and close, objects that can be folded, food that is safe to eat only after it has been properly cooked, etc).
Having an associative memory is important to this process, but how the agent makes relevant associations between memories will largely depend upon this other aspect related to model building/simulating/inferring.
2 Likes
It’s still clustering, just with time as an additional dimension. If you need to do it online, that’s connectivity clustering, = graph construction.
1 Like
Taking a position (eg. clustering) and then exploring from that position is a good science tactic.
I am interested in finding out a little more about clustering at the moment.
2 Likes
There are basically two kinds of clustering: connectivity-based, by pairwise similarity, and centroid-based, by group-wise similarity, in various combinations. My scheme starts with connectivity, which then alternates with centroid-based, in cycles. What makes it different is indefinitely deep hierarchical parameterization by derivatives from my comparison functions.
2 Likes
You can cluster with associative memory pairs or triplets or whatever.
If they have all seen the same thing before the output values or vectors will tend to be the same.
If they have large vector outputs those will tend to be orthogonal for unseen inputs.
It is probably better to use random projection (RP) based associative memory for that. As each associative memory can be constructed with different random projections and essentially ‘view’ the input information in different ways.
1 Like
I have “associative memory”: each compared node references all other compared nodes. It’s just a name, the devil is in the details. I find that to manage complexity, the design must be organic.
1 Like