Triadic Memory — A Fundamental Algorithm for Cognitive Computing

Science Related Papers
#142

If you are familiar with python here is a simplified version of address computing.
the sdr is basically an ordered list of 1 bits.
See below if all bits from 0 to 6 are 1, then the resulting addresses fill the space from 0 to 20 (21 address pairs)

def address(sdr):
    for y in range(1,len(sdr)):
        for x in range(y):
            yield (sdr[y]*(sdr[y]-1)//2+sdr[x])

[a for a in address([0,1,2,3,4,5,6])]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]

To be more clear how bits are paired:

def bit_pairs(sdr):
    for y in range(1,len(sdr)):
        for x in range(y):
            print(sdr[x],sdr[y])

bit_pairs([4,5,6])

In my implementations I need to make sure the SDRs are ordered.

1 Like
#143

@POv Yes. I have seen the current C-version in github. It looks now correct and has a better performance than my version. Thanks for your update.

#144

@POv Could you please share C-version of SDRnoise so that I can test and compare some results with your Mathematica Dyadic or MonadicMemory? Thanks

#145

I’ve just finished my first pass at an implementation of triadic memory in Rust:

My very limited testing shows that it works, though I’ve not yet done benchmarks (it should be at least as fast as the C version, though).

I really struggled to understand the binarize function in the C code, but eventually it clicked. One reason for my confusion was the use of an SDR as the input, since the values in the SDR’s array were not interpreted as indices where there’s a connection, but rather connection weights inside the memory itself. So, I moved that functionality into a constructor for SDRs that takes an array of connection weights as input:

    pub fn from_weights(weights: &[u8], pop: usize) -> Self {
        // first, find the "pop"-largest connection weight to use as a threshold value
        let uniq_weights: HashSet<_> = weights.iter().copied().collect();

        // using a heap means we only have to sort `pop` elements instead of the whole thing
        let mut sorted_weights: BinaryHeap<_> = uniq_weights.into_iter().collect();
        let mut ranked_max = 1;
        for _ in 0..pop {
            if let Some(w) = sorted_weights.pop() {
                ranked_max = w;
            } else {
                break;
            }
        }
        ranked_max = ranked_max.max(1);

        // now, walk through the weights and if the value is higher than `max`, add the index of it to the `ones`.
        let mut ones = Vec::with_capacity(pop);
        for (i, &v) in weights.iter().enumerate() {
            if v >= ranked_max {
                ones.push(i as u16);
            }
        }

        Self {
            ones,
            size: weights.len(),
        }
    }
2 Likes
#146

@nebkor Awesome. I modified the binarize function, removing that confusing hack. Thanks for your feedback!

1 Like
#147

Welcome.

I’m not familiar with rust but what I can tell about binarize() is it:

  1. takes in a vector of size N, the actual N and P = desired output size
  2. by sorting finds out the P highest values in the vector and uses the lowest of these as a threshold
  3. outputs the positions of all values greater or equal than the threshold. That means if there are more positions matching the threshold, then the output is longer than P
  4. except when P-th highest value is 0 in which case (I believe) it picks some arbitrary zero-s to fill up the result to P positions
2 Likes
#148

Yep :slight_smile: The code I pasted does steps 1-3 as you describe (though I pass the weights through a set to unique-ify them, and then only sort the top P elements by pulling them from a maxheap). For step 4, all I do, and all the C code does, is ensure that the threshold value was at least 1.

My SDR implementation doesn’t store a P value, it just returns the length of the array that contains the sparse bit locations, which is usually P, but could be more or less, depending on how things went with the fetch method that called the “binaryize” (from_weights) constructor.

1 Like
#149

Here are the numbers I just got, with N = 1000 and P = 11:

Inserted 10000 triples in 267ms, 37453 insertions per second
Queried 10000 triples for X in 7481ms, 1336 queries per second, total err: 0
Queried 10000 triples for Y in 4511ms, 2216 queries per second, total err: 0
Queried 10000 triples for Z in 345ms, 28985 queries per second, total err: 0

I think I need to better understand @rogert 's nybble trick to fix up those query times.