# Ideas for better word encoding

**URL:** <https://discourse.numenta.org/t/ideas-for-better-word-encoding/992>\
**Category:** Numenta Theory\
**Tags:** encoders\
**Created:** [July 12, 2016, 11:00pm UTC](https://discourse.numenta.org/t/ideas-for-better-word-encoding/992 "2016-07-12T23:00:10Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![mraptor](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/mraptor/32/4469_2.png) [@mraptor](https://discourse.numenta.org/u/mraptor)\
**Post date:** [July 12, 2016, 11:00pm UTC](https://discourse.numenta.org/t/ideas-for-better-word-encoding/992/1 "2016-07-12T23:00:10Z")

</div>

Ideas for better word encoding …

I’m testing different variations of encoding WORDS (of characters) and passing them trough bbHTM:SPooler to see what happens.

Currently I tried 3 variants :

1. Use 5 bits for all possible 26 combinations (dense)
2. One-hot 26 bit vector
3. 3 active bits out of 78 bits per character (i.e. category encoder)

So far the option 3 behaves the best…  
I compare the SP generated word-SDR and I get for example :

13 bits overlap for “interface” and “intersection”, which is good … the interesting part is that I get zero-overlap for  
"management" and “measurement”, you can see why if you put them one below the other :

```
 interface
 intersection

 management
 measurement

```

Do you see how the second example “slip” by one character 🙂

The question is do you have some other ideas for Encoder that will capture the slippage ?!

---

<div class="post-metadata">

**Author:** ![garry.morrison](https://avatars.discourse-cdn.com/v4/letter/g/dc4da7/32.png) [@garry.morrison](https://discourse.numenta.org/u/garry.morrison)\
**Post date:** [July 13, 2016, 3:41am UTC](https://discourse.numenta.org/t/ideas-for-better-word-encoding/992/2 "2016-07-13T03:41:16Z")

</div>

What about letter ngrams? That’s what I use for my project.  
Here is the python to convert a string to ngrams:

```
def create_letter_n_grams(s,N):
    for i in range(len(s)-N+1):
      yield s[i:i+N]
```

Using 1,2,3 letter ngrams I get these similarity results:  
45.8% interface, intersection  
47.8% management, measurement

---

<div class="post-metadata">

**Author:** ![mraptor](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/mraptor/32/4469_2.png) [@mraptor](https://discourse.numenta.org/u/mraptor)\
**Post date:** [July 13, 2016, 6:56pm UTC](https://discourse.numenta.org/t/ideas-for-better-word-encoding/992/3 "2016-07-13T18:56:00Z")

</div>

That’s good, but I want to pass to SP encoded-word not encoded-string !  
Also how do you convert them to binary ? what encoding do you use for the n-grams ? If I use 2-grams I no longer can encode it with 26-category encoder ( per char), because there are too many 325! combinations !