Repo for merging various Encoders

Engineering
1

Hey all,

for the community repo we’re in the need for more c++ encoders.

I was thinking about that and I’d like to stratch that project and create a shared base of encoders + HTM system. A universal environment for running HTM systems and experiments.

First of all, I’ll be developing, merging and looking for help, ideas of specialized encoders (various modalities- vision, hearing, touch, language, chess positions, numbers, …) So if you have a specialized HTM project, please do hit me up and we can join efforts.

Second, for the basic/common encoders, such as expressing cathegories, integeres, real numbers, …even images, video -> there exists tons of encoders for other ML frameworks (DNN).
Is there a common (c++, open-source, active) collection of such encoders? So we could just reuse them. As encoder “just” provides a binary vector from the world, rest is up to HTM.

2 Likes
2

Cool idea!
What are the requirements for a encoder to be accepted? Are very experimental encoders good (ex: GridCell, Bitking’s Hex grid, vision encoder). Are handy but trivial to implement encoders allowed (Ex: Frequency encoder (FFT + scalar encoder))

1 Like
3

I experimented with encoding vision and here is what I came up with:

  • Use openCV’s retina algorithms. It’s well researched and it works well.

  • I came up with a good method of encoding the resulting image into an SDR. I describe it here: Encoding vision for HTM. I think that this encoder could also work for audio data, but I haven’t tested audio.

1 Like
4

definitely! We do support some experimental(research) features in Connections, SP, etc. So encoders for doing active HTM research are welcome.

yes, probably to some degree (we won’t want to end up with 1000 encoders), but so far there are none, so any reasonable code is good.

First, I’d like to make the encoders part of nupic.cpp, as are part of python repo. If the project takes up, I was thinking of making a separate repo “nn.encoders”, so even non-HTM projects can join on the encoders…

1 Like
5

This is good!
Now that we have a working vision (MNIST) example MNIST example by breznak · Pull Request #242 · htm-community/htm.core · GitHub , once I get to merging the “image encoder” from nupic.vision, I’d like to compare that with your VisionEncoder!

6

What do you mean for the nn.encoder repo? I think HTM and SNN are the two only models that needs a encoder. (But they encode data differently tho)

7

well, the initial idea was for htm community, so each project does not have to implement encoders. But then nupic.cpp (with bindings) might be just the right fit.
Then I thought other networks (not only HTM, Spiking…) need encoding, for all DNNs (autoencoders, …) but then the architecture/requirement for encoding will be different, so that idea (shared encoders for many NNs) probably won’t kick.

8

I’m happy to announce a number of (most of the numenta’s py) encoders in c++ added in

The encoders are working and with tests, we’re looking for volunteers to help them integrate within the repository (may need some fixes regarding multiplatform support, modern c++11/17 etc)

2 Likes
9

@breznak long time coming, thanks!

10

We’re now discussing how to implement “decode” operation.
If it makes sense to have decode in encoders (but not rest of HTM pipeline supports it), if classifier is enough, and where decode must exist.

Please join with ideas & help in https://github.com/htm-community/nupic.cpp/issues/297
Biological way to decode SDR back to Objects