HTM in Godot Engine

I am interested in getting an HTM implementation for the open source game engine Godot.

The reasons for this are:

  • It would be a fun way to play with HTM.
  • HTM has potential applications in computer game AI.
  • Visual simulations of HTM are a good way to promote understanding and interest.

I am not sure the best way to proceed however. The options appear to be as follows:

  • c++: It is possible to create an api wrapper for a c++ lib in Godot. This would mean being able to use the main implementation of HTM, but I think it would be a lot of work and I was not able to find the API documentation after a small amount of searching.
  • c#: If there is an existing c# implementation that would be great, but it is also the language I would use if I was to create my own implementation. c# is supported by the engine and I am familiar with the language (I am not familiar with c++ at all). I feel like creating a minimal implementation could be easier than creating a wrapper for the existing one, at least for me. I may be massively misjudging either or both tasks.
  • Python: Godot script is a dialect of python, and it may be possible to port the python HTM stuff. I doubt this is a viable option for a variety of reasons but I add it for completeness.
  • Other: I am open to other options I might not have considered.

If anyone has any suggestions, links to sources or insights, I would be very grateful.

4 Likes

Welcome to the HTM forum!
I have another idea.
How about using sockets if they’re supported by Godot?
Let them run separately and communicate by sockets.
It might not be so efficient but it’ll be easy and fast to implement!

It is an interesting suggestion. In essence what you are proposing is that the Godot instance would fire up a separate program that ran an HTM engine, right?. I am not sure how to do that but I will look into it. If you have any links to similar projects or writeups of running something like that I would be very grateful.

Would this involve using nupic core, or the python tools, or would a different implementation work better?

1 Like

Googling something like ‘using sockets to communicate between processes’ might work.
I’ve never had an experience with Godot so I’m sorry but can’t help you more specifically.

Anything you like! As long as it supports sockets. Which are abundant.

I suggest you look at the community fork of NuPIC Core. It is all C++ with pybind for Python integration. Look at the pybind files to find the interface there. You would probably need to create some bindings for golang. You might look at the example of the pybind SP bindings to get an idea of the interface to expose.

Hi Duncan

Can you please share why you chose Godot and not other engines (such as Panda3d, for example) - I’m also interested in such HTM visualizations in a game engine and looking at other options right now …
Thanks

Lior

I was not aware of panda3d. It actually looks quite interesting. Part of the reason I was interested in Godot was because it supported python, but now I am confronted with the fact that it is in fact a hybrid dialect of python2 which is not what I had in mind. The tip is much appreciated and I will look into panda further. Nupic is python2 though so that does not solve the problem, or is there a working version of that for panda already?

I landed on Godot because of the bullet physics engine, which has a good reputation for speed and efficiency. This has no bearing on HTMs but it is relevant for the wider project I am interested in. That project has a small demo implementation here https://github.com/Dunkhan/DarwinsGarden
I am very much open to other options. For a long time I was a unity developer, but I am looking to broaden my horizons and I feel more and more that Unity is inferior to open source options.

If you have other suggestions for engines that I should consider please share them, and if there is any engine with a decent physics simulation that supports a nice scripting language and already has an easy way to interface with an existing HTM framework I would be very inclined to switch. The code I have needs to be rewritten anyway.

Hi Duncan,
I’m going for webots as very realistic robot simulation engine (including vision and sensors) and checking ogre3d as visualization/debug tool for htm structures (all in C++) but it’s still under investigation…

I looked at ogre, but I don’t have the c++ skills. Link your project once you have something to show though, I am very interested.

Numenta has some research code that links NuPIC to the Unity 3d engine, I can’t remember if it was ever released into the open, but it could be helpful in getting you started.

Thanks Brev,
I looked at unity but it looked too complex … I don’t really need a game engine (physics, collisions etc…) Just a rendering engine to show a column and GC in action… for that Ogre seems perfect …

Lior

1 Like