How to Run HTM School Visualizations

Engineering
,
1

You should be familiar with basic programming environments to get these visualizations running properly. There are several pieces to this, and 3 different repositories involved.

System Requirements

  • Python 2.7
  • Node 6.10+

Dependencies

Warnings

These codebases are experimental. That means two things:

  1. They may not work for you.
  2. Issues may not be fixed immediately.
  3. These visualizations are best viewed at least 2500px wide).
  4. I cannot count.

I typically develop this codebase against NuPIC tip of master. It may not work with earlier versions.

Bugs / Contributions

Please create issues and pull requests on individual repos. If I can validate they work on my local system I will generally merge them.

Screen Shot 2017-05-23 at 10.36.32 AM.png1000Ɨ693 389 KB

Three Parts

  1. HTM Server: https://github.com/htm-community/nupic-history-server
  2. HTM 3D Animation: https://github.com/numenta/cell-viz/
  3. HTM Visualization Client: https://github.com/htm-community/htm-school-viz

The HTM Server is an HTTP server that runs an HTM system. It exposes REST endpoints for clients to create models, send data, receive results, and query for HTM cellular state. This is currently implemented by the NuPIC History Server, but could be another HTM implementation that matches the HTTP interface.

The HTM 3D Animation is used in later episodes. It is a THREE.JS project called cell-viz that is called to create specific canned HTM animation scenes.

The HTM Visualization Client is a web server that displays a web interface for user interaction. It contains code to interface with the HTM Server and display results to the user using the HTM 3D Animation if necessary. All of the episode-specific code is here.

Step One: Checkout the code

Checkout or download the codebases and above and set environment variables to the absolute paths to their locations on your system.

export HTM_SERVER=<location of https://github.com/htm-community/nupic-history-server checkout>
export CELL_VIZ=<location of https://github.com/numenta/cell-viz/ checkout>
export HTM_VIZ=<location of https://github.com/htm-community/htm-school-viz checkout>

Step Two: Start the HTM Server

REMEMBER! You must have NuPIC installed properly for this to work.

cd $HTM_SERVER
python webserver.py 8080

This should start the HTM server on http://localhost:8080/. There is no web interface for this server, so if you hit it with your web browser, youā€™ll just get back NuPIC History Server in the HTTP response.

Step Three: Setup the HTM 3D Animation Web Package

Build the bundle

The cell-viz project needs to build a web bundle for the Visualization Client, so we need to package it up first.

cd $CELL_VIZ
webpack

You should see some output to the console and a compressed JavaScript file at out/dyson.js.

Link the Bundle

The Visualization Client expects this bundle to be in its file structure, so run this command to link it.

ln -s $CELL_VIZ/out/dyson.js $HTM_VIZ/static/js/third/dyson-bundle.js

Step Three: Run the HTM Visualization Client

cd $HTM_VIZ
npm start

The user interface should be running on http://localhost:8001/. This server will interface with both the user and the HTM Server running on http://localhost:8080/.

Notes:

In the 3D animations, use the WASD keys to turn, arrow keys to go forward, backward, or strafe.

9 Likes
2

Iā€™ve been asked by several people in and outside our community how to run the HTM School animations. So here is a guide. If you have problems, please post here and Iā€™ll try to keep this wiki page updated with the most recent instructions.

1 Like
3

I could use some help with the Ep8 learning comparison example. I have a couple problems.

  1. Itā€™s so slow. It appears that it is writing data to file. And then after I pause the simulation and inspect a single columnā€™s history, it takes about 1 second per iteration to retrieve the data. Which means in practice, I canā€™t really use this to demo since just inspecting a single column takes 5 minutes to load. (Part of this may be because I have an encrypted drive.)

Is there some way I can verify that Redis is actually being used? Iā€™m not familiar with it and can only guess. I tried to list any keys that might be active from the CLI but so far itā€™s empty. Plus, the /working directory is full up with lots of files which might not be the correct behavior.

  1. The similarity dots donā€™t exhibit the same behavior as in the youtube demonstration. In fact, the performance of the random SP and the learning SP appear indistinguishable. I will attach an image if I can.
    Screen Shot 2017-05-31 at 10.35.28 PM.png1393Ɨ707 84.7 KB
4

Thanks for reporting. I switched from Redis storage to file storage for SP state storage using capnp serialization awhile back. I still use Redis for some stuff, but not for that. I know it takes forever, but I needed to do it for later episodes. I should have both persistence methods in place I guess. Sorry I donā€™t have time to look into it right now, but Iā€™ll be coming back to this codebase at some point in the future.

5

By the way, I may not even be using these codebases for future sensorimotor integration visualizations.

6

I understand. However, itā€™s the only interactive demo you have that shows actual spatial pooling learning.

The cell-viz in Ep9 and Ep10 kind of works on my system, but it doesnā€™t allow me to drill down into columns, their connections, and how they change over time.

Do you think the performance indistinguishability shown on the two spatial poolers is a symptom of the code changing out from underneath it?

7

I canā€™t say without investigating. I may have introduced a bug in one of those episodes when I did the persistence layer swap.

8

Perhaps you know the version combination of htm-school-viz and nupic-history-server that you used for the HTM school video of Ep8.

I did my best and checked out the commits right before the HTM School video was published. However, I think you did some fine tuning either during or before the filming the actual video. Maybe you know the last known combination that worked?

I just tried these together which is around the video date of Aug 19, 2016:
https://github.com/htm-community/htm-school-viz/commit/951cbe23a229634ba24867bf82bcacbfd31cc531
https://github.com/htm-community/nupic-history-server/commit/2c4c95d890758e24eab3680c3a6daa6edf581644

9

@jacobeverist I had to go back in time a ways, but here is a combination that works:

10

Thanks! Iā€™ll give it a try.

11

Hi Matt,

Iā€™m trying to run the visualizations on windows 7 (which is probably my first mistake), but I have been unable to get the webserver running. When I run the code

python webserver.py 8080

I get the error

no module name capnp

So I try to install pycapnp, but I keep getting the error

no libcapnp detected

As an aside I already have MinGW GCC for Python as well as mingGW 64 installed. Any help would be much appreciated.

12

You must have NuPIC installed properly. Did you pip install nupic?

13

Yes, and I have been using nupic 0.6.0 for a while now without a problem.

14

As stated in the original post, youā€™ll need to develop from tip of master.

15

Hi,

When I run ā€œwebpackā€ in $CELL_VIZ, the ā€œoutā€ folder is not createdā€¦ Hereā€™s what I get when I run ā€œwebpackā€ (after installing it globally, as was suggested in Fails to run on custom port Ā· Issue #32 Ā· htm-community/cell-viz Ā· GitHub):

Hash: 1bb19d06e3b611a7a026
Version: webpack 3.8.1
Time: 939ms
Asset Size Chunks Chunk Names
cell-viz-1.1.2.bundle.js 1.2 MB 0 [emitted] [big] main
[1] ./src/BaseGridVisualization.js 7.88 kB {0} [built]
[4] ./src/HtmCells.js 3.33 kB {0} [built]
[5] ./src/InputCells.js 3.77 kB {0} [built]
[6] ./src/HtmMiniColumns.js 4.83 kB {0} [built]
[7] ./src/CompleteHtmVisualization.js 10.7 kB {0} [built]
[8] ./src/SingleLayerVisualization.js 2.32 kB {0} [built]
[9] ./src/SpToInputVisualization.js 3.44 kB {0} [built]
[10] ./src/HighbrowLayerVisualization.js 11.1 kB {0} [built]
[11] ./src/HighbrowColumnVisualization.js 9.1 kB {0} [built]
[12] multi ./src/threejs/renderers/Projector.js ./src/threejs/controls/FlyControls.js ./src/threejs/textures/threex.dynamictexture.js ./src/HtmCells.js ./src/InputCells.js ./src/HtmMiniColumns.js ./src/BaseGridVisualization.js ./src/CompleteHtmVisualization.js ./src/SingleLayerVisualization.js ./src/SpToInputVisualization.js ./src/HighbrowLayerVisualization.js ./src/HighbrowColumnVisualization.js ./src/HtmViz.js 172 bytes {0} [built]
[13] ./src/threejs/renderers/Projector.js 21 kB {0} [built]
[14] ./src/threejs/controls/FlyControls.js 7.37 kB {0} [built]
[15] ./src/threejs/textures/threex.dynamictexture.js 4.85 kB {0} [built]
[16] ./src/HtmViz.js 1.52 kB {0} [built]
[17] ./src/HtmCellStates.js 1.5 kB {0} [built]
+ 3 hidden modules

Any ideas on how to make this work? Thanks in advance!

16

It creates a bin folder, not out. You should have bin/cell-viz-1.1.2.bundle.js.

17

Thank you for a prompt reply!
I was able to make it run by doing

ls -s $CELL_VIZ/bin/cell-viz-1.1.2.bundle.js $HTM_VIZ/htm-school-viz/static/js/third/cell-viz-1.1.2.bundle.js

I also had to ā€œnpm installā€ a bunch of modules: express-http-proxy, express, express-handlebars, serve-static, glob.

I wanted to run HTM School Visualizations to see cell activations for arbitrary sequences. Iā€™m struggling to understand how HTM ā€œmergesā€ contexts. For example, if an input sequence is ā€œA,B,C,D,C,B,A,B,C,D,C,B,A,B,ā€¦ā€, how does HTM figures out that the context of the first, third, and fifth ā€˜Bā€™ is the same. I couldnā€™t find explanations about this in the HTM papers. Iā€™ll make a separate post about this.

Unfortunately, itā€™s not possible to check visualizations for arbitrary sequences with this toolā€¦ Itā€™s too bad, cause now I guess I need to dig into the TM code :frowning_face:

18

Join the club! :wink:

19

Yes digging into the TM is definitely how youā€™ll understand context in HTM, thatā€™s the TMā€™s job. Thereā€™s good pseudocode here:

20

I do have newly created bin/cell-viz-1.1.2.bundle.js (which replaced the older version downloaded from the github through git clone command). What I donā€™t have is out/dyson.js file as stated in the tutorial above. Perhaps I am supposed to ignore commands related to out/dyson.js stated in the tutorial above and the github repositories???

Linking cell-viz-1.1.2.bundle.js helped (Thanks @oiegorov :slight_smile:), letā€™s me run visualisation 11.

For visualisation 10 though, I continue to get a page with only the text: ā€œEp10/Choose Gifā€ :frowning_face: