Building HTM Systems (WIP Document)

Continuing from your Twitter thread, why does it need to have a server side component at all? And do you have an idea of the source(s) of the streaming data?

I guess I don’t really have a good reason. Something I’ll have to think about.

My original plan was to have one server with one HTM running. The server would constantly run data through the HTM. All the web pages would visualize components of the running HTM in different ways. They could use the actual HTM input / output by tapping into the server’s streams.

But now that I think more about it, I really don’t need to have a server. I could have the entire thing running in JavaScript on each browser. The data I was planning on using would be programmatically generated anyway. In that case I could just continue use parcel like I’ve been with @codeallthethingz on the other JS project.

3 posts were merged into an existing topic: 2D Object Recognition Project

Not having a server side component would make it much easier to get started with. If there’s a need to save state (tutorial progress etc), you can always persist the react/redux state to local storage.

I’m fairly certain we’ll be building a React static app. I have done a bit of prep-work at https://github.com/rhyolight/building-htm-systems/tree/react. If anyone wants to follow along or help out, here is the planning board I am using. If you think you might help, create a Trello account and I’ll start a new team. Please join me on Thursday, I’ll be working on this all day long and live-streaming.

To run, checkout the react branch on my fork above and run:

npm run dev

You’ll need node.js and npm of course, and don’t forget to npm install.

I will be putting some of the more boring tasks (like LaTeX conversion and syntax highlighting) off until we get the major changes worked out. On Thursday I will be converting the first “simple scalar encoder” visualization into a React component.

Holy cow, Brev, this site is outstanding! Exactly the kind of thing I want! And he wrote an interactive tutorial on building interactive tutorials! This is super-useful…

2 Likes

@brev
Great find.
That is an amazing tutorial.

2 Likes

I am excited to start working on this tomorrow. I’ve already got a head start, so we will hit the ground running. I’ll lay out the big vision of the project, then we’ll dive right into the React architecture. Starting with the first visual, I’ll be converting them into React components while applying lessons from the interactive tutorial linked above. Pretty quickly we’ll need to get observable data stores working. Should be fun.

There will also be some low-hanging fruit that others can help out with. For example, we’ll need code syntax highlighting, LaTeX conversion, layout / styling. These are side-projects that other folks can help with.

PRs welcome!

If you do plan on helping, please join our Trello team. I’m using it for project planning, it is a really lightweight planning tool. I think it will be just enough for Twitch projects. (I switched to GH Projects.) I hope to see some of you online tomorrow morning at 9AM PDT.

1 Like

In the first part of today’s stream, I went over the big picture for this project:

Then my internet crashed. Once I recovered, there is another 3 hours where I got one d3js diagram rendered in React (no interactions or data updates yet).

Next week I am PTO, but I will be back on Apr 25 to add observable data and interactions.

1 Like

I will be streaming on this project Tuesday and Thursday this week 9AM PDT (for most of my work day, join any time). Keep up with my schedule on the Building HTM Systems events page.

Today, thanks to help from Vratislav (who I can’t find on the forum), we did the following:

  • updated simplehtm to install properly on npm
  • imported and used encoders defined in simplehtm for client-side scalar encoding diagram
  • installed LaTeX support
  • installed code syntax highlighting support

We also figured out how to allow code highlighting by watchers during my live-stream in VSCode, which is pretty darn cool. Whole session is available to watch now on YouTube here:

3 Likes

This is a response to the React or Not React video, I figured this would be a better place for discussion.

I created a fork that shows how you could create a static site that bundles a similar way as Next.js does (so you can use modern javascript).

I think not using React is the best decision. We used React for a year long project at my software consulting job, after that we used it for 6 month software engineering project and my team regretted it both times, despite fully embracing the React ideology. We fought it all the time when trying to do simple things, and it did not play nice with anything external. Most of all, it’s a black box when things go wrong. It’s great for giant companies because it forces modularity and prevents spaghetti code, but it comes at a heavy cost. In particular, the work you want to do (talking between components) is extremely painful in React.

I strongly recommend not even using functional components because of the way React handles the virtual DOM. React only updates when it thinks it needs to update. If you have an external library changing things, React isn’t going to see those changes and it is going to start screwing things up because it doesn’t have all the information. The other issue with the virtual DOM is that it often overwrites the real dom with new nodes. Meaning if you use JQuery to grab a real node, that real node will likely get deleted and replaced very quickly with React, and JQuery is left pointing to a deleted node, which just causes more broken behavior. When you code in React, you pretty much have to code everything in React.

It is really easy to write unmaintainable Javascript, but you can write maintainable Javascript if you plan it out and document it well, and that is what I personally prefer. I actually use JSX without React.

I do want to mention Vue.js though because it might actually be what you’re looking for. You don’t need to change your existing codebase, you could write a static HTML site, and then insert a Vue component for just one graph on one page. It has events/listeners like normal Javascript, so you can do something similar to:

let component1 = new Vue({
    // as soon as the component loads, set a timer for 1 second
    // then emit an event
    mounted: function() {
        setTimeout( ()=> this.$emit('its-been-1-second'), 1000 ) 
    }
})
let component2 = new Vue({
    // Just render a div with some text
    template: '<div>{{ test }}</div>'
    data: {
        text: 'hello'
    },
})
component1.on('its-been-1-second' , ()=>{ 
    component2.text = "It has been one second!"
})
// Attach them to the DOM
$('#whereever-you-want-this-component').appendChild(component2.$elem)
$('#whereever-you-want-this-component').appendChild(component1.$elem) 

Sadly Vue still uses a Virutal DOM within it’s components, but it won’t get in the way of everything like React does.

2 Likes

@jeff-hykin That’s very helpful, thank you. I am most afraid that React is going to mess up my workflow with D3js. All my pages are going to have lots of D3js, and even though I got past my issues setting up the parent/child components, I’m still have issues with page repaints:

cc @David_Duckworth

My team had similar slowness issues with input boxes. We wanted to run input box validation (phone number, email, etc) I had to go do special hacky optimizations just to get where you could type without it lagging.

You can try storing the value outside of the component state (in any regular variable, or on this) and then just manually update the D3 element onclick/ondrag instead of updating the state. It would defeat the entire purpose of React state, but it might get the correct behavior. To get the D3 element like you would with jquery though, you have to use a thing called React References.

You can also dive into the chrome performance tools (one of the tabs in the chrome debugge) and it’ll let you pick apart the code to see what is causing the slowdown.

Maybe tomorrow I’ll look at making a fork of the React version and port it to Vue and see if the slowness is still there. Vue will let you check on a sub-value in state, and put a custom handler on it. (e.g. state.counter.onUpdate() ) If it’s still slow then it’s pretty likely it is just a virtual Dom issue.

2 Likes

I am not sure I agree with react being in the way or hampering development there is a reason it is so popular and used in production as much as it is. I have been using it without issues for a very long time. You are not using jQuery and the components are view only basically requiring only state changes when you need to rebuild the component. The page should only rerender when you change state and that “rerender” may actually not do anything if the non d3 elements haven’t changed. I am also not sure using a new framework (Vue) is the answer as it is not that different then React under the covers and any issue you run into with React you will most likely run into with Vue. I will take a look at why your page is rerending now. btw React has been shown to be much more performant then using something like jQuery.

1 Like

So first thing i notice is you are changing state on the value on hover. Is that something you want? If it is you are rebuilding the d3 diagram every time you hover which is the wrong thing to do so you would just not re-build the grid on value changes. That is why your lag is happening. Also you don’t need a my = this, you can just use arrow functions which will bind this in scope on declaration.

Let me know if you have a branch that i can pull and i can lend a hand at next live stream. Have no fear :slight_smile: there is easy fix for your issue :slight_smile:

2 Likes

Just take a look at master. It has the latest React. One thing I noticed was that closing the debugging console prevented the lag I was seeing :man_shrugging:.

I will be working on this today (offline) and tomorrow (live streaming).

sounds good, let me know if you have issues with updating value in the state (i am not sure that is what you want and if it is there is different way to do it)

yeah so i see you have sliced up my code in the SimpleScalarEncoder component breaking the updating (i can explain why it breaks it tomorrow in live stream) but i went into my fork and switched the scrubber to just use a input element for now (with high/low etc) and it is very performant (notice below the Value input changing whiile hovering in d3 changes – d3<->react very fast)…

so the issue is with your code changes that i can point out. I think it makes sense to use the input element for now instead of the number scrubber

this is the Number scrubber i changed …

import React from 'react'
import PropTypes from 'prop-types'

const style =  { 
	border: 'solid gray 1px', 
	borderRadius: '4px', 
	fontWeight: 'bold',
	padding: '2px',
}

const Number = ({ high = 100, low = 0, onUpdate, precision = 0, value = 0}) =>  <input type="number" min={low} max={high} style={style} value={value} onChange={(e) => onUpdate(e.target.value)} step="1" />
  
Number.propTypes = {
	high: PropTypes.number,
	low: PropTypes.number,
	onUpdate: PropTypes.func.isRequired,
	precision: PropTypes.number,
	value: PropTypes.any,
}

export default Number
1 Like

Skipping ahead to the SP…

We’ve made it partially through encoders, but everyone would rather skip ahead to Spatial Pooling. So we are going to skip some encoders so we can prepare for Spatial Pooling visuals.

So next up is a streaming combined encoder:

17%20PM

We’ll be working on this Thursday 9am. See my events for details. Next week, if that goes well, we’ll start on the “potential pools” diagram:

05%20PM

But we need the combined encoding first, because it will be a part of the potential pools diagram.

I hope you will be able to join me on Twitch if interested!

2 Likes