Using a Classifier with htm.core for Skeleton-Based Human Activity Recognition

Engineering NuPIC
1

Hello Everyone,

I am working on a research project involving Human Activity Recognition using the NTU RGB+D 3D skeleton dataset, and I am currently building a pipeline with htm.core to process the skeletal data. So far, I have implemented encoders for 3D joint coordinates and have successfully passed the encoded SDRs through Spatial Pooler and Temporal Memory components of htm.core.

Now, I am looking to implement/integrate a classifier preferably something like the SDRClassifier or any HTM-compatible approach—to map the temporal memory outputs to activity labels (e.g., walking, sitting, falling, etc.). However, I couldn’t find a clear example or guide within the htm.core repository on how to do this.

  • Is the SDRClassifier available or supported in the current version of htm.core?
  • If not, what are some recommended ways to implement a classifier that works well with the SDR outputs from Temporal Memory?

Any help or direction would be greatly appreciated. Thanks

1 Like
2

A meaningful result could be possible by training any classifier on the activation state of each cell.
If TM is organized as (e.g.) 1000 columns by 16 cells each, then you can compound a SDR of 16000 bits which can be used as training input to the classifier.

Which classifier algorythm?
Limits might be imposed by:

  • how long training time/how big a hardware can you afford - because the input space bit vector could be quite large
  • whether you need continuous online training or not.

I think the SDRClassifier in particular is a simple linear classifier which is designed to be fast on SDR inputs and is online too, it can be fed one SDR at a time for both inference and training.
Its limits are simplicity of linear classifying which is equivalent with a fully connected NN with no hidden layers. So you might be tempted to test any other classifier from (e.g.) ScikitLearn toolkit like random forest, NN, etc.