Hello everyone.
I watched videos from HTM school, but i dont figure out how to do HTM from input.
can you explain for me to understand clearly?
1 Like
Hi @tikky. NuPIC, it its current form, is not optimized to do image classification. I know some in the community have run NuPIC and other HTMs on MNIST, but we don’t get good results. You must understand that the current implementation in NuPIC is focused on temporal sequence memory, and image classification is a spatial problem, not temporal.
That being said, there is a nupic.vision repository with some vision experiments, but I have never used it, and we don’t generally support it as we focus on supporting core algorithms instead. I welcome any other community members who have done image processing with HTM to speak up.
I do believe that future advances in sensorimotor inference will help with image classification by driving saccadic movements over images, but we are not to that point yet.
Hi @rhyolight. I think now i will use HTM for temporal sequence memory. but i dont understand how it work. i can understand SDRs…, but i dont understand how connect them in a project. and can you give link about temporal sequence memory HTM. thank you.
I’m glad you asked! I just finished this Quick Start for NuPIC that should help. @marion and I are also working on a Quick Start for the Network API as well.
Also, take a look at these tutorials that build a NuPIC application from scratch. They are a bit older, but still mostly relevant.
Is anyone using NuPIC to analyze video for anomalies? Each frame would be time-stamped and the frame image would be a long string of data. An entire video clip would then be a table. I have no ideas what would be detected as an anomaly, but it would be fun to try.
How long, and is semantic information stored in the string somehow? If it is just some kind of hash I don’t think it will work. One thing people have thought of in the past is to use a DL system to extract features from each image, then encode the features into an SDR. That is what Frank Carey was going to try in this hackathon project:
1 Like
It’s an interesting question, what semantic information might be found in a sequence of video frames. I was thinking more along the lines of detecting anomalies, perhaps finding frames that were edited or maybe extracting information in such a way that a sequence of low resolution video clips might be massaged to provide high resolution images.
Each frame of a video clip is basically a pattern of bytes. I’d be surprised if somebody isn’t already investigating this with NuPIC.
1 Like