Image processing by HTM

Engineering
1

hello Matt, I’m planning to use nupic for some real world image applications , for example to detect tumor brain images , I have 6000 MRI brain tumor (their size is 256256180) and 4000 MRI control subject(their size is 256256180). first I use CNN for feature extraction of images for example by VGG16 I create feature vector of size 4096 and then give the features to HTm for anomaly detection and classify tumor and control brain. my question is,this idea is practical or not? can HTM predict on such real problem?
(in this problem i suggest the brain slices of each subject is one time series)

3 Likes
2

For standard spatial classification tasks like this, HTMs will not beat CNNs with today’s libraries. I suggest you use CNNs without HTMs for this.

3

What if the idea is not to beat CNNs, but to develop a new application for HTM?

@rhyolight, do you think HTM is capable of addressing this kind of problem? If so, I’m sure we can all learn some from this.

@shiva, if you have access to all these MRI samples, does that mean you have experience with this? Have you done work like that before?

4

I think that HTM will become SOTA in image classification when it can incorporate space and movement into the model via sensor movement. This can be done virtually (see 2D Object Recognition Project) by having an agent with sensors explore a 2D environment (image). After training I bet you could have a system that can make a classification after only a few virtual movements. That is the direction I’m hoping to take this project anyway.

2 Likes
5

I’m currently also researching into vision processing with HTM.
There’s a working image classification demo on MNIST in the community repo,

See discussion why current topology/inhibition does not cut it

We have WIP (new SP with MacroColumns, or ColumnPooler-based) solutions that achieve >90%.
That still does not beat the SotA (~99%), but HTM has other interesting properties in this area: it learns darn fast, as compared to DNNs that suffer for big-data need.

Ping me with the MRI classification, I have similar project on retina scans OCT.

2 Likes