I have uploaded a new project on GitHub called image2sdr. It is a simple NodeJS service which you can post an image file to, and it returns an SDR.
Behind the scenes what it is doing is:
- Classify image via Clarifai and return the top 10 concepts and their confidence level
- Parse the concepts found and retrieve word SDRs via cortical.io
- Stack the word SDRs, scored by their confidence level
- Generate a standard bit-array SDR with the desired sparsity, containing the bits with the top scores (using random tie breaker)
This project is in response to a conversation we had during the most recent Hackers’ Hangout.
Couple of notes:
- You will need to copy config_example.yml into a file named config.yml, and enter your own Cortical.io and Clarifai API keys. You can get free API keys from the respective websites.
- The service does not currently support proxy settings. If running behind a proxy, you’ll need to add the necessary code
- The word SDRs from cortical.io are not a fixed sparsity (common words are more dense than less common ones), so might need some tweaking of the sparsity and sdr_size parameters (or refactoring if there is a better way)
- Multi-word concepts from Clarifai are skipped (such as “hard hat”). To enable them, special consideration will need to be given to negative concepts (such as “no people”).
- This is an open source HTM community project, so feel free to make updates and evolve the idea