Parameter Optimization and identification

Engineering HTM.Java
1

Hi All,

I am exploring HTM.java for solving the problems of anomaly detection and classification. I have read the theory of HTM and also done the anomaly detection example for getting raw anomaly score. I have seen in the example https://github.com/numenta/htm.java/wiki/NAPI-Quick-Start-Guide that we can make different layers and regions and connect them with each other in the HTM network.

I have a question that can we optimize our results and predictions with changing the count of layer, regions and other such configurations and parameters according to a particular domain or there exist a generic optimized configurations for all domain problems?

If we have to take these configurations according to our domain and problem type than any one have worked on auto identification and evolving of these configuration by any technique according to specific domain?

Regards,
@Usama_Furqan

1 Like
2

Hi @Usama_Furqan,

For the most part, you want to just use one Region and one Layer. The multi-Layer/Region capability is there mostly as a “place holder” for when hierarchy is added to NuPIC. For now, it really doesn’t buy you anymore capabilities to add multiple Layers or Regions.

3

Thank you @cogmission. I got your point about hierarchy (Region and layer).
Basically I am understanding and playing with different tests in the PersistenceAPITest. I have noticed that while creating network,we initialize it with the different parameters.
For exmaple in getLoadedHotGymNetwork_FileSensor

    Parameters p = NetworkTestHarness.getParameters().copy();
    p = p.union(NetworkTestHarness.getHotGymTestEncoderParams());
    p.set(KEY.RANDOM, new FastRandom(42));
    p.set(KEY.INFERRED_FIELDS, getInferredFieldsMap("consumption", CLAClassifier.class));

These parameters have different static values. As in the NetworkTestHarness.getParameters()

public static Parameters getParameters() {
    Parameters parameters = Parameters.getAllDefaultParameters();
    parameters.set(KEY.INPUT_DIMENSIONS, new int[] { 8 });
    parameters.set(KEY.COLUMN_DIMENSIONS, new int[] { 20 });
    parameters.set(KEY.CELLS_PER_COLUMN, 6);
    
    //SpatialPooler specific
    parameters.set(KEY.POTENTIAL_RADIUS, -1);//3
    parameters.set(KEY.POTENTIAL_PCT, 0.5);//0.5
    parameters.set(KEY.GLOBAL_INHIBITION, false);
    parameters.set(KEY.LOCAL_AREA_DENSITY, -1.0);
    parameters.set(KEY.NUM_ACTIVE_COLUMNS_PER_INH_AREA, 5.0);
    parameters.set(KEY.STIMULUS_THRESHOLD, 1.0);
    parameters.set(KEY.SYN_PERM_INACTIVE_DEC, 0.01);
    parameters.set(KEY.SYN_PERM_ACTIVE_INC, 0.1);
    parameters.set(KEY.SYN_PERM_TRIM_THRESHOLD, 0.05);
    parameters.set(KEY.SYN_PERM_CONNECTED, 0.1);
    parameters.set(KEY.MIN_PCT_OVERLAP_DUTY_CYCLES, 0.1);
    parameters.set(KEY.MIN_PCT_ACTIVE_DUTY_CYCLES, 0.1);
    parameters.set(KEY.DUTY_CYCLE_PERIOD, 10);
    parameters.set(KEY.MAX_BOOST, 10.0);
    parameters.set(KEY.SEED, 42);
    
    //Temporal Memory specific
    parameters.set(KEY.INITIAL_PERMANENCE, 0.2);
    parameters.set(KEY.CONNECTED_PERMANENCE, 0.8);
    parameters.set(KEY.MIN_THRESHOLD, 5);
    parameters.set(KEY.MAX_NEW_SYNAPSE_COUNT, 6);
    parameters.set(KEY.PERMANENCE_INCREMENT, 0.05);
    parameters.set(KEY.PERMANENCE_DECREMENT, 0.05);
    parameters.set(KEY.ACTIVATION_THRESHOLD, 4);
    
    return parameters;
}

All parameters have their own significance and they could be set with different values during declaring the network.

I have studied the significance of few parameters stated in the documents “BAMI SP algorithm” and “BAMI SP algorithm” v0.5 as given in the topic
HTM Parameters for Dummies . For example we could have different values of columnDimensions, inputDimensions, potentialRadius and globalInhibition (which could be true or false). Different values of parameters used in different scenarios and they can impact the output of network, like we would have multiple cells per column when we want to recognize time-based sequences and we would have single cells per column when we want first order prediction (static spatial inference).

I have a question that is there any approach exists or work done to identify (predict) values of different parameters with respect to domain problem?
Like genetic algorithms are used for such purpose !
Or we have generic values of parameters that could be applicable on any problem and return optimized results?

Regards,
@Usama_Furqan

4

That would be an excellent project for the community. I proposed something like that for a Hackathon project two years ago which was an “interactive parameter sanity checker”. As certain values were modified other values would increase or decrease to adjust and you could see the relationship of the parameters to each other while trying to keep them all sane. Some research would have to be done in order to perfect the Inter-relationships between the parameters - but Subutai said he might get be able to help… however two years later we now have more of a brain trust in the community and there are probably more people qualified to help - as this is more of a critical time, we probably want Subutai concentrating on research :wink:

5

Hi @cogmission
I have discussed it with my supervisor @wmqazi in the university and we are interested to work on this project. Is it possible that you can share me few details of your Hackathon project “interactive parameter sanity checker” just for learning and understanding your approach. May be it would help me a lot.
Also is it possible that we can do this “Evolving HTM parameters” project with collaboration? That it benefits us both !

Regards,
@Usama_Furqan

1 Like
6

@Usama_Furqan,

Unfortunately, I submitted the project to the community as an idea, but never intended to work on it myself. So there aren’t any requirements or new development on the idea - it was just that, an idea. I think its great that your supervisor is interested in the project - but unfortunately I don’t have bandwidth to begin a new project right now as I’m swamped with a new Cortical.io demo we’re getting ready to announce soon.

The idea for the parameter checker will require domain knowledge which you will have to take some time to acquire by experimenting with them over time, along with engaging peeps here by writing questions to the community. I don’t really have any “magic” insight into this - I just thought it would be great if it existed and would definitely fulfill a niche in a needed area that can be difficult to manage. Please don’t expect continuous engagement on this from me as I really don’t have the time, I’m sorry to say. I hope however that this doesn’t dissuade you from investigating this as a possible project, but I understand if you were hoping for more experienced partnership. Hopefully you can entice other community members to assist? We’ll see?

Cheers,
David