Error on getting anomaly score on giving input

Engineering HTM.Java
1

Hi All,

I am following the example as written in NetworkSetupExample.java (https://gist.github.com/cogmission/26700c261f3ad70c719b110b2e6d04a7) , I want to get anomaly score for the file rec-center-hourly.csv (found inside the resources of htm.java). In order to get anomaly scores i have added a method getExampleNetwork() in the PersistenceAPITest.java and adjust its headers as given below.

private Network getExampleNetwork() {
// This is the loop for inputting data into the network. This could be in another class, process, or thread 
though
Parameters p = NetworkTestHarness.getParameters().copy();
p = p.union(NetworkTestHarness.getDayDemoTestEncoderParams());
p.set(KEY.RANDOM, new FastRandom(42));
p.set(KEY.INFERRED_FIELDS, getInferredFieldsMap("consumption", SDRClassifier.class));

Sensor<ObservableSensor<String[]>> sensor = Sensor.create(
    ObservableSensor::create, SensorParams.create(Keys::obs, new Object[] {"name", 
        PublisherSupplier.builder()
            .addHeader("timestamp,consumption")        // The "headers" are the titles of your comma separated fields; (could be "timestamp,consumption,location" for 3 fields)
            .addHeader("datetime,float")           // The "Data Type" of the field (see FieldMetaTypes) (could be "datetime,float,geo" for 3 field types corresponding to above)
            .addHeader("T,B").build() }));   // Special flag. "B" means Blank (see Tests for other examples)

Network network = Network.create("test network", p).add(Network.createRegion("r1")
    .add(Network.createLayer("1", p)
        .alterParameter(KEY.AUTO_CLASSIFY, true)    // <--- Remove this line if doing anomalies and not predictions
        .add(Anomaly.create())                      // <--- Remove this line if doing predictions and not anomalies
        .add(new TemporalMemory())
        .add(new SpatialPooler())
        .add(sensor)));

return network;
}

And then i have added a test for using this network to get the anomaly score as follow,

@Test
public void setupNetwork() throws FileNotFoundException, IOException {
PrintWriter writer = new PrintWriter("/home/usamafurqan/anomalyscore.txt", "UTF-8");
// Call your network creation method
Network network = getExampleNetwork();

// Subscribes and receives Network Output
network.observe().subscribe(new Observer<Inference>() { 
    @Override public void onCompleted() { /* Any finishing touches after Network is finished */ }
    @Override public void onError(Throwable e) { /* Handle your errors here */ }
    @Override public void onNext(Inference inf) {
        /* This is the OUTPUT of the network after each input has been "reacted" to. */
    	writer.println("" + inf.getRecordNum() + ", " + inf.getAnomalyScore());
    }
});

Publisher pub = network.getPublisher();

/////////////////////////////////////////////////////////
//   Network must be "started" when using a Publisher  //
/////////////////////////////////////////////////////////
network.start();

// there should be only one thread pushing data to the network.
String file = "/home/usamafurqan/Desktop/rec-center-hourly.csv";
try (BufferedReader br = new BufferedReader(new FileReader(file))) {
	String line;
    while ((line = br.readLine()) != null) {
       // process the line.
        pub.onNext(line);
    }
}    
writer.close();
//while(data.hasNext()) {
  //  pub.onNext("your,comma-separated,data");

    // Sometimes you are entering more than one dataset or group of unrelated data, you will want
    // to reset inbetween so the HTM doesn't learn the transistion to the new group.
   // network.reset(); //<--- If all your data is related, remove this line
//}
}

For line by line input data stream i have copied the file in another directory and remove the header lines so that i should input only timestamp and float values.

But when i have started the tests the setupNetwork() test failed with the following error,

image.png1235×529 66 KB

May be it is simple but due to lack of domain knowledge of inside HTM java programs i am badly stuck,
Any idea how can i resolve this error?

Regards,
@Usama_Furqan

2

Hi @Usama_Furqan,

Welcome!

Can you put up a gist of the exact code you are running? The “NetworkTestHarness” has setup code for two demo networks. There is s “Day Demo” which basically cycles through numbers representing “days of the week”, and there is the “Hot Gym” network which takes in “timestamp and consumption” data. They aren’t interchangeable.

I see below that you are using the “Day Demo” network setup while loading the “Hot Gym” data from a file on your computer. Mixing these two won’t work. They have different “Encoder” setups. The setup code in the “NetworkTestHarness” shows how to set up the information necessary for encoding different types of data, and is meant to provide a guide as to how to set up the data structures.

We’re really happy you’re taking the time to use and investigate the code, it isn’t an easy thing to do - but you will need to spend some time going over the tests and running examples to understand what is going on - that and the wiki (see the navigation bar on the right)… together with the Quick Start Guide should help out a lot.

Find a running test, and play with it and break it; making changes to various things to learn how different parts impact the running of the code? Compare the differences between the tests - their data - and their encoder and network setups, to get an idea of the parts that are “common” between them?

Good Luck!

3

Hi @cogmission

Thank you , I got that point, I was using wrong method to get params of network. NetworkTestHarness class have method getHotGymTestEncoderParams() , it works correct for this data.

And you are right i should spend some time understanding tests and running examples to get an idea of different modules of HTM.java

Regards,
@Usama_Furqan