Reload stored network using persistence API

Engineering HTM.Java
#1

I created a network with a file sensor to read data from a input csv file

        this.temporalMemory = new TemporalMemory();
        this.spatialPooler = new SpatialPooler();
        this.anomaly = Anomaly.create();


        this.sensor = Sensor.create(FileSensor::create, SensorParams.create(
                Keys::path, "", ResourceLocator.path(fileName+ ".csv")));

Parameters p = OnlineNetworkHarness.getParameters();
p = p.union(OnlineNetworkHarness.getEncoderParams(inferenceField));
return Network.create(name+ “-ONLINE”, p)
.add(Network.createRegion(“Region 1”)
.add(Network.createLayer(“Layer 2/3”, p)
.alterParameter(Parameters.KEY.AUTO_CLASSIFY, Boolean.TRUE)
.add(this.anomaly)
.add(this.temporalMemory)
.add(this.spatialPooler)
.add(this.sensor)));

When the file stream completes, I save the network using the persistence API like this:

return new Subscriber() {
@Override
public void onCompleted() {
try {
SerialConfig config = new SerialConfig(network.getName());
Persistence.get(config).store(network);
} catch (Exception e) {
e.printStackTrace();
}
}

        @Override
        public void onError(Throwable e) {
            e.printStackTrace();
        }

        @Override
        public void onNext(Inference i) {
        }
    };

After that, I want to start an online network using an observable sensor that consumes a webservice periodically.

In order to accomplish that, I create an Observable Sensor:

PublisherSupplier supplier = PublisherSupplier.builder()
.addHeader(“timestamp,” + campoInferencia)
// .addHeader(“timestamp,consumption”)
.addHeader(“datetime,float”)
// .addHeader(“T,B”) //see SensorFlags.java for more info
.addHeader(“T,”) //see SensorFlags.java for more info
.build();
this.sensor = Sensor.create(ObservableSensor::create, SensorParams.create(
Keys::obs, new Object[]{"", supplier}));

Then I reload the network:

SerialConfig config = new SerialConfig(name);
        return Persistence.get(config).load(name);

Register the Publisher:

try{
this.publisher = this.network.getPublisher();
}
catch(Exception e){
supplier.setNetwork(this.network);
this.publisher = supplier.get();
}

Finally, I subscribe:

//Create the ObservableSensor; note the use of the PublisherSupplier instead of the Publisher
this.network.observe().subscribe(subscriber);

Problem is that the sensor stream automaticaly completes because when the network was stored, it has been halted, I think.

I’ve already tryed:

networ.getHead().getHead().add(sensor);

It throws an exception because the layer is already closed.

And I also tryed :slight_smile:

network.setSensor(sensor) ;

But it has no effect in the layer’s sensor.

So I’d like to know how do I save a network that uses a file sensor, reload it and then change its sensor to an observable sensor that doesn’t complete its stream before I terminate it manually.

1 Like
#2

Hi @Matheus_Araujo,

Thanks for using HTM.java!

You’ll have to give me 24 hours to take a look at it because it’s been a while now since I’ve looked at that code (So I can’t tell you right off the top of my head), and today is my wife’s birthday…

Talk to you soon,
David

2 Likes
#3

Cogmission, any news about that?

I’ve worked around and used an obervablesensor that reads a csv file and manualy calls onNext() instead of a filesensor and it worked well.

2 Likes
#4

@Matheus_Araujo,

Hey I’m sorry I can’t get right back to you. But you are on the right track. FileSensors cannot be persisted (for obvious reasons - 1 being that the nature of files is that their presence on a local filesystem is not transportable). The ObservableSensor exists to give you more flexibility when needed. There are lot’s of examples in the integration tests of how to save, stop and revive a network.

I applaud you for finding your own solution! That’s very cool!

Cheers,
David

P.S. Also hint, always use a PublisherSupplier rather than a Publisher because those are needed for persistence. The PublisherSupplier abstracts away the delivery of a publisher so that the source of it can vary dynamically (needed for the reconstitution of a network).

1 Like
#5

Thank you!