SDRClassifier serialization

Engineering HTM.Java
1

Hello,

We are using the HTM to detect anomalies in our Spark Streaming process.

After running our project the following exception arises during the deserialization:
java.io.InvalidClassException: no.uib.cipr.matrix.sparse.FlexCompRowMatrix; no valid constructor

It seems that the SDRClassifier uses FlexCompRowMatrix class and the FlexCompRowMatrix extends AbstractMatrix class which neither implements Serializable interface nor has a constructor with no-arg arguments.

Is it possible to do something to avoid the InvalidClassException in our case?

The used version is org.numenta/htm.java 0.6.13.

1 Like
2

This might be a bug, in which case you can report here.

3

The created issue is numenta/htm.java #537

4

@alexsch,

Hi, as also noted on the issue, I’m giving this a look. There shouldn’t be any problems with serialization/deserialization as there is a test for this I’m pretty sure. Andrew and I were well aware of the need for his new Classifier to fit into the newer serialization framework - so I’m very surprised that you have this error…

Let’s see what we can find out…

5

Hi @alexsch,

The problem is that the FlexCompRowMatrix class we use comes from a “culled” and highly optimized version which I forked from AlgorithmFoundry - hence the name, “algorithmfoundry-shade-culled-1.x.jar” for the library name (which is included in the “libs” directory of the project). It adds the ability to delete rows and columns from the matrix class. I wanted to expand its ability to work with sparse matrices, since this library offers many advanced features needed for other things like the KNN Classifier development work.

We made this available via Gradle or Maven via the repository on my server using the following maven url:
http://metaware.us/maven3

…as seen in the Gradle “build.gradle” file:

The only reason this might not work for you is that you are specifying another algorithmfoundry repository and picking up the wrong version of the class?? You have to make sure you are using our version…

Let me know if this helps?

6

We do not use additional repositories.

The dependency on the numenta library in our pom file is:

<dependency>
    <groupId>org.numenta</groupId>
    <artifactId>htm.java</artifactId>
    <version>0.6.13</version>
</dependency>

The dependecy:tree shows the algorithmfoundry library only in one place:

[INFO] +- org.numenta:htm.java:jar:0.6.13:compile
[INFO] |  +- joda-time:joda-time:jar:2.5:compile
[INFO] |  +- com.chaschev:chutils:jar:1.4:compile
[INFO] |  +- net.sf.trove4j:trove4j:jar:3.0.3:compile
[INFO] |  +- io.reactivex:rxjava:jar:1.0.10:compile
[INFO] |  +- com.cedarsoftware:java-util:jar:1.19.3:compile
[INFO] |  |  +- org.apache.logging.log4j:log4j-api:jar:2.1:compile
[INFO] |  |  \- org.apache.logging.log4j:log4j-core:jar:2.1:compile
[INFO] |  +- de.ruedigermoeller:fst:jar:2.45:compile
[INFO] |  |  +- org.javassist:javassist:jar:3.19.0-GA:compile
[INFO] |  |  \- org.objenesis:objenesis:jar:2.1:compile
[INFO] |  +- org.openjdk.jmh:jmh-core:jar:1.11.3:compile
[INFO] |  \- algorithmfoundry:algorithmfoundry-shade-culled:jar:1.3:compile

It is the algorithmfoundry:algorithmfoundry-shade-culled:jar:1.3:compile