Problem running newly saved / loaded SP & TM

Engineering NuPIC
41

I got only value -zero.Because I used different classifiers My working code as saving :

with open("out_sp.tmp", "wb") as f1:
    sp.writeToFile(f1)
with open("out_tm.tmp", "wb") as f2:
    tm.writeToFile(f2)
with open("out_classifier.tmp", "wb") as f3:
    classifier.writeToFile(f3)
with open("out_classifier1.tmp", "wb") as f4:
    classifier1.writeToFile(f4)
with open("out_classifier2.tmp", "wb") as f5:
    classifier2.writeToFile(f5)

And then my loading :

with open(“out_sp.tmp”, “rb”) as f1:
sp2 = SpatialPooler.readFromFile(f1)
with open(“out_tm.tmp”, “rb”) as f1:
tm2 = TemporalMemory.readFromFile(f1)
with open(“out_classifier.tmp”, “rb”) as f2:
classifier4 = SDRClassifier.readFromFile(f2)
with open(“out_classifier1.tmp”, “rb”) as f3:
classifier5= SDRClassifier.readFromFile(f3)
with open(“out_classifier2.tmp”, “rb”) as f4:
classifier6= SDRClassifier.readFromFile(f4)

42

Not the end of the history:) Should I serialize SDRClassifier? Because I got the different result if I serialized classifier or didn’t serialize?

Thanks a lot !

43

I’m using the OPF for model saving & loading within a loop. Not sure if this helps at all but here’s the function I have for it:

https://pastebin.com/K8UpPepS

1 Like
44

Thanx a lot. Unfortunately, I can’t use OPF models, because I use 3 classifiers on one model.
These is my code :

if (count%100) == 0:
with open(“out_sp.tmp”, “wba”) as f1:
sp.writeToFile(f1)
with open(“out_tm.tmp”, “wba”) as f2:
tm.writeToFile(f2)
with open(“out_classifier.tmp”, “a”) as f3:
classifier.writeToFile(f3)
with open(“out_classifier1.tmp”, “a”) as f4:
classifier1.writeToFile(f4)
with open(“out_classifier2.tmp”, “a”) as f5:
classifier2.writeToFile(f5)

What is your mind about this code? I have a different result if have common classifier or if I serialize classifier. I don’t understand why.

45

@scott should @sergey serialize the classifier?

46

Maybe some problems about file size?

47

I’m still confused about your problem. Maybe I can help better if you post your complete program code somewhere so I can run it and see the issues myself.

48

good idea. Few minutes .

49

This is my code ,when I saved model:
https://pastebin.com/75fCiLiX
This is my code when i loaded learning model:
https://pastebin.com/RkGYXBYc

When I didn’t serialize classifier , I had not prediction in general. If I used these two functions in one script , I had normal prediction with common classifier. If I serialized classifier I had bad prediction .

Thanks for your help !

50

Thanks for posting. I’m actually on vacation right now and will look at this when I get back into the office (sorry!).

51

Sorry for interrupt your vacation. Maybe different members on forum can look code.

Have a good vacation !

Thank you very much

1 Like
52

If serializing and deserializing the classifier results in different predictions then there is probably a bug.

53

In my code or in nupic. Did you look at my code?Maybe it’s my fault.

54

@Sergey I looked at your code and it looks like you need to serialize the last 100 records from the stream.
In the learning stage code you are saving a checkpoint every 100 records (lines 556-566) but you are missing the last 100 records. You are also exiting the loop without serializing the results on line 448.
You need to add the serialization code to outside the loop making sure the last 100 records are saved. See patch below:

--- learning_stage.py	2018-07-24 07:46:20.000000000 -0700
+++ learning_stage_new.py	2018-07-24 07:52:20.000000000 -0700
@@ -564,6 +564,18 @@
             classifier1.writeToFile(f4)
         with open("out_classifier2.tmp", "w") as f5:
             classifier2.writeToFile(f5)
+
+    # Save last batch
+    with open("out_sp.tmp", "w") as f1:
+        sp.writeToFile(f1)
+    with open("out_tm.tmp", "w") as f2:
+        tm.writeToFile(f2)
+    with open("out_classifier.tmp", "w") as f3:
+        classifier.writeToFile(f3)
+    with open("out_classifier1.tmp", "w") as f4:
+        classifier1.writeToFile(f4)
+    with open("out_classifier2.tmp", "w") as f5:
+        classifier2.writeToFile(f5)
   # builder = SpatialPoolerProto.new_message()
   # sp.write(builder)
   # serializedMessage = builder.to_bytes_packed()

Let me know if it works for you

2 Likes
55

Thanx for your help . I change my code :
https://pastebin.com/VYTD4saq

But I have very bad predict. And saving worked about 5 hours :slight_smile:

Maybe you have any ideas about my code ?