# KjException when serializing

**URL:** <https://discourse.numenta.org/t/kjexception-when-serializing/3901>\
**Category:** NuPIC\
**Created:** [May 13, 2018, 4:18am UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901 "2018-05-13T04:18:08Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sergey](https://avatars.discourse-cdn.com/v4/letter/s/9fc348/32.png) [@Sergey](https://discourse.numenta.org/u/Sergey)\
**Post date:** [May 13, 2018, 4:18am UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/1 "2018-05-13T04:18:08Z")

</div>

I’m trying to use the writeToFie function from “New Serialization”, which I found here: [http://nupic.docs.numenta.org/1.0.0/guides/serialization.html](http://nupic.docs.numenta.org/1.0.0/guides/serialization.html), but I got the next error:  
apnp.lib.capnp.KjException: src/capnp/arena.c++:106: failed: Exceeded message traversal limit. See capnp::ReaderOptions.  
stack: 0x7f666095c6f9 0x7f666096032d 0x7f66608c1d1c 0x7f666093d374 0x7f6660904c5f 0x7f666092cd57 0x7f666092d7af 0x7f666081af5c 0x7f666081b40e 0x7f66607b8746 0x7f66607c32b0 0x7f66607c19f2 0x7f66607c160f 0x7f66607c5290 0x4c45fa 0x4c9d7f

I would appreciate any help.  
Thanks in advance.

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [May 13, 2018, 4:52am UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/2 "2018-05-13T04:52:12Z")

</div>

@Sergey Please update to the latest nupic and try again. You want nupic 1.0.3 / nupic.bindings 1.0.0.

---

<div class="post-metadata">

**Author:** ![Sergey](https://avatars.discourse-cdn.com/v4/letter/s/9fc348/32.png) [@Sergey](https://discourse.numenta.org/u/Sergey)\
**Post date:** [May 14, 2018, 10:19am UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/3 "2018-05-14T10:19:53Z")

</div>

Thanks for your help. I update nupic ,but I got same error. Maybe you know another approach ?

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [May 14, 2018, 4:35pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/4 "2018-05-14T16:35:13Z")

</div>

Can you please share the code that is causing this problem? What object are you writing and how are you writing it?

Thanks

---

<div class="post-metadata">

**Author:** ![Sergey](https://avatars.discourse-cdn.com/v4/letter/s/9fc348/32.png) [@Sergey](https://discourse.numenta.org/u/Sergey)\
**Post date:** [May 14, 2018, 6:53pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/5 "2018-05-14T18:53:52Z")

</div>

Of course. This code causing problem (the second line):

> ```
> with open("out_sp.tmp", "wb") as f1:
> sp.writeToFile(f1)
> 
> ```

I want to write Spatial Pooler and Temporal Memory, but now we discuss about SP .

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [May 14, 2018, 6:58pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/6 "2018-05-14T18:58:16Z")

</div>

> [@Sergey](#):
>
> Exceeded message traversal limit.

@lscheinkman @scott Does this sound familiar? This SP write should be pretty cut-and-dry.

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [May 14, 2018, 7:32pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/7 "2018-05-14T19:32:20Z")

</div>

I just realized it is probably because it’s an OPF model?

> [@Model writeToFile Error : Exceeded message traversal limit. See capnp::ReaderOptions](https://discourse.numenta.org/t/model-writetofile-error-exceeded-message-traversal-limit-see-capnp-readeroptions/2801):
>
> Hi I am getting “Exceeded message traversal limit. See capnp::ReaderOptions.” while writing the model , Has anyone faced similar issue ? Here is the error log Traceback (most recent call last): model.writeToFile(f,True) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nupic/serializable.py", line 116, in writeToFile self.write(proto) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nupic/frameworks/opf/htm\_pred…

@Sergey exactly what object is `sp` in your code? How do you create it?

---

<div class="post-metadata">

**Author:** ![scott](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/scott/32/170_2.png) [@scott](https://discourse.numenta.org/u/scott)\
**Post date:** [May 14, 2018, 7:58pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/8 "2018-05-14T19:58:28Z")

</div>

Could be related to this?

> <https://github.com/capnproto/pycapnp/issues/116>

---

<div class="post-metadata">

**Author:** ![Sergey](https://avatars.discourse-cdn.com/v4/letter/s/9fc348/32.png) [@Sergey](https://discourse.numenta.org/u/Sergey)\
**Post date:** [May 14, 2018, 9:06pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/9 "2018-05-14T21:06:32Z")

</div>

This code how i create sp :

```python
with open(PARAMS_PATH, "r") as f:
    modelParams = yaml.safe_load(f)["modelParams"]
    # enParams = modelParams["sensorParams"]["encoders"]
    spParams = modelParams["spParams"]
    tmParams = modelParams["tmParams"]

```

```python
sp = SpatialPooler(
    inputDimensions=(encodingWidth,),
    columnDimensions=(spParams["columnCount"],),
    potentialPct=spParams["potentialPct"],
    potentialRadius=encodingWidth,
    globalInhibition=spParams["globalInhibition"],
    localAreaDensity=spParams["localAreaDensity"],
    numActiveColumnsPerInhArea=spParams["numActiveColumnsPerInhArea"],
    synPermInactiveDec=spParams["synPermInactiveDec"],
    synPermActiveInc=spParams["synPermActiveInc"],
    synPermConnected=spParams["synPermConnected"],
    boostStrength=spParams["boostStrength"],
    seed=spParams["seed"],
    wrapAround=True
)

```

---

<div class="post-metadata">

**Author:** ![Sergey](https://avatars.discourse-cdn.com/v4/letter/s/9fc348/32.png) [@Sergey](https://discourse.numenta.org/u/Sergey)\
**Post date:** [May 14, 2018, 10:37pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/10 "2018-05-14T22:37:10Z")

</div>

I don’t know exactly. I didn’t use any class about capnp.

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [May 14, 2018, 10:43pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/11 "2018-05-14T22:43:15Z")

</div>

Have you passed any data into your SP before trying to serialize it?

---

<div class="post-metadata">

**Author:** ![Sergey](https://avatars.discourse-cdn.com/v4/letter/s/9fc348/32.png) [@Sergey](https://discourse.numenta.org/u/Sergey)\
**Post date:** [May 14, 2018, 11:31pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/12 "2018-05-14T23:31:11Z")

</div>

Yes, I have a function, where I did sp.compute and pass to TM an then classifier. After these steps in this function I try to serialize SP.

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [May 14, 2018, 11:35pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/13 "2018-05-14T23:35:57Z")

</div>

Ok, does this happen if you _don’t_ send data into it and try to serialize?

---

<div class="post-metadata">

**Author:** ![Sergey](https://avatars.discourse-cdn.com/v4/letter/s/9fc348/32.png) [@Sergey](https://discourse.numenta.org/u/Sergey)\
**Post date:** [May 14, 2018, 11:50pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/14 "2018-05-14T23:50:54Z")

</div>

I try now. I put this code at the begin of my function. I got same error.

> ```
> with open("out_sp.tmp", "wb") as f1:
> sp.writeToFile(f1)
> 
> ```

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [May 14, 2018, 11:56pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/15 "2018-05-14T23:56:07Z")

</div>

Are you on Windows?

---

<div class="post-metadata">

**Author:** ![Sergey](https://avatars.discourse-cdn.com/v4/letter/s/9fc348/32.png) [@Sergey](https://discourse.numenta.org/u/Sergey)\
**Post date:** [May 14, 2018, 11:56pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/16 "2018-05-14T23:56:50Z")

</div>

Linux

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [May 15, 2018, 3:58pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/17 "2018-05-15T15:58:58Z")

</div>

Does this script fail with the same error?

```python
from nupic.algorithms.spatial_pooler import SpatialPooler

def testWriteSp():
  sp = SpatialPooler(
    inputDimensions=(400,),
    columnDimensions=(1024,),
    wrapAround=True
  )
  with open("out_sp.tmp", "wb") as f1:
    sp.writeToFile(f1)

if __name__ == " __main__":
  testWriteSp()

```

(it works for me)

---

<div class="post-metadata">

**Author:** ![Sergey](https://avatars.discourse-cdn.com/v4/letter/s/9fc348/32.png) [@Sergey](https://discourse.numenta.org/u/Sergey)\
**Post date:** [May 15, 2018, 5:40pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/18 "2018-05-15T17:40:39Z")

</div>

I don’t find different things in my function . This is the same approach. Maybe you find differents in these functions :

```python
def runLearning(numRecords):
  
  learning_time = time()
  with open("test3.csv", "r") as fin:
    reader = csv.reader(fin)
    headers = reader.next()
    reader.next()
    reader.next()

    for count, record in enumerate(reader):

      if count >= numRecords: break

      # Convert data string into Python date object.
      #dateString = datetime.datetime.strptime(record[0], "%m/%d/%y %H:%M")
      # Convert data value string into float.
      event_value = float(record[6]) # device 5
      event_value_3 = float(record[4]) # device 3
      event_value_2 = float(record[3]) #device 2
      event_value_7 = float(record[8]) # device 7
      bezline_all = float(record[10])

      flow_value = float(record[0])
      # To encode, we need to provide zero-filled numpy arrays for the encoders
      # to populate.
      eventBits = numpy.zeros(eventEncoder.getWidth())
      eventBits_2 = numpy.zeros(eventEncoder2.getWidth())
      eventBits_3 = numpy.zeros(eventEncoder1.getWidth())
      eventBits_7 = numpy.zeros(eventEncoder7.getWidth())

      baseline_Bits = numpy.zeros(baselineEncoder.getWidth())
      flowBits = numpy.zeros(flowEncoder.getWidth())

      # Now we call the encoders to create bit representations for each value.
      eventEncoder.encodeIntoArray(event_value, eventBits)
      eventEncoder1.encodeIntoArray(event_value_3,eventBits_3)
      eventEncoder2.encodeIntoArray(event_value_2,eventBits_2)
      eventEncoder7.encodeIntoArray(event_value_7,eventBits_7)

      baselineEncoder.encodeIntoArray(bezline_all,baseline_Bits)
      flowEncoder.encodeIntoArray(flow_value, flowBits)

      # Concatenate all these encodings into one large encoding for Spatial
      # Pooling.
      encoding = numpy.concatenate(
        [eventBits,flowBits,baseline_Bits,eventBits_2,flowBits,baseline_Bits,eventBits_3,flowBits,baseline_Bits,eventBits_7,flowBits,baseline_Bits]
      )

      # Create an array to represent active columns, all initially zero. This
      # will be populated by the compute method below. It must have the same
      # dimensions as the Spatial Pooler.
      activeColumns = numpy.zeros(spParams["columnCount"])
      # activeColumns1 = numpy.zeros(spParams["columnCount"])

      # Execute Spatial Pooling algorithm over input space.

      sp.compute(encoding,True,activeColumns)

     # sp.compute(encoding1, True, activeColumns)

      activeColumnIndices = numpy.nonzero(activeColumns)[0]

      # Execute Temporal Memory algorithm over active mini-columns.
      tm.compute(activeColumnIndices, learn=True)

      activeCells = tm.getActiveCells()

      # Get the bucket info for this input value for classification.
      bucketIdx = eventEncoder.getBucketIndices(event_value)[0]
      bucketIdx_2 = eventEncoder2.getBucketIndices(event_value_2)[0]
      bucketIdx_3 = eventEncoder1.getBucketIndices(event_value_3)[0]
      bucketIdx_7 = eventEncoder7.getBucketIndices(event_value_7)[0]
      print "BucketIdx_3:",bucketIdx_3
      print "BucketIdx:",bucketIdx

      # Run classifier to translate active cells back to scalar value.
      classifierResult = classifier.compute(
        recordNum=count,
        patternNZ=activeCells,
        classification={
          "bucketIdx": bucketIdx,
          "actValue": event_value
        },
        learn=True,
        infer=False
      )
      classifierResult1 = classifier1.compute(
        recordNum=count,
        patternNZ=activeCells,
        classification={
          "bucketIdx": bucketIdx_3,
          "actValue": event_value_3
        },
        learn=True,
        infer=False
      )
      classifierResult7 = classifier7.compute(
        recordNum=count,
        patternNZ=activeCells,
        classification={
          "bucketIdx": bucketIdx_7,
          "actValue": event_value_7
        },
        learn=True,
        infer=False
      )
      classifierResult2 = classifier2.compute(
        recordNum=count,
        patternNZ=activeCells,
        classification={
          "bucketIdx": bucketIdx_2,
          "actValue": event_value_2
        },
        learn=True,
        infer=False
      )
    learning_time_end = time()
    print "Time",(learning_time - learning_time_end)
    with open("out_sp.tmp", "wb") as f1:
        sp.writeToFile(f1)
    with open("out_tm.tmp", "wb") as f:
        tm.writeToFile(f)

    return result

```

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [May 15, 2018, 5:59pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/19 "2018-05-15T17:59:41Z")

</div>

That’s not all your code. I don’t see where the `sp` is initiated.

But did the script I posted above work for you? Can you save it into a file and run it with python? If it fails for you, I can rule out data and model params as a source of the problem.

---

<div class="post-metadata">

**Author:** ![Sergey](https://avatars.discourse-cdn.com/v4/letter/s/9fc348/32.png) [@Sergey](https://discourse.numenta.org/u/Sergey)\
**Post date:** [May 15, 2018, 9:10pm UTC](https://discourse.numenta.org/t/kjexception-when-serializing/3901/20 "2018-05-15T21:10:19Z")

</div>

Your script works for me .

My initiation :

```python
with open(PARAMS_PATH, "r") as f:
    modelParams = yaml.safe_load(f)["modelParams"]
    # enParams = modelParams["sensorParams"]["encoders"]
    spParams = modelParams["spParams"]

sp = SpatialPooler(
    inputDimensions=(encodingWidth,),
    columnDimensions=(spParams["columnCount"],),
    potentialPct=spParams["potentialPct"],
    potentialRadius=encodingWidth,
    globalInhibition=spParams["globalInhibition"],
    localAreaDensity=spParams["localAreaDensity"],
    numActiveColumnsPerInhArea=spParams["numActiveColumnsPerInhArea"],
    synPermInactiveDec=spParams["synPermInactiveDec"],
    synPermActiveInc=spParams["synPermActiveInc"],
    synPermConnected=spParams["synPermConnected"],
    boostStrength=spParams["boostStrength"],
    seed=spParams["seed"],
    wrapAround=True
)

```

[Next page](https://discourse.numenta.org/t/kjexception-when-serializing/3901.md?page=2)