# Std::bad\_alloc runtime error in swarming

**URL:** <https://discourse.numenta.org/t/std-bad-alloc-runtime-error-in-swarming/2087>\
**Category:** NuPIC\
**Tags:** question\
**Created:** [April 4, 2017, 1:54am UTC](https://discourse.numenta.org/t/std-bad-alloc-runtime-error-in-swarming/2087 "2017-04-04T01:54:38Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![oreore](https://avatars.discourse-cdn.com/v4/letter/o/c57346/32.png) [@oreore](https://discourse.numenta.org/u/oreore)\
**Post date:** [April 4, 2017, 1:54am UTC](https://discourse.numenta.org/t/std-bad-alloc-runtime-error-in-swarming/2087/1 "2017-04-04T01:54:39Z")

</div>

Hi,

I encountered std::bad\_alloc runtime error when I tried to do swarming.  
The error occured after running tens of minutes.

My key configuration was like the followings:  
MaxWorker=4  
swarmSize=medium  
input file size ~= 80Mb (~180,000 records)  
number of Included fields = 4 (and one of them is the predicted field)

I’m working on Win7 64bit desktop with 16G ram.

Error does not occur when I set the swarmSize to small.

Is anybody who faced this problem before?  
What can be done for avoiding this error?

Thanks.

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [April 4, 2017, 3:56pm UTC](https://discourse.numenta.org/t/std-bad-alloc-runtime-error-in-swarming/2087/2 "2017-04-04T15:56:50Z")

</div>

> [@oreore](#):
>
> input file size ~= 80Mb (~180,000 records)

That’s a lot of data to run swarming over. There is some kind of memory problem. There is a way you can limit the amount of data processed by each model run during the swarm. Change `"iterationCount": -1` to `"iterationCount": 3000` to limit it to only 3000 rows of data per model. This might help.

---

<div class="post-metadata">

**Author:** ![oreore](https://avatars.discourse-cdn.com/v4/letter/o/c57346/32.png) [@oreore](https://discourse.numenta.org/u/oreore)\
**Post date:** [April 5, 2017, 12:48am UTC](https://discourse.numenta.org/t/std-bad-alloc-runtime-error-in-swarming/2087/3 "2017-04-05T00:48:33Z")

</div>

Thank you rhyolight!

Then, limiting it to 3000 rows should be on the assumption that the first 3000 rows have enough pattern to extract, right?  
Or, the 3000 rows are automatically sampled from anywhere of the entire dataset?

---

<div class="post-metadata">

**Author:** ![oreore](https://avatars.discourse-cdn.com/v4/letter/o/c57346/32.png) [@oreore](https://discourse.numenta.org/u/oreore)\
**Post date:** [April 5, 2017, 12:59am UTC](https://discourse.numenta.org/t/std-bad-alloc-runtime-error-in-swarming/2087/4 "2017-04-05T00:59:02Z")

</div>

OK, the same error is also observed during anomal detection of the same dataset (180000 record) with more than 10 fields at the same time.

I understood that the problem is definitely about the memory insufficiency.  
Then, do you think this problem can be solved with a system with very high memory capacity (currently 16G -\> increase it to maybe 128G)?

Or, could you please give us any general guidelines for using HTM without memory issue other than that you commented above?

- recommened spec of computing system (in terms of memory)
- number of columns that HTM can handle simultaneously

Thanks.

---

<div class="post-metadata">

**Author:** ![rhyolight](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.numenta.org/rhyolight/32/3922_2.png) [@rhyolight](https://discourse.numenta.org/u/rhyolight)\
**Post date:** [April 5, 2017, 3:45am UTC](https://discourse.numenta.org/t/std-bad-alloc-runtime-error-in-swarming/2087/5 "2017-04-05T03:45:09Z")

</div>

10 fields are a lot. The more fields you use, the large the input space for the spatial pooler. Can I see your swarm parameters? What are the column dimensions for your spatial pooler? And how many cells per column in the TM settings?

---

<div class="post-metadata">

**Author:** ![oreore](https://avatars.discourse-cdn.com/v4/letter/o/c57346/32.png) [@oreore](https://discourse.numenta.org/u/oreore)\
**Post date:** [April 5, 2017, 4:25am UTC](https://discourse.numenta.org/t/std-bad-alloc-runtime-error-in-swarming/2087/6 "2017-04-05T04:25:57Z")

</div>

I slightly modified the swarm\_description.py file and did not specified parameters like column dimensions and number of cells for now. (Attached below)  
Here I included only 7 fields but there are more actually.  
What I want to do is predicting the value of TargetF (Device Fault label) of (t + n) time and it is likely that temporal patterns in F1~F6 up to time t might be collaboratively influensive to the TargetF field value.  
But, at this moment, I don’t know which field is the most important one.  
That’s why I would like to put as many field as possible to prioritize their importance as a result of swarming.  
Do you think this approach is feasible?

Thanks.

```python
SWARM_DESCRIPTION = {
  "includedFields": [
    {
      "fieldName": 'Modified_Time',
      "fieldType": "datetime"
    },
     {
       "fieldName": 'F1',
       "fieldType": "float",
     },
    {
      "fieldName": 'F2',
      "fieldType": "string",
    },
    {
      "fieldName": 'F3',
      "fieldType": "float",
    },
    {
      "fieldName": 'F4',
      "fieldType": "string",
    },
     {
       "fieldName": 'F5',
       "fieldType": "float",
     },
     {
       "fieldName": 'F6',
       "fieldType": "float",
     },
     {
       "fieldName": 'TargetF',
       "fieldType": "string",
     }
  ],
  "streamDef": {
    "info": "sac_0A",
    "version": 1,
    "streams": [
      {
        "info": "SAC Stream",
        "source": "file://0A_test.txt",
        "columns": [
          "*"
        ]
      }
    ]
  },

  "inferenceType": "TemporalMultiStep",
  "inferenceArgs": {
    "predictionSteps": [
      1
    ],
    "predictedField": 'TargetF'
  },
  "iterationCount": -1,
  "swarmSize": "small"
}

```