Hello Andrew,
I tried to run you code, however I was running into an Python3 error message, which seems pretty cryptic to me. Do you have any idea how to fix this?
The first time I run your script, I got an error from ray
, that it was missing the ray.init
.
So I uncommented the line:
ray.init(ignore_reinit_error=True)
After that modification I run into this long error message.
2020-07-31 14:18:05,644 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
2020-07-31 14:18:05,645 INFO node.py:498 -- Process STDOUT and STDERR is being redirected to /tmp/ray/session_2020-07-31_23-18-05_645093_8863/logs.
2020-07-31 14:18:05,749 INFO services.py:409 -- Waiting for redis server at 127.0.0.1:27007 to respond...
2020-07-31 14:18:05,860 INFO services.py:409 -- Waiting for redis server at 127.0.0.1:49171 to respond...
2020-07-31 14:18:05,862 INFO services.py:806 -- Starting Redis shard with 1.63 GB max memory.
2020-07-31 14:18:05,877 INFO node.py:512 -- Process STDOUT and STDERR is being redirected to /tmp/ray/session_2020-07-31_23-18-05_645093_8863/logs.
2020-07-31 14:18:05,878 INFO services.py:1442 -- Starting the Plasma object store with 2.44 GB memory using /dev/shm.
Processed SP input 100 out of 4390...
That took 1.8991518020629883 seconds.
2020-07-31 14:18:10,594 ERROR worker.py:1668 -- WARNING: 12 workers have been started. This could be a result of using a large number of actors, or it could be a consequence of using nested tasks (see https://github.com/ray-project/ray/issues/3644) for some a discussion of workarounds.
(pid=8889) 2020-07-31 14:18:11,204 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8889) 2020-07-31 14:18:11,204 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.
(pid=8888) 2020-07-31 14:18:11,329 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8888) 2020-07-31 14:18:11,344 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.
(pid=8890) 2020-07-31 14:18:11,385 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8890) 2020-07-31 14:18:11,385 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.
(pid=8887) 2020-07-31 14:18:11,522 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8887) 2020-07-31 14:18:11,523 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.
2020-07-31 14:18:12,139 ERROR worker.py:1668 -- WARNING: 16 workers have been started. This could be a result of using a large number of actors, or it could be a consequence of using nested tasks (see https://github.com/ray-project/ray/issues/3644) for some a discussion of workarounds.
(pid=8937) 2020-07-31 14:18:12,391 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8937) 2020-07-31 14:18:12,394 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.
(pid=8940) 2020-07-31 14:18:12,847 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8940) 2020-07-31 14:18:12,852 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.
(pid=8939) 2020-07-31 14:18:12,904 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8939) 2020-07-31 14:18:12,904 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.
(pid=8938) 2020-07-31 14:18:13,042 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8938) 2020-07-31 14:18:13,046 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.
2020-07-31 14:18:13,877 ERROR worker.py:1668 -- WARNING: 20 workers have been started. This could be a result of using a large number of actors, or it could be a consequence of using nested tasks (see https://github.com/ray-project/ray/issues/3644) for some a discussion of workarounds.
(pid=8964) 2020-07-31 14:18:14,071 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8964) 2020-07-31 14:18:14,071 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.
(pid=8961) 2020-07-31 14:18:14,291 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8961) 2020-07-31 14:18:14,292 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.
(pid=8958) 2020-07-31 14:18:14,357 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8958) 2020-07-31 14:18:14,364 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.
(pid=8967) 2020-07-31 14:18:14,804 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8967) 2020-07-31 14:18:14,805 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.
(pid=8986) 2020-07-31 14:18:15,554 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8986) 2020-07-31 14:18:15,554 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.
2020-07-31 14:18:15,679 ERROR worker.py:1668 -- WARNING: 24 workers have been started. This could be a result of using a large number of actors, or it could be a consequence of using nested tasks (see https://github.com/ray-project/ray/issues/3644) for some a discussion of workarounds.
Traceback (most recent call last):
File "HotGym Example Scripts__Multiprocessing.py", line 155, in <module>
tm, at = HotGym_TM_Example(sp,enc,dates,power)
File "HotGym Example Scripts__Multiprocessing.py", line 113, in HotGym_TM_Example
act, pred = tm.process_input(enc.encode([dates[index],power[index]]))
File "/home/adam/Documents/htm/PyHTM/PyHTM_Multiprocessing_Branch.py", line 1169, in process_input
self.cells[i].update_perms.remote(self.last_active_cells,correct_prediction = True)
File "/home/adam/.local/lib/python3.6/site-packages/ray/actor.py", line 149, in remote
return self._remote(args, kwargs)
File "/home/adam/.local/lib/python3.6/site-packages/ray/actor.py", line 170, in _remote
return invocation(args, kwargs)
File "/home/adam/.local/lib/python3.6/site-packages/ray/actor.py", line 164, in invocation
num_return_vals=num_return_vals)
File "/home/adam/.local/lib/python3.6/site-packages/ray/actor.py", line 554, in _actor_method_call
driver_id=self._ray_actor_driver_id,
File "/home/adam/.local/lib/python3.6/site-packages/ray/worker.py", line 636, in submit_task
args_for_raylet.append(put(arg))
File "/home/adam/.local/lib/python3.6/site-packages/ray/worker.py", line 2219, in put
worker.put_object(object_id, value)
File "/home/adam/.local/lib/python3.6/site-packages/ray/worker.py", line 383, in put_object
self.store_and_register(object_id, value)
File "/home/adam/.local/lib/python3.6/site-packages/ray/worker.py", line 317, in store_and_register
self.task_driver_id))
File "/home/adam/.local/lib/python3.6/site-packages/ray/worker.py", line 246, in get_serialization_context
_initialize_serialization(driver_id)
File "/home/adam/.local/lib/python3.6/site-packages/ray/worker.py", line 1141, in _initialize_serialization
serialization_context = pyarrow.default_serialization_context()
File "/home/adam/.local/lib/python3.6/site-packages/ray/pyarrow_files/pyarrow/serialization.py", line 350, in default_serialization_context
register_default_serialization_handlers(context)
File "/home/adam/.local/lib/python3.6/site-packages/ray/pyarrow_files/pyarrow/serialization.py", line 345, in register_default_serialization_handlers
_register_custom_pandas_handlers(serialization_context)
File "/home/adam/.local/lib/python3.6/site-packages/ray/pyarrow_files/pyarrow/serialization.py", line 148, in _register_custom_pandas_handlers
import pandas as pd
File "/usr/local/lib/python3.6/dist-packages/pandas/__init__.py", line 57, in <module>
from pandas.io.api import *
File "/usr/local/lib/python3.6/dist-packages/pandas/io/api.py", line 19, in <module>
from pandas.io.packers import read_msgpack, to_msgpack
File "/usr/local/lib/python3.6/dist-packages/pandas/io/packers.py", line 69, in <module>
from pandas.util._move import (
ValueError: module functions cannot set METH_CLASS or METH_STATIC
(pid=8992) 2020-07-31 14:18:16,144 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8992) 2020-07-31 14:18:16,144 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.
(pid=8989) 2020-07-31 14:18:16,282 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8989) 2020-07-31 14:18:16,285 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.
(pid=8995) 2020-07-31 14:18:16,397 WARNING worker.py:1331 -- WARNING: Not updating worker name since `setproctitle` is not installed. Install this with `pip install setproctitle` (or ray[debug]) to enable monitoring of worker processes.
(pid=8995) 2020-07-31 14:18:16,397 ERROR worker.py:1337 -- Calling ray.init() again after it has already been called.