Connection problem to MySQL, hot_gym_swarm test on Ubuntu 16.04LTS

Engineering NuPIC
, ,
1

Hello, I am a newbie here.
Installing NuPIC on Ubuntu 16.04 LTS seemed to be successful, but when running the hot_gym_swarm test i get the following error:

python ./nupic/examples/opf/clients/hotgym/prediction/one_gym/swarm.py
This script runs a swarm on the input data (rec-center-hourly.csv) and
creates a model parameters file in the model_params directory containing
the best model found by the swarm. Dumps a bunch of crud to stdout because
that is just what swarming does at this point. You really don’t need to
pay any attention to it.

=================================================
= Swarming on rec-center-hourly data…
= Medium swarm. Sit back and relax, this could take awhile.

Generating experiment files in directory: /home/dominikscheurenberg/swarm…
Writing 314 lines…
Writing 114 lines…
done.
None
WARNING:nupic.database.client_jobs_dao:[] First failure in <function connect at 0x7f981794dcf8>; initial retry in 0.1 sec.; timeoutSec=300. Caller stack:
File “./nupic/examples/opf/clients/hotgym/prediction/one_gym/swarm.py”, line 111, in
swarm(INPUT_FILE)
File “./nupic/examples/opf/clients/hotgym/prediction/one_gym/swarm.py”, line 103, in swarm
modelParams = swarmForBestModelParams(SWARM_DESCRIPTION, name)
File “./nupic/examples/opf/clients/hotgym/prediction/one_gym/swarm.py”, line 80, in swarmForBestModelParams
verbosity=0
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/nupic/swarming/permutations_runner.py”, line 271, in runWithConfig
return _runAction(runOptions)
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/nupic/swarming/permutations_runner.py”, line 212, in _runAction
returnValue = _runHyperSearch(runOptions)
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/nupic/swarming/permutations_runner.py”, line 141, in _runHyperSearch
search = _HyperSearchRunner(runOptions)
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/nupic/swarming/permutations_runner.py”, line 412, in init
self.__cjDAO = _clientJobsDB()
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/nupic/swarming/permutations_runner.py”, line 376, in _clientJobsDB
return cjdao.ClientJobsDAO.get()
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/nupic/support/decorators.py”, line 56, in exceptionLoggingWrap
return func(*args, **kwargs)
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/nupic/database/client_jobs_dao.py”, line 548, in get
cjDAO.connect()
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/nupic/support/decorators.py”, line 56, in exceptionLoggingWrap
return func(*args, **kwargs)
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/nupic/support/decorators.py”, line 234, in retryWrap
timeoutSec, ‘’.join(traceback.format_stack()), exc_info=True)
Traceback (most recent call last):
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/nupic/support/decorators.py”, line 208, in retryWrap
result = func(*args, **kwargs)
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/nupic/database/client_jobs_dao.py”, line 633, in connect
with ConnectionFactory.get() as conn:
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/nupic/database/connection.py”, line 167, in get
return cls._connectionPolicy.acquireConnection()
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/nupic/database/connection.py”, line 553, in acquireConnection
dbConn = self._pool.connection(shareable=False)
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/DBUtils/PooledDB.py”, line 331, in connection
con = self.steady_connection()
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/DBUtils/PooledDB.py”, line 279, in steady_connection
*self._args, **self._kwargs)
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/DBUtils/SteadyDB.py”, line 134, in connect
failures, ping, closeable, *args, **kwargs)
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/DBUtils/SteadyDB.py”, line 186, in init
self._store(self._create())
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/DBUtils/SteadyDB.py”, line 190, in _create
con = self._creator(*self._args, **self._kwargs)
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/pymysql/init.py”, line 88, in Connect
return Connection(*args, **kwargs)
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/pymysql/connections.py”, line 626, in init
self._connect()
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/pymysql/connections.py”, line 818, in _connect
2003, “Can’t connect to MySQL server on %r (%s)” % (self.host, e))
OperationalError: (2003, “Can’t connect to MySQL server on ‘localhost’ ([Errno 111] Connection refused)”)

Since it seems, that the installation was successfull:

pip install nupic
Requirement already satisfied (use --upgrade to upgrade): nupic in /home/dominikscheurenberg/.local/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): pytest-cov==2.5.0 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): unittest2==0.5.1 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): nupic.bindings==1.0.0 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): DBUtils==1.1 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): pytest-xdist==1.16.0 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): mock==1.0.1 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): asteval==0.9.1 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): numpy==1.12.1 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): validictory==0.9.1 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): pyproj==1.9.3 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): pytest==3.0.7 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): pycapnp==0.5.12; platform_system == “Linux” or platform_system == “Darwin” in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): coverage==3.7.1 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): prettytable==0.7.2 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): psutil==1.0.1 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil==2.1 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): ordereddict==1.1 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): PyMySQL==0.6.2 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.10 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from nupic)
Requirement already satisfied (use --upgrade to upgrade): execnet>=1.1 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from pytest-xdist==1.16.0->nupic)
Requirement already satisfied (use --upgrade to upgrade): py>=1.4.22 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from pytest-xdist==1.16.0->nupic)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/local/lib/python2.7/dist-packages (from pytest==3.0.7->nupic)
Requirement already satisfied (use --upgrade to upgrade): six in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from python-dateutil==2.1->nupic)
Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /home/dominikscheurenberg/.local/lib/python2.7/site-packages (from execnet>=1.1->pytest-xdist==1.16.0->nupic)

I guess its a problem with the mysql configuration. I tried this (https://github.com/numenta/nupic/issues/2794 and https://github.com/numenta/nupic/issues/2676), but still got the same error.
I am grateful for any hints or workarounds.

1 Like
2

Have you installed mysql server in your machine?

3

Yes i did using sudo apt-get install mysql-server
I have to say i am not really familiar with mysql and do not know how tor what to set up.

here is the output when i access it:

Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 26
Server version: 5.7.19-0ubuntu0.16.04.1 (Ubuntu)

Copyright © 2000, 2017, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.

mysql> quit

4

The swarm accesses mysql with the user of root and empty password in the default circumstance. Have u set password yet?

5

yes i did set a password. Tried it again without a password now, the error now looks slightly different.

python ./nupic/examples/swarm/test_db.py
This script will validate that your MySQL is setup correctly for
NuPIC. MySQL is required for NuPIC swarming. The settings are
defined in a configuration file found in
$NUPIC/src/nupic/support/nupic-default.xml Out of the box those
settings contain MySQL’s default access credentials.

The nupic-default.xml can be duplicated to define user specific
changes calling the copied file
$NUPIC/src/nupic/support/nupic-site.xml Refer to the
nupic-default.xml for additional instructions.

Defaults: localhost, 3306, root, no password

Retrieved the following NuPIC configuration using: nupic-default.xml
host : localhost
port : 3306
user : root
passwd :
Couldn’t connect to the database or you don’t have the permissions required to create databases and tables. Please ensure you have MySQL
installed, running, accessible using the NuPIC configuration settings, and the user specified has permission to create both databases and tables.
Traceback (most recent call last):
File “./nupic/examples/swarm/test_db.py”, line 118, in
dbValidator()
File “./nupic/examples/swarm/test_db.py”, line 114, in dbValidator
testDbConnection(host, port, user, passwd)
File “./nupic/examples/swarm/test_db.py”, line 58, in testDbConnection
conn = pymysql.connect(host=host, port=port, user=user, passwd=passwd)
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/pymysql/init.py”, line 88, in Connect
return Connection(*args, **kwargs)
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/pymysql/connections.py”, line 626, in init
self._connect()
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/pymysql/connections.py”, line 818, in _connect
2003, “Can’t connect to MySQL server on %r (%s)” % (self.host, e))
pymysql.err.OperationalError: (2003, “Can’t connect to MySQL server on ‘localhost’ ((1251, u’Client does not support authentication protocol requested by server; consider upgrading MySQL client’))”)

6

try setting password in the nupic-default.xml and see what’s going on.

7

Ok, so i removed and re-installed mysql-server and set a password. Then wrote the password in the nupic-default.xml and also created the nupic-site.xml.

I still get the same error:

python ./nupic/examples/swarm/test_db.py
This script will validate that your MySQL is setup correctly for
NuPIC. MySQL is required for NuPIC swarming. The settings are
defined in a configuration file found in
$NUPIC/src/nupic/support/nupic-default.xml Out of the box those
settings contain MySQL’s default access credentials.

The nupic-default.xml can be duplicated to define user specific
changes calling the copied file
$NUPIC/src/nupic/support/nupic-site.xml Refer to the
nupic-default.xml for additional instructions.

Defaults: localhost, 3306, root, no password

Retrieved the following NuPIC configuration using: nupic-default.xml
host : localhost
port : 3306
user : root
passwd :
Couldn’t connect to the database or you don’t have the permissions required to create databases and tables. Please ensure you have MySQL
installed, running, accessible using the NuPIC configuration settings, and the user specified has permission to create both databases and tables.
Traceback (most recent call last):
File “./nupic/examples/swarm/test_db.py”, line 118, in
dbValidator()
File “./nupic/examples/swarm/test_db.py”, line 114, in dbValidator
testDbConnection(host, port, user, passwd)
File “./nupic/examples/swarm/test_db.py”, line 58, in testDbConnection
conn = pymysql.connect(host=host, port=port, user=user, passwd=passwd)
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/pymysql/init.py”, line 88, in Connect
return Connection(*args, **kwargs)
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/pymysql/connections.py”, line 626, in init
self._connect()
File “/home/dominikscheurenberg/.local/lib/python2.7/site-packages/pymysql/connections.py”, line 818, in _connect
2003, “Can’t connect to MySQL server on %r (%s)” % (self.host, e))
pymysql.err.OperationalError: (2003, ‘Can’t connect to MySQL server on ‘localhost’ ((1045, u"Access denied for user ‘root’@‘localhost’ (using password: NO)"))’)

8

I install nupic and mysql server in window10, Mysql force me to set password for root, so I set password in nupic-default.xml. It works

Perhaps your issue is out of my capability.

9

Ok, anyways thank you very much for your help !

10

Hey,problem is solved. There was a missing mysql library. Somehow installing the LAMP suite enabled me to set up a mysql server without a password that passed /examples/swarm/test_db.py.

so topic is closed, again thanks for helping.

1 Like
11

Congratulation

12

Hi @Domhtm,

I just want to how you installed the LAMP suite, are there a couple of commands you could show? I’m on Ubuntu and have been dealing with this swarming issue for a while. Thanks!

– Sam

13

Hey,
i basically used this description
(https://legacy.thomas-leister.de/apache2-webserver-php-und-mysql-installieren/ its actually made for 14.04 but it worked for me and sorry its in german…).

I think my main problem was the sequence in which i installed the single aps and creating a server without a password.

heres the commands for the server with a blank password.

create database localhost 
grant all on localhost.* to 'root'@'localhost' identified by ''

i hope i could help you

15

how did you solve it?

16

I actually haven’t solved it as of yet. I’m getting the no password error even though I’ve entered my password into both the nupic-set.xml abd nupic-default.xml files, and had another error when simply trying to install LAMP either. :thinking:

17

What is the path to your nupic-set.xml file? Did you pip install nupic or build from source?

18

Hey Matt! So I did pip install nupic, and the path is:

/home/sheiser1/nupic-master/src/nupic/support

I created the nupic-site.xml file as a duplicate of the nupic-default.xml with my password plugged in, though still getting the ‘NO password’ error

19

I think that pip has it’s own idea of where nupic-default.xml and nupic-site.xml are. Unless you installed from your source directory, editing that copy of nupic-site.xml probably doesn’t help. But you can probably still fix this by finding out where your copy is. You can ask python where your local site-packages are with this:

python -c "import site; print(site.getsitepackages())"

Search those paths for nupic-*.xml.

20

Ok I ran that line and got the directory (/home/sheiser1/anaconda3/lib/python3.6/site-packages), though there was no nupic folder or nupic-default.xml file there :thinking:

21

You’ve got to find out where pip installed NuPIC.