Keras 설치

Published on

https://appsilon.com/ship-recognition-in-satellite-imagery-part-i/
https://appsilon.com/ship-recognition-in-satellite-imagery-part-ii/
Posted by Michal Maj  16 January, 2018
install.packages("keras")
#also installing the dependencies ‘config’, ‘reticulate’, ‘tensorflow’, ‘tfruns’

GPU version

GPU를 사용하기 위해서, 

 library(keras)
 install_keras(tensorflow = "gpu") # GPU version
#install_keras()                    # CPU version

별도의 Python 가상환경이 설치되어 있지 않다면, miniconda를 설치여부를 결정하도록 한다. 

> install_keras(tensorflow="gpu")
Using virtual environment '~/.virtualenvs/r-reticulate' ...
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
'
Collecting tensorflow-gpu==1.14.0					Downloading https://files.pythonhosted.org/packages/41/6d/2348df00a34baaabdef0fdb4f46f962f7a8a6720362c26c3a44a249767ea/tensorflow_gpu-1.14.0-cp27-cp27mu-manylinux1_x86_64.whl (377.0MB)
Collecting tensorflow-hub							Using cached https://files.pythonhosted.org/packages/ac/64/3bba86ca49ef21a4add11a4d37e3f6cd05d2e61d207ebe26a8a96b340826/tensorflow_hub-0.6.0-py2.py3-none-any.whl
Collecting tensorflow-probability					Using cached https://files.pythonhosted.org/packages/3e/3a/c10b6c22320531c774402ac7186d1b673374e2a9d12502cbc8d811e4601c/tensorflow_probability-0.7.0-py2.py3-none-any.whl
Collecting keras									Downloading https://files.pythonhosted.org/packages/1b/18/2e1ef121e5560ac24c7ac9e363aa5fa7006c40563c989e7211aba95b793a/Keras-2.3.0-py2.py3-none-any.whl (377kB)
Collecting absl-py>=0.7.0 (from tensorflow-gpu==1.14.0)	
Collecting wrapt>=1.11.1 (from tensorflow-gpu==1.14.0)
Collecting protobuf>=3.6.1 (from tensorflow-gpu==1.14.0)	Using cached https://files.pythonhosted.org/packages/c7/60/19c2c3b563c8a5ebbc5f17982fd794f415cfc4633a8248ab3e23a47662bc/protobuf-3.9.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting keras-preprocessing>=1.0.5 (from tensorflow-gpu==1.14.0)	Using cached https://files.pythonhosted.org/packages/28/6a/8c1f62c37212d9fc441a7e26736df51ce6f0e38455816445471f10da4f0a/Keras_Preprocessing-1.1.0-py2.py3-none-any.whl
Collecting gast>=0.2.0 (from tensorflow-gpu==1.14.0)		Downloading https://files.pythonhosted.org/packages/1f/04/4e36c33f8eb5c5b6c622a1f4859352a6acca7ab387257d4b3c191d23ec1d/gast-0.3.2.tar.gz
Collecting enum34>=1.1.6 (from tensorflow-gpu==1.14.0)		Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting astor>=0.6.0 (from tensorflow-gpu==1.14.0)		Using cached https://files.pythonhosted.org/packages/d1/4f/950dfae467b384fc96bc6469de25d832534f6b4441033c39f914efd13418/astor-0.8.0-py2.py3-none-any.whl
Collecting six>=1.10.0 (from tensorflow-gpu==1.14.0)		Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting tensorboard<1.15.0,>=1.14.0 (from tensorflow-gpu==1.14.0)	Using cached https://files.pythonhosted.org/packages/f4/37/e6a7af1c92c5b68fb427f853b06164b56ea92126bcfd87784334ec5e4d42/tensorboard-1.14.0-py2-none-any.whl
Collecting numpy<2.0,>=1.14.5 (from tensorflow-gpu==1.14.0)	Using cached https://files.pythonhosted.org/packages/d7/b1/3367ea1f372957f97a6752ec725b87886e12af1415216feec9067e31df70/numpy-1.16.5-cp27-cp27mu-manylinux1_x86_64.whl
Collecting tensorflow-estimator<1.15.0rc0,>=1.14.0rc0 (from tensorflow-gpu==1.14.0)	Using cached https://files.pythonhosted.org/packages/3c/d5/21860a5b11caf0678fbc8319341b0ae21a07156911132e0e71bffed0510d/tensorflow_estimator-1.14.0-py2.py3-none-any.whl
Collecting mock>=2.0.0 (from tensorflow-gpu==1.14.0)		Using cached https://files.pythonhosted.org/packages/05/d2/f94e68be6b17f46d2c353564da56e6fb89ef09faeeff3313a046cb810ca9/mock-3.0.5-py2.py3-none-any.whl
Collecting google-pasta>=0.1.6 (from tensorflow-gpu==1.14.0)			Using cached https://files.pythonhosted.org/packages/35/95/d41cd87d147742ef72d5d1dc317318486e3fbffdadf24a60e70dedf01d56/google_pasta-0.1.7-py2-none-any.whl
Collecting backports.weakref>=1.0rc1 (from tensorflow-gpu==1.14.0)		Using cached https://files.pythonhosted.org/packages/88/ec/f598b633c3d5ffe267aaada57d961c94fdfa183c5c3ebda2b6d151943db6/backports.weakref-1.0.post1-py2.py3-none-any.whl
Collecting termcolor>=1.1.0 (from tensorflow-gpu==1.14.0)	
Collecting grpcio>=1.8.6 (from tensorflow-gpu==1.14.0)		Using cached https://files.pythonhosted.org/packages/d6/c3/65db90ec27181edf491c26aa998ae631e50cd1f04ee8d8d513a95e3937f3/grpcio-1.23.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting wheel (from tensorflow-gpu==1.14.0)				Using cached https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl
Collecting keras-applications>=1.0.6 (from tensorflow-gpu==1.14.0)		
Collecting cloudpickle>=0.6.1 (from tensorflow-probability)	Downloading https://files.pythonhosted.org/packages/c1/49/334e279caa3231255725c8e860fa93e72083567625573421db8875846c14/cloudpickle-1.2.2-py2.py3-none-any.whl
Collecting decorator (from tensorflow-probability)			Using cached https://files.pythonhosted.org/packages/5f/88/0075e461560a1e750a0dcbf77f1d9de775028c37a19a346a6c565a257399/decorator-4.4.0-py2.py3-none-any.whl
Collecting scipy>=0.14 (from keras)					Using cached https://files.pythonhosted.org/packages/1d/f6/7c16d60aeb3694e5611976cb4f1eaf1c6b7f1e7c55771d691013405a02ea/scipy-1.2.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pyyaml (from keras)
Collecting h5py (from keras)						Using cached https://files.pythonhosted.org/packages/12/90/3216b8f6d69905a320352a9ca6802a8e39fdb1cd93133c3d4163db8d5f19/h5py-2.10.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting setuptools (from protobuf>=3.6.1->tensorflow-gpu==1.14.0)	Using cached https://files.pythonhosted.org/packages/b2/86/095d2f7829badc207c893dd4ac767e871f6cd547145df797ea26baea4e2e/setuptools-41.2.0-py2.py3-none-any.whl
Collecting futures>=3.1.1; python_version < "3" (from tensorboard<1.15.0,>=1.14.0->tensorflow-gpu==1.14.0)	Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Collecting markdown>=2.6.8 (from tensorboard<1.15.0,>=1.14.0->tensorflow-gpu==1.14.0)		Using cached https://files.pythonhosted.org/packages/c0/4e/fd492e91abdc2d2fcb70ef453064d980688762079397f779758e055f6575/Markdown-3.1.1-py2.py3-none-any.whl
Collecting werkzeug>=0.11.15 (from tensorboard<1.15.0,>=1.14.0->tensorflow-gpu==1.14.0)		Using cached https://files.pythonhosted.org/packages/b7/61/c0a1adf9ad80db012ed7191af98fa05faa95fa09eceb71bb6fa8b66e6a43/Werkzeug-0.15.6-py2.py3-none-any.whl
Collecting funcsigs>=1; python_version < "3.3" (from mock>=2.0.0->tensorflow-gpu==1.14.0)	Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl

Building wheels for collected packages: gast
  Building wheel for gast (setup.py): started
  Building wheel for gast (setup.py): finished with status 'done'
  Created wheel for gast: filename=gast-0.3.2-cp27-none-any.whl size=9678 sha256=0aefa032bbdca8e55cb00d63c4deb0b511e3b04198304652510c5f1f6919a858
  Stored in directory: /home/sixx/.cache/pip/wheels/59/38/c6/234dc39b4f6951a0768fbc02d5b7207137a5b1d9094f0d54bf
Successfully built gast

Installing collected packages: enum34, six, absl-py, wrapt, setuptools, protobuf, numpy, keras-preprocessing, gast, astor, futures, markdown, grpcio, wheel, werkzeug, tensorboard, tensorflow-estimator, funcsigs, mock, google-pasta, backports.weakref, termcolor, h5py, keras-applications, tensorflow-gpu, tensorflow-hub, cloudpickle, decorator, tensorflow-probability, scipy, pyyaml, keras
Successfully installed absl-py-0.8.0 astor-0.8.0 backports.weakref-1.0.post1 cloudpickle-1.2.2 decorator-4.4.0 enum34-1.1.6 funcsigs-1.0.2 futures-3.3.0 gast-0.3.2 google-pasta-0.1.7 grpcio-1.23.0 h5py-2.10.0 keras-2.3.0 keras-applications-1.0.8 keras-preprocessing-1.1.0 markdown-3.1.1 mock-3.0.5 numpy-1.16.5 protobuf-3.9.1 pyyaml-5.1.2 scipy-1.2.2 setuptools-41.2.0 six-1.12.0 tensorboard-1.14.0 tensorflow-estimator-1.14.0 tensorflow-gpu-1.14.0 tensorflow-hub-0.6.0 tensorflow-probability-0.7.0 termcolor-1.1.0 werkzeug-0.15.6 wheel-0.33.6 wrapt-1.11.2

Installation complete.

CPU version

Keras 패키지에서 install_keras()를 통해 설치한다. 

 library(keras)
#install_keras(tensorflow = "gpu") # GPU version
 install_keras()                    # CPU version
> install_keras()      

Creating virtual environment '~/.virtualenvs/r-reticulate' ...
Using python: /usr/bin/python3.5
New python executable in /home/sixx/.virtualenvs/r-reticulate/bin/python2
Also creating executable in /home/sixx/.virtualenvs/r-reticulate/bin/python

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting pip				Using cached https://files.pythonhosted.org/packages/30/db/9e38760b32e3e7f40cce46dd5fb107b8c73840df38f0046d8e6514e675a1/pip-19.2.3-py2.py3-none-any.whl
Collecting wheel			Using cached https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl
Collecting setuptools		Using cached https://files.pythonhosted.org/packages/b2/86/095d2f7829badc207c893dd4ac767e871f6cd547145df797ea26baea4e2e/setuptools-41.2.0-py2.py3-none-any.whl
Installing collected packages: pip, wheel, setuptools
Successfully installed pip-19.2.3 setuptools-41.2.0 wheel-0.33.6

Using virtual environment '~/.virtualenvs/r-reticulate' ...
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting tensorflow==1.14.0						Downloading https://files.pythonhosted.org/packages/d3/59/d88fe8c58ffb66aca21d03c0e290cd68327cc133591130c674985e98a482/tensorflow-1.14.0-cp27-cp27mu-manylinux1_x86_64.whl (109.2MB)
Collecting tensorflow-hub							Downloading https://files.pythonhosted.org/packages/ac/64/3bba86ca49ef21a4add11a4d37e3f6cd05d2e61d207ebe26a8a96b340826/tensorflow_hub-0.6.0-py2.py3-none-any.whl (84kB)
Collecting tensorflow-probability					Downloading https://files.pythonhosted.org/packages/3e/3a/c10b6c22320531c774402ac7186d1b673374e2a9d12502cbc8d811e4601c/tensorflow_probability-0.7.0-py2.py3-none-any.whl (981kB)
Collecting keras									Downloading https://files.pythonhosted.org/packages/f8/ba/2d058dcf1b85b9c212cc58264c98a4a7dd92c989b798823cc5690d062bb2/Keras-2.2.5-py2.py3-none-any.whl (336kB)
Collecting absl-py>=0.7.0 (from tensorflow==1.14.0)	Downloading https://files.pythonhosted.org/packages/3c/0d/7cbf64cac3f93617a2b6b079c0182e4a83a3e7a8964d3b0cc3d9758ba002/absl-py-0.8.0.tar.gz (102kB)
Collecting wrapt>=1.11.1 (from tensorflow==1.14.0)	Downloading https://files.pythonhosted.org/packages/23/84/323c2415280bc4fc880ac5050dddfb3c8062c2552b34c2e512eb4aa68f79/wrapt-1.11.2.tar.gz
Collecting protobuf>=3.6.1 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/c7/60/19c2c3b563c8a5ebbc5f17982fd794f415cfc4633a8248ab3e23a47662bc/protobuf-3.9.1-cp27-cp27mu-manylinux1_x86_64.whl (1.2MB)
Collecting keras-preprocessing>=1.0.5 (from tensorflow==1.14.0)	Downloading https://files.pythonhosted.org/packages/28/6a/8c1f62c37212d9fc441a7e26736df51ce6f0e38455816445471f10da4f0a/Keras_Preprocessing-1.1.0-py2.py3-none-any.whl (41kB)
Collecting gast>=0.2.0 (from tensorflow==1.14.0)	Downloading https://files.pythonhosted.org/packages/4d/17/a92e707853e2fb48aea76dcdc200ea9a2f7d1ce6d1eff07ddfcf326184cb/gast-0.3.1.tar.gz
Collecting enum34>=1.1.6 (from tensorflow==1.14.0)	Downloading https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Collecting astor>=0.6.0 (from tensorflow==1.14.0)	Downloading https://files.pythonhosted.org/packages/d1/4f/950dfae467b384fc96bc6469de25d832534f6b4441033c39f914efd13418/astor-0.8.0-py2.py3-none-any.whl
Collecting six>=1.10.0 (from tensorflow==1.14.0)	Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting tensorboard<1.15.0,>=1.14.0 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/f4/37/e6a7af1c92c5b68fb427f853b06164b56ea92126bcfd87784334ec5e4d42/tensorboard-1.14.0-py2-none-any.whl (3.1MB)
Collecting numpy<2.0,>=1.14.5 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/d7/b1/3367ea1f372957f97a6752ec725b87886e12af1415216feec9067e31df70/numpy-1.16.5-cp27-cp27mu-manylinux1_x86_64.whl (17.0MB)
Collecting tensorflow-estimator<1.15.0rc0,>=1.14.0rc0 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/3c/d5/21860a5b11caf0678fbc8319341b0ae21a07156911132e0e71bffed0510d/tensorflow_estimator-1.14.0-py2.py3-none-any.whl (488kB)
Collecting mock>=2.0.0 (from tensorflow==1.14.0) 	Downloading https://files.pythonhosted.org/packages/05/d2/f94e68be6b17f46d2c353564da56e6fb89ef09faeeff3313a046cb810ca9/mock-3.0.5-py2.py3-none-any.whl
Collecting google-pasta>=0.1.6 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/35/95/d41cd87d147742ef72d5d1dc317318486e3fbffdadf24a60e70dedf01d56/google_pasta-0.1.7-py2-none-any.whl (55kB)
Collecting backports.weakref>=1.0rc1 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/88/ec/f598b633c3d5ffe267aaada57d961c94fdfa183c5c3ebda2b6d151943db6/backports.weakref-1.0.post1-py2.py3-none-any.whl
Collecting termcolor>=1.1.0 (from tensorflow==1.14.0)	Downloading https://files.pythonhosted.org/packages/8a/48/a76be51647d0eb9f10e2a4511bf3ffb8cc1e6b14e9e4fab46173aa79f981/termcolor-1.1.0.tar.gz
Collecting grpcio>=1.8.6 (from tensorflow==1.14.0)	Downloading https://files.pythonhosted.org/packages/d6/c3/65db90ec27181edf491c26aa998ae631e50cd1f04ee8d8d513a95e3937f3/grpcio-1.23.0-cp27-cp27mu-manylinux1_x86_64.whl (2.2MB)
Collecting wheel (from tensorflow==1.14.0)			Using cached https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl
Collecting keras-applications>=1.0.6 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/21/56/4bcec5a8d9503a87e58e814c4e32ac2b32c37c685672c30bc8c54c6e478a/Keras_Applications-1.0.8.tar.gz (289kB)
Collecting cloudpickle>=0.6.1 (from tensorflow-probability) Downloading https://files.pythonhosted.org/packages/09/f4/4a080c349c1680a2086196fcf0286a65931708156f39568ed7051e42ff6a/cloudpickle-1.2.1-py2.py3-none-any.whl
Collecting decorator (from tensorflow-probability) 	Downloading https://files.pythonhosted.org/packages/5f/88/0075e461560a1e750a0dcbf77f1d9de775028c37a19a346a6c565a257399/decorator-4.4.0-py2.py3-none-any.whl
Collecting scipy>=0.14 (from keras)					Downloading https://files.pythonhosted.org/packages/1d/f6/7c16d60aeb3694e5611976cb4f1eaf1c6b7f1e7c55771d691013405a02ea/scipy-1.2.2-cp27-cp27mu-manylinux1_x86_64.whl (24.8MB)
Collecting pyyaml (from keras)						Downloading https://files.pythonhosted.org/packages/e3/e8/b3212641ee2718d556df0f23f78de8303f068fe29cdaa7a91018849582fe/PyYAML-5.1.2.tar.gz (265kB)
Collecting h5py (from keras)						Downloading https://files.pythonhosted.org/packages/12/90/3216b8f6d69905a320352a9ca6802a8e39fdb1cd93133c3d4163db8d5f19/h5py-2.10.0-cp27-cp27mu-manylinux1_x86_64.whl (2.8MB)
Collecting setuptools (from protobuf>=3.6.1->tensorflow==1.14.0) Using cached https://files.pythonhosted.org/packages/b2/86/095d2f7829badc207c893dd4ac767e871f6cd547145df797ea26baea4e2e/setuptools-41.2.0-py2.py3-none-any.whl
Collecting futures>=3.1.1; python_version < "3" (from tensorboard<1.15.0,>=1.14.0->tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl
Collecting markdown>=2.6.8 (from tensorboard<1.15.0,>=1.14.0->tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/c0/4e/fd492e91abdc2d2fcb70ef453064d980688762079397f779758e055f6575/Markdown-3.1.1-py2.py3-none-any.whl (87kB)
Collecting werkzeug>=0.11.15 (from tensorboard<1.15.0,>=1.14.0->tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/b7/61/c0a1adf9ad80db012ed7191af98fa05faa95fa09eceb71bb6fa8b66e6a43/Werkzeug-0.15.6-py2.py3-none-any.whl (328kB)
Collecting funcsigs>=1; python_version < "3.3" (from mock>=2.0.0->tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl

Building wheels for collected packages: absl-py, wrapt, gast, termcolor, keras-applications, pyyaml
...
Successfully built absl-py wrapt gast termcolor keras-applications pyyaml

Installing collected packages: six, enum34, absl-py, wrapt, setuptools, protobuf, numpy, keras-preprocessing, gast, astor, futures, markdown, grpcio, wheel, werkzeug, tensorboard, tensorflow-estimator, funcsigs, mock, google-pasta, backports.weakref, termcolor, h5py, keras-applications, tensorflow, tensorflow-hub, cloudpickle, decorator, tensorflow-probability, scipy, pyyaml, keras
Successfully installed absl-py-0.8.0 astor-0.8.0 backports.weakref-1.0.post1 cloudpickle-1.2.1 decorator-4.4.0 enum34-1.1.6 funcsigs-1.0.2 futures-3.3.0 gast-0.3.1 google-pasta-0.1.7 grpcio-1.23.0 h5py-2.10.0 keras-2.2.5 keras-applications-1.0.8 keras-preprocessing-1.1.0 markdown-3.1.1 mock-3.0.5 numpy-1.16.5 protobuf-3.9.1 pyyaml-5.1.2 scipy-1.2.2 setuptools-41.2.0 six-1.12.0 tensorboard-1.14.0 tensorflow-1.14.0 tensorflow-estimator-1.14.0 tensorflow-hub-0.6.0 tensorflow-probability-0.7.0 termcolor-1.1.0 werkzeug-0.15.6 wheel-0.33.6 wrapt-1.11.2

Installation complete.

설치시 에러 처리 (for mac)

The script wheel is installed in ‘/Users/onesixx/.virtualenvs/r-reticulate/bin’ which is not on PATH.

...
Building wheel for h5py (setup.py): started
Building wheel for h5py (setup.py): finished with status 'error'
ERROR: Complete output from command /usr/local/opt/python/bin/python3.7 -u -c 'import setuptools, tokenize;__file__='"'"'/private/var/folders/8t/_3rhwzhn4rxfb0mq0_cnxxd00000gn/T/pip-install-7sqq82cs/h5py/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\\r\
'"'"', '"'"'\
'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/8t/_3rhwzhn4rxfb0mq0_cnxxd00000gn/T/pip-wheel-9s_6ot3y --python-tag cp37:
ERROR: running bdist_wheel
...

HDF5(Hierarchical Data Format) for Python — The h5py package is a Pythonic interface to the HDF5 binary data format. 

~|➜ brew install hdf5

GPU 설치전 사전 진행사항

https://www.cyberciti.biz/faq/linux-tell-which-graphics-vga-card-installed/
https://www.tensorflow.org/install/source#linux

Hardware requirements

NVIDIA® GPU card with CUDA® Compute Capability 3.5 or higher.
CUDA-enabled GPU cards에서 확인

ex> GeForce GTX 1080 / 8GB

설치된 GPU 확인

GPU Hardware 확인 (by PCI 아이템 리스트)

pci 리스트를 업데이트하고, 리스트를 읽어서 VGA compatible controller를 찾아보면, 

$ sudo update-pciids
Downloaded daily snapshot dated 2019-09-14 03:15:02

$ lspci -v | less
또는 간단하게 찾기 위해 
$ lspci | grep -i --color 'vga\\|3d\\|2d'
02:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1080] (rev a1) (prog-if 00 [VGA controller])

$ sudo lshw -C display
$ sudo lshw -short | grep -i --color display
/0/100/3/0                     display     GP104 [GeForce GTX 1080]

찾은 GPU의 ID를 이용해서, 해당 카드의 세부정보를 찾아보면, 

$ sudo lspci -v -s 02:00.0
02:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1080] (rev a1) (prog-if 00 [VGA controller])
        Subsystem: Micro-Star International Co., Ltd. [MSI] Device 3367
        Physical Slot: 1
        Flags: bus master, fast devsel, latency 0, IRQ 26
        Memory at fa000000 (32-bit, non-prefetchable) [size=16M]
        Memory at e0000000 (64-bit, prefetchable) [size=256M]
        Memory at f0000000 (64-bit, prefetchable) [size=32M]
        I/O ports at e000 [size=128]
        [virtual] Expansion ROM at 000c0000 [disabled] [size=128K]

        Capabilities: [60] Power Management version 3
        Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [78] Express Legacy Endpoint, MSI 00
        Capabilities: [100] Virtual Channel
        Capabilities: [250] Latency Tolerance Reporting
        Capabilities: [128] Power Budgeting 
        Capabilities: [420] Advanced Error Reporting
        Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 
        Capabilities: [900] #19

        Kernel driver in use: nvidia
        Kernel modules: nvidiafb, nouveau, nvidia_430_drm, nvidia_430

GPU driver 확인

$ ubuntu-drivers devices
== /sys/devices/pci0000:00/0000:00:03.0/0000:02:00.0 ==
vendor   : NVIDIA Corporation
modalias : pci:v000010DEd00001B80sv00001462sd00003367bc03sc00i00
driver   : nvidia-418 - third-party non-free
driver   : nvidia-387 - third-party non-free
driver   : nvidia-415 - third-party free
driver   : nvidia-410 - third-party non-free
driver   : nvidia-430 - third-party free recommended
...
$ cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module  430.26  Tue Jun  4 17:40:52 CDT 2019
GCC version:  gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10)

Software requirements

Tutorial: GeForce GTX 1080Ti GPU NVIDIA Driver Installation in Ubuntu 18.04 (2019)
https://www.tensorflow.org/install/gpu#ubuntu_1604_cuda_10

ex> UBUNTU 16.04 , CUDA 10.0

1. Nvidia GPU driver 설치 (재설치/Upgrade)

1. 설치된 이전 NVIDIA 드라이버 제거 

$ sudo apt-get purge nvidia*

2. repository PPA (Personal Package Archives) 추가후, apt 업데이트

$ sudo add-apt-repository ppa:graphics-drivers/ppa
$ sudo apt-get update

3. NVIDIA 관련 드라이버 확인후, 적합한 NVIDIA driver 와 setting확인툴 설치후,
적용을 위해 Reboot . (추천 버전은 nvidia-driver-418)

$ sudo apt-get install nvidia-{TAB} 으로 확인
#$ sudo apt-get install nvidia-430 nvidia-settings
$ sudo apt-get install nvidia-418 nvidia-settings

$ sudo reboot

4. 드라이버 설치 확인하고, 드라이버의 상세정보 확인 

$ nvidia-smi
Mon Sep 16 15:51:33 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 430.26       Driver Version: 430.26       CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 1080    Off  | 00000000:02:00.0 Off |                  N/A |
|  0%   46C    P5    34W / 240W |      0MiB /  8118MiB |      3%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
cat /proc/driver/nvidia/gpus/{tab}/information
$ cat /proc/driver/nvidia/gpus/0000\\:02\\:00.0/information
Model:           GeForce GTX 1080
IRQ:             68
GPU UUID:        GPU-9efb87f2-c84d-9a34-85ae-8c38b9798eb8
Video BIOS:      86.04.17.00.51
Bus Type:        PCIe
DMA Size:        47 bits
DMA Mask:        0x7fffffffffff
Bus Location:    0000:02:00.0
Device Minor:    0
Blacklisted:     No

2. CUDA 관련 설치 (CUDA Toolkit/ CUPTI/cuDNN/ TensorRT)

  • CUDA® Toolkit —TensorFlow supports CUDA 10.0 (TensorFlow >= 1.13.0)
https://tensorflow.rstudio.com/tools/local_gpu.html
https://www.tensorflow.org/install/install_linux#nvidia_requirements_to_run_tensorflow_with_gpu_support

현재 CUDA 설치 확인

$ dpkg -l | grep cuda-repo-ubuntu1604
ii  cuda-repo-ubuntu1604     10.1.243-1      amd64        cuda repository configuration files

Downgrade cuda 10.1 to 10.0

만약 버전이 10.0이 아인면, downgrade.
TensorFlow 최신버전이 아직 CUDA 10.1을 지원하지 않는다.

https://www.tensorflow.org/install/gpu
$ sudo apt-get --purge remove "cuda*"
$ sudo reboot

NVIDIA CUDA network repository installation package 설치

$ sudo apt-get install gnupg-curl
$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.0.130-1_amd64.deb
$ sudo dpkg -i cuda-repo-ubuntu1804_10.0.130-1_amd64.deb

Add HTTPS support for apt-key

$ sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub
$ sudo apt-get update

NVIDIA Machine Learning network repository 설치

$ wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/nvidia-machine-learning-repo-ubuntu1604_1.0.0-1_amd64.deb
$ sudo apt install ./nvidia-machine-learning-repo-ubuntu1604_1.0.0-1_amd64.deb
$ sudo apt-get update

CUDA Toolkit v10.0 와 CUPTI, cuDNN v7.x 설치

CUDA Toolkit을 설치한다.
CUPTI(CUDA Profiling Tools Interface)는 CUDA Toolkit 설치시 같이 설치되고, Nvidia Driver도 설치된다.
cuDNN(CUDA® Deep Neural Network library)

$ sudo apt-get install --no-install-recommends \\
     cuda-10-0 \\
     libcudnn7=7.6.2.24-1+cuda10.0  \\
     libcudnn7-dev=7.6.2.24-1+cuda10.0

TensorRT 설치

위에서 libcudnn7 가 제대로 설치된 후, TensorRT설치한다. 

$ sudo apt-get install --no-install-recommends \\
     libnvinfer5=5.1.5-1+cuda10.0 \\
     libnvinfer-dev=5.1.5-1+cuda10.0

환경변수 정의 – ENVIRONMENT VARIABLES

  • CUDA HOME
  • CUDA binary 와 CUPTI 의 binary 경로
  • PATH 추가

기본 

$ sudo apt-get install --no-install-recommends \\
     libnvinfer5=5.1.5-1+cuda10.0 \\
     libnvinfer-dev=5.1.5-1+cuda10.0

RStudio Server – MULTI-USER

기본적으로 bash startup file (/etc/profile) 에 정의하여 모든 사용자에 동일하게 적용해야하지만,
RStudio Server를 활용하는 경우, RStudioServer가 R 세션에 대해 system profile 스크립트를 실행하지 않기 때문에, RStudio 설정파일(/etc/rstudio/rserver.conf )에 라이브러리경로를 정의한다. 

$ sudo vi /etc/rstudio/rserver.conf

# Server Configuration File
rsession-ld-library-path=/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64

CUDA Home과 PATH 는 /usr/lib/R/etc/Rprofile.site 활용

$ sudo vi /usr/lib/R/etc/Rprofile.site

Sys.setenv(CUDA_HOME="/usr/local/cuda")
Sys.setenv(PATH=paste(Sys.getenv("PATH"), "/usr/local/cuda/bin", sep = ":"))

* TensorFlow 설치 for all user  multi-user installation 섹션 참고

SINGLE-USER

기본적으로 bash startup file ( ~/.profile) 에 정의한고, 적용을 위해 reboot한다.
(물론  ~/.bash_profile 이나 ~/.bash_login  이 있으면 ~/.profile을 읽지 않기 때문에 해당 파일이 있는지도 확인해야한다.  ~/.bashrc 류는 Terminal 세션만 적용되지만, ~/.profile은 desktop application(RStudio)까지 읽어서 사용하므로 ~/.profile을 사용한다.)

$ sudo vi ~/.profile

# Server Configuration File
rsession-ld-library-path=/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64

CUDA Home과 PATH 는 ~/.Rprofile 활용

$ sudo vi ~/.Rprofile

Sys.setenv(CUDA_HOME="/usr/local/cuda")
Sys.setenv(PATH=paste(Sys.getenv("PATH"), "/usr/local/cuda/bin", sep = ":"))

* <참고> 설치 packages 

cuda-10-0
cuda-command-line-tools-10-0
cuda-compiler-10-0
cuda-cublas-10-0
cuda-cublas-dev-10-0
cuda-cudart-10-0
cuda-cudart-dev-10-0
cuda-cufft-10-0
cuda-cufft-dev-10-0
cuda-cuobjdump-10-0
cuda-cupti-10-0
cuda-curand-10-0
cuda-curand-dev-10-0
cuda-cusolver-10-0
cuda-cusolver-dev-10-0
cuda-cusparse-10-0
cuda-cusparse-dev-10-0
cuda-demo-suite-10-0
cuda-documentation-10-0

cuda-driver-dev-10-0
cuda-drivers 418.87.00-1

cuda-gdb-10-0
cuda-gpu-library-advisor-10-0
cuda-libraries-10-0
cuda-libraries-dev-10-0
cuda-license-10-0
cuda-memcheck-10-0
cuda-misc-headers-10-0
cuda-npp-10-0
cuda-npp-dev-10-0
cuda-nsight-10-0
cuda-nsight-compute-10-0
cuda-nvcc-10-0
cuda-nvdisasm-10-0
cuda-nvgraph-10-0
cuda-nvgraph-dev-10-0
cuda-nvjpeg-10-0
cuda-nvjpeg-dev-10-0
cuda-nvml-dev-10-0
cuda-nvprof-10-0
cuda-nvprune-10-0
cuda-nvrtc-10-0
cuda-nvrtc-dev-10-0
cuda-nvtx-10-0
cuda-nvvp-10-0
cuda-runtime-10-0
cuda-samples-10-0
cuda-toolkit-10-0
cuda-tools-10-0
cuda-visual-tools-10-0

libcuda1-418
nvidia-418
nvidia-418-dev
nvidia-modprobe
nvidia-opencl-icd-418
nvidia-settings

libcudnn7 7.6.2.24-1+cuda10.0
libcudnn7-dev 7.6.2.24-1+cuda10.0
Categories: DeepLearning
Tags:

onesixx

Blog Owner

Subscribe
Notify of
guest

0 Comments
Inline Feedbacks
View all comments

Related Posts

DeepLearning

Optuna

모델의 성능을 개선하기 위해 주로 사용하는 방법은 다음과 같습니다. – 학습 데이터 추가– 딥러닝 네크워크 개선– 하이퍼 파라미터 탐색 하이퍼 파라미터 모델 학습과정을 제어하기 위해 사용하는 파라미터학습 과정 전에 미리 설정 (예, learning rate, bact size, optimizer …) 하이퍼 파라미터는 모델의 성능에 직접적인 영향을 주기 때문에, 주어진 데이터와 알고리즘으로 최상의 결과를 내는 하이퍼 Read more…

DeepLearning

epoch, batch size, iteration의 의미

epoch, batch size, iteration의 의미 https://losskatsu.github.io/machine-learning/epoch-batch/#4-iteration%EC%9D%98-%EC%9D%98%EB%AF%B8 예시 한번 모든 데이터를 학습시키면 좋겠지만, 메모리에 한계 때문에, 하나의 epoch에 모든 데이터를 한꺼번에 집어넣을 수 없다. 전체데이터를 1번학습하려면(1epoch 순방향 역방향 둘 다 포함), 데이터를 100개(batch size)씩 나누어, miniBatch로 Read more…

DeepLearning

Kitty

MS CoCo dataset PASCAL VOC Google Open Images Kitti dataset The Oxford-IIIT Pet Dataset BCCD Dataset https://github.com/open-mmlab/mmdetection/blob/master/demo/MMDet_Tutorial.ipynb ~/mmdetection/demo/MMDet_Tutorial.ipynb (커스터마이징된 데이터셋에 대해 detector를 Train시키기) Image 1개당 Annotation파일(txt) 1개 Training폴더 아래 – Image/000068.jpeg– label /000068.txt train.txt, val.txt Read more…

0
Would love your thoughts, please comment.x
()
x