Keras 설치
https://appsilon.com/ship-recognition-in-satellite-imagery-part-i/
https://appsilon.com/ship-recognition-in-satellite-imagery-part-ii/
Posted by Michal Maj 16 January, 2018
install.packages("keras")
#also installing the dependencies ‘config’, ‘reticulate’, ‘tensorflow’, ‘tfruns’
GPU version
GPU를 사용하기 위해서,
library(keras) install_keras(tensorflow = "gpu") # GPU version #install_keras() # CPU version
별도의 Python 가상환경이 설치되어 있지 않다면, miniconda를 설치여부를 결정하도록 한다.
> install_keras(tensorflow="gpu") Using virtual environment '~/.virtualenvs/r-reticulate' ... DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support ' Collecting tensorflow-gpu==1.14.0 Downloading https://files.pythonhosted.org/packages/41/6d/2348df00a34baaabdef0fdb4f46f962f7a8a6720362c26c3a44a249767ea/tensorflow_gpu-1.14.0-cp27-cp27mu-manylinux1_x86_64.whl (377.0MB) Collecting tensorflow-hub Using cached https://files.pythonhosted.org/packages/ac/64/3bba86ca49ef21a4add11a4d37e3f6cd05d2e61d207ebe26a8a96b340826/tensorflow_hub-0.6.0-py2.py3-none-any.whl Collecting tensorflow-probability Using cached https://files.pythonhosted.org/packages/3e/3a/c10b6c22320531c774402ac7186d1b673374e2a9d12502cbc8d811e4601c/tensorflow_probability-0.7.0-py2.py3-none-any.whl Collecting keras Downloading https://files.pythonhosted.org/packages/1b/18/2e1ef121e5560ac24c7ac9e363aa5fa7006c40563c989e7211aba95b793a/Keras-2.3.0-py2.py3-none-any.whl (377kB) Collecting absl-py>=0.7.0 (from tensorflow-gpu==1.14.0) Collecting wrapt>=1.11.1 (from tensorflow-gpu==1.14.0) Collecting protobuf>=3.6.1 (from tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/c7/60/19c2c3b563c8a5ebbc5f17982fd794f415cfc4633a8248ab3e23a47662bc/protobuf-3.9.1-cp27-cp27mu-manylinux1_x86_64.whl Collecting keras-preprocessing>=1.0.5 (from tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/28/6a/8c1f62c37212d9fc441a7e26736df51ce6f0e38455816445471f10da4f0a/Keras_Preprocessing-1.1.0-py2.py3-none-any.whl Collecting gast>=0.2.0 (from tensorflow-gpu==1.14.0) Downloading https://files.pythonhosted.org/packages/1f/04/4e36c33f8eb5c5b6c622a1f4859352a6acca7ab387257d4b3c191d23ec1d/gast-0.3.2.tar.gz Collecting enum34>=1.1.6 (from tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl Collecting astor>=0.6.0 (from tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/d1/4f/950dfae467b384fc96bc6469de25d832534f6b4441033c39f914efd13418/astor-0.8.0-py2.py3-none-any.whl Collecting six>=1.10.0 (from tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl Collecting tensorboard<1.15.0,>=1.14.0 (from tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/f4/37/e6a7af1c92c5b68fb427f853b06164b56ea92126bcfd87784334ec5e4d42/tensorboard-1.14.0-py2-none-any.whl Collecting numpy<2.0,>=1.14.5 (from tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/d7/b1/3367ea1f372957f97a6752ec725b87886e12af1415216feec9067e31df70/numpy-1.16.5-cp27-cp27mu-manylinux1_x86_64.whl Collecting tensorflow-estimator<1.15.0rc0,>=1.14.0rc0 (from tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/3c/d5/21860a5b11caf0678fbc8319341b0ae21a07156911132e0e71bffed0510d/tensorflow_estimator-1.14.0-py2.py3-none-any.whl Collecting mock>=2.0.0 (from tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/05/d2/f94e68be6b17f46d2c353564da56e6fb89ef09faeeff3313a046cb810ca9/mock-3.0.5-py2.py3-none-any.whl Collecting google-pasta>=0.1.6 (from tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/35/95/d41cd87d147742ef72d5d1dc317318486e3fbffdadf24a60e70dedf01d56/google_pasta-0.1.7-py2-none-any.whl Collecting backports.weakref>=1.0rc1 (from tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/88/ec/f598b633c3d5ffe267aaada57d961c94fdfa183c5c3ebda2b6d151943db6/backports.weakref-1.0.post1-py2.py3-none-any.whl Collecting termcolor>=1.1.0 (from tensorflow-gpu==1.14.0) Collecting grpcio>=1.8.6 (from tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/d6/c3/65db90ec27181edf491c26aa998ae631e50cd1f04ee8d8d513a95e3937f3/grpcio-1.23.0-cp27-cp27mu-manylinux1_x86_64.whl Collecting wheel (from tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl Collecting keras-applications>=1.0.6 (from tensorflow-gpu==1.14.0) Collecting cloudpickle>=0.6.1 (from tensorflow-probability) Downloading https://files.pythonhosted.org/packages/c1/49/334e279caa3231255725c8e860fa93e72083567625573421db8875846c14/cloudpickle-1.2.2-py2.py3-none-any.whl Collecting decorator (from tensorflow-probability) Using cached https://files.pythonhosted.org/packages/5f/88/0075e461560a1e750a0dcbf77f1d9de775028c37a19a346a6c565a257399/decorator-4.4.0-py2.py3-none-any.whl Collecting scipy>=0.14 (from keras) Using cached https://files.pythonhosted.org/packages/1d/f6/7c16d60aeb3694e5611976cb4f1eaf1c6b7f1e7c55771d691013405a02ea/scipy-1.2.2-cp27-cp27mu-manylinux1_x86_64.whl Collecting pyyaml (from keras) Collecting h5py (from keras) Using cached https://files.pythonhosted.org/packages/12/90/3216b8f6d69905a320352a9ca6802a8e39fdb1cd93133c3d4163db8d5f19/h5py-2.10.0-cp27-cp27mu-manylinux1_x86_64.whl Collecting setuptools (from protobuf>=3.6.1->tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/b2/86/095d2f7829badc207c893dd4ac767e871f6cd547145df797ea26baea4e2e/setuptools-41.2.0-py2.py3-none-any.whl Collecting futures>=3.1.1; python_version < "3" (from tensorboard<1.15.0,>=1.14.0->tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl Collecting markdown>=2.6.8 (from tensorboard<1.15.0,>=1.14.0->tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/c0/4e/fd492e91abdc2d2fcb70ef453064d980688762079397f779758e055f6575/Markdown-3.1.1-py2.py3-none-any.whl Collecting werkzeug>=0.11.15 (from tensorboard<1.15.0,>=1.14.0->tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/b7/61/c0a1adf9ad80db012ed7191af98fa05faa95fa09eceb71bb6fa8b66e6a43/Werkzeug-0.15.6-py2.py3-none-any.whl Collecting funcsigs>=1; python_version < "3.3" (from mock>=2.0.0->tensorflow-gpu==1.14.0) Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl Building wheels for collected packages: gast Building wheel for gast (setup.py): started Building wheel for gast (setup.py): finished with status 'done' Created wheel for gast: filename=gast-0.3.2-cp27-none-any.whl size=9678 sha256=0aefa032bbdca8e55cb00d63c4deb0b511e3b04198304652510c5f1f6919a858 Stored in directory: /home/sixx/.cache/pip/wheels/59/38/c6/234dc39b4f6951a0768fbc02d5b7207137a5b1d9094f0d54bf Successfully built gast Installing collected packages: enum34, six, absl-py, wrapt, setuptools, protobuf, numpy, keras-preprocessing, gast, astor, futures, markdown, grpcio, wheel, werkzeug, tensorboard, tensorflow-estimator, funcsigs, mock, google-pasta, backports.weakref, termcolor, h5py, keras-applications, tensorflow-gpu, tensorflow-hub, cloudpickle, decorator, tensorflow-probability, scipy, pyyaml, keras Successfully installed absl-py-0.8.0 astor-0.8.0 backports.weakref-1.0.post1 cloudpickle-1.2.2 decorator-4.4.0 enum34-1.1.6 funcsigs-1.0.2 futures-3.3.0 gast-0.3.2 google-pasta-0.1.7 grpcio-1.23.0 h5py-2.10.0 keras-2.3.0 keras-applications-1.0.8 keras-preprocessing-1.1.0 markdown-3.1.1 mock-3.0.5 numpy-1.16.5 protobuf-3.9.1 pyyaml-5.1.2 scipy-1.2.2 setuptools-41.2.0 six-1.12.0 tensorboard-1.14.0 tensorflow-estimator-1.14.0 tensorflow-gpu-1.14.0 tensorflow-hub-0.6.0 tensorflow-probability-0.7.0 termcolor-1.1.0 werkzeug-0.15.6 wheel-0.33.6 wrapt-1.11.2 Installation complete.
CPU version
Keras 패키지에서 install_keras()를 통해 설치한다.
library(keras) #install_keras(tensorflow = "gpu") # GPU version install_keras() # CPU version
> install_keras() Creating virtual environment '~/.virtualenvs/r-reticulate' ... Using python: /usr/bin/python3.5 New python executable in /home/sixx/.virtualenvs/r-reticulate/bin/python2 Also creating executable in /home/sixx/.virtualenvs/r-reticulate/bin/python Installing setuptools, pkg_resources, pip, wheel...done. Running virtualenv with interpreter /usr/bin/python2 DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support Collecting pip Using cached https://files.pythonhosted.org/packages/30/db/9e38760b32e3e7f40cce46dd5fb107b8c73840df38f0046d8e6514e675a1/pip-19.2.3-py2.py3-none-any.whl Collecting wheel Using cached https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl Collecting setuptools Using cached https://files.pythonhosted.org/packages/b2/86/095d2f7829badc207c893dd4ac767e871f6cd547145df797ea26baea4e2e/setuptools-41.2.0-py2.py3-none-any.whl Installing collected packages: pip, wheel, setuptools Successfully installed pip-19.2.3 setuptools-41.2.0 wheel-0.33.6 Using virtual environment '~/.virtualenvs/r-reticulate' ... DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support Collecting tensorflow==1.14.0 Downloading https://files.pythonhosted.org/packages/d3/59/d88fe8c58ffb66aca21d03c0e290cd68327cc133591130c674985e98a482/tensorflow-1.14.0-cp27-cp27mu-manylinux1_x86_64.whl (109.2MB) Collecting tensorflow-hub Downloading https://files.pythonhosted.org/packages/ac/64/3bba86ca49ef21a4add11a4d37e3f6cd05d2e61d207ebe26a8a96b340826/tensorflow_hub-0.6.0-py2.py3-none-any.whl (84kB) Collecting tensorflow-probability Downloading https://files.pythonhosted.org/packages/3e/3a/c10b6c22320531c774402ac7186d1b673374e2a9d12502cbc8d811e4601c/tensorflow_probability-0.7.0-py2.py3-none-any.whl (981kB) Collecting keras Downloading https://files.pythonhosted.org/packages/f8/ba/2d058dcf1b85b9c212cc58264c98a4a7dd92c989b798823cc5690d062bb2/Keras-2.2.5-py2.py3-none-any.whl (336kB) Collecting absl-py>=0.7.0 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/3c/0d/7cbf64cac3f93617a2b6b079c0182e4a83a3e7a8964d3b0cc3d9758ba002/absl-py-0.8.0.tar.gz (102kB) Collecting wrapt>=1.11.1 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/23/84/323c2415280bc4fc880ac5050dddfb3c8062c2552b34c2e512eb4aa68f79/wrapt-1.11.2.tar.gz Collecting protobuf>=3.6.1 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/c7/60/19c2c3b563c8a5ebbc5f17982fd794f415cfc4633a8248ab3e23a47662bc/protobuf-3.9.1-cp27-cp27mu-manylinux1_x86_64.whl (1.2MB) Collecting keras-preprocessing>=1.0.5 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/28/6a/8c1f62c37212d9fc441a7e26736df51ce6f0e38455816445471f10da4f0a/Keras_Preprocessing-1.1.0-py2.py3-none-any.whl (41kB) Collecting gast>=0.2.0 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/4d/17/a92e707853e2fb48aea76dcdc200ea9a2f7d1ce6d1eff07ddfcf326184cb/gast-0.3.1.tar.gz Collecting enum34>=1.1.6 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl Collecting astor>=0.6.0 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/d1/4f/950dfae467b384fc96bc6469de25d832534f6b4441033c39f914efd13418/astor-0.8.0-py2.py3-none-any.whl Collecting six>=1.10.0 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl Collecting tensorboard<1.15.0,>=1.14.0 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/f4/37/e6a7af1c92c5b68fb427f853b06164b56ea92126bcfd87784334ec5e4d42/tensorboard-1.14.0-py2-none-any.whl (3.1MB) Collecting numpy<2.0,>=1.14.5 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/d7/b1/3367ea1f372957f97a6752ec725b87886e12af1415216feec9067e31df70/numpy-1.16.5-cp27-cp27mu-manylinux1_x86_64.whl (17.0MB) Collecting tensorflow-estimator<1.15.0rc0,>=1.14.0rc0 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/3c/d5/21860a5b11caf0678fbc8319341b0ae21a07156911132e0e71bffed0510d/tensorflow_estimator-1.14.0-py2.py3-none-any.whl (488kB) Collecting mock>=2.0.0 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/05/d2/f94e68be6b17f46d2c353564da56e6fb89ef09faeeff3313a046cb810ca9/mock-3.0.5-py2.py3-none-any.whl Collecting google-pasta>=0.1.6 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/35/95/d41cd87d147742ef72d5d1dc317318486e3fbffdadf24a60e70dedf01d56/google_pasta-0.1.7-py2-none-any.whl (55kB) Collecting backports.weakref>=1.0rc1 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/88/ec/f598b633c3d5ffe267aaada57d961c94fdfa183c5c3ebda2b6d151943db6/backports.weakref-1.0.post1-py2.py3-none-any.whl Collecting termcolor>=1.1.0 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/8a/48/a76be51647d0eb9f10e2a4511bf3ffb8cc1e6b14e9e4fab46173aa79f981/termcolor-1.1.0.tar.gz Collecting grpcio>=1.8.6 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/d6/c3/65db90ec27181edf491c26aa998ae631e50cd1f04ee8d8d513a95e3937f3/grpcio-1.23.0-cp27-cp27mu-manylinux1_x86_64.whl (2.2MB) Collecting wheel (from tensorflow==1.14.0) Using cached https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl Collecting keras-applications>=1.0.6 (from tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/21/56/4bcec5a8d9503a87e58e814c4e32ac2b32c37c685672c30bc8c54c6e478a/Keras_Applications-1.0.8.tar.gz (289kB) Collecting cloudpickle>=0.6.1 (from tensorflow-probability) Downloading https://files.pythonhosted.org/packages/09/f4/4a080c349c1680a2086196fcf0286a65931708156f39568ed7051e42ff6a/cloudpickle-1.2.1-py2.py3-none-any.whl Collecting decorator (from tensorflow-probability) Downloading https://files.pythonhosted.org/packages/5f/88/0075e461560a1e750a0dcbf77f1d9de775028c37a19a346a6c565a257399/decorator-4.4.0-py2.py3-none-any.whl Collecting scipy>=0.14 (from keras) Downloading https://files.pythonhosted.org/packages/1d/f6/7c16d60aeb3694e5611976cb4f1eaf1c6b7f1e7c55771d691013405a02ea/scipy-1.2.2-cp27-cp27mu-manylinux1_x86_64.whl (24.8MB) Collecting pyyaml (from keras) Downloading https://files.pythonhosted.org/packages/e3/e8/b3212641ee2718d556df0f23f78de8303f068fe29cdaa7a91018849582fe/PyYAML-5.1.2.tar.gz (265kB) Collecting h5py (from keras) Downloading https://files.pythonhosted.org/packages/12/90/3216b8f6d69905a320352a9ca6802a8e39fdb1cd93133c3d4163db8d5f19/h5py-2.10.0-cp27-cp27mu-manylinux1_x86_64.whl (2.8MB) Collecting setuptools (from protobuf>=3.6.1->tensorflow==1.14.0) Using cached https://files.pythonhosted.org/packages/b2/86/095d2f7829badc207c893dd4ac767e871f6cd547145df797ea26baea4e2e/setuptools-41.2.0-py2.py3-none-any.whl Collecting futures>=3.1.1; python_version < "3" (from tensorboard<1.15.0,>=1.14.0->tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl Collecting markdown>=2.6.8 (from tensorboard<1.15.0,>=1.14.0->tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/c0/4e/fd492e91abdc2d2fcb70ef453064d980688762079397f779758e055f6575/Markdown-3.1.1-py2.py3-none-any.whl (87kB) Collecting werkzeug>=0.11.15 (from tensorboard<1.15.0,>=1.14.0->tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/b7/61/c0a1adf9ad80db012ed7191af98fa05faa95fa09eceb71bb6fa8b66e6a43/Werkzeug-0.15.6-py2.py3-none-any.whl (328kB) Collecting funcsigs>=1; python_version < "3.3" (from mock>=2.0.0->tensorflow==1.14.0) Downloading https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl Building wheels for collected packages: absl-py, wrapt, gast, termcolor, keras-applications, pyyaml ... Successfully built absl-py wrapt gast termcolor keras-applications pyyaml Installing collected packages: six, enum34, absl-py, wrapt, setuptools, protobuf, numpy, keras-preprocessing, gast, astor, futures, markdown, grpcio, wheel, werkzeug, tensorboard, tensorflow-estimator, funcsigs, mock, google-pasta, backports.weakref, termcolor, h5py, keras-applications, tensorflow, tensorflow-hub, cloudpickle, decorator, tensorflow-probability, scipy, pyyaml, keras Successfully installed absl-py-0.8.0 astor-0.8.0 backports.weakref-1.0.post1 cloudpickle-1.2.1 decorator-4.4.0 enum34-1.1.6 funcsigs-1.0.2 futures-3.3.0 gast-0.3.1 google-pasta-0.1.7 grpcio-1.23.0 h5py-2.10.0 keras-2.2.5 keras-applications-1.0.8 keras-preprocessing-1.1.0 markdown-3.1.1 mock-3.0.5 numpy-1.16.5 protobuf-3.9.1 pyyaml-5.1.2 scipy-1.2.2 setuptools-41.2.0 six-1.12.0 tensorboard-1.14.0 tensorflow-1.14.0 tensorflow-estimator-1.14.0 tensorflow-hub-0.6.0 tensorflow-probability-0.7.0 termcolor-1.1.0 werkzeug-0.15.6 wheel-0.33.6 wrapt-1.11.2 Installation complete.
설치시 에러 처리 (for mac)
The script wheel is installed in ‘/Users/onesixx/.virtualenvs/r-reticulate/bin’ which is not on PATH.
...
Building wheel for h5py (setup.py): started
Building wheel for h5py (setup.py): finished with status 'error'
ERROR: Complete output from command /usr/local/opt/python/bin/python3.7 -u -c 'import setuptools, tokenize;__file__='"'"'/private/var/folders/8t/_3rhwzhn4rxfb0mq0_cnxxd00000gn/T/pip-install-7sqq82cs/h5py/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\\r\
'"'"', '"'"'\
'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/8t/_3rhwzhn4rxfb0mq0_cnxxd00000gn/T/pip-wheel-9s_6ot3y --python-tag cp37:
ERROR: running bdist_wheel
...
HDF5(Hierarchical Data Format) for Python — The h5py package is a Pythonic interface to the HDF5 binary data format.
~|➜ brew install hdf5
GPU 설치전 사전 진행사항
https://www.cyberciti.biz/faq/linux-tell-which-graphics-vga-card-installed/
https://www.tensorflow.org/install/source#linux
Hardware requirements
NVIDIA® GPU card with CUDA® Compute Capability 3.5 or higher.
CUDA-enabled GPU cards에서 확인
ex> GeForce GTX 1080 / 8GB
설치된 GPU 확인
GPU Hardware 확인 (by PCI 아이템 리스트)
pci 리스트를 업데이트하고, 리스트를 읽어서 VGA compatible controller를 찾아보면,
$ sudo update-pciids Downloaded daily snapshot dated 2019-09-14 03:15:02 $ lspci -v | less 또는 간단하게 찾기 위해 $ lspci | grep -i --color 'vga\\|3d\\|2d' 02:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1080] (rev a1) (prog-if 00 [VGA controller]) $ sudo lshw -C display $ sudo lshw -short | grep -i --color display /0/100/3/0 display GP104 [GeForce GTX 1080]
찾은 GPU의 ID를 이용해서, 해당 카드의 세부정보를 찾아보면,
$ sudo lspci -v -s 02:00.0
02:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1080] (rev a1) (prog-if 00 [VGA controller])
Subsystem: Micro-Star International Co., Ltd. [MSI] Device 3367
Physical Slot: 1
Flags: bus master, fast devsel, latency 0, IRQ 26
Memory at fa000000 (32-bit, non-prefetchable) [size=16M]
Memory at e0000000 (64-bit, prefetchable) [size=256M]
Memory at f0000000 (64-bit, prefetchable) [size=32M]
I/O ports at e000 [size=128]
[virtual] Expansion ROM at 000c0000 [disabled] [size=128K]
Capabilities: [60] Power Management version 3
Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [78] Express Legacy Endpoint, MSI 00
Capabilities: [100] Virtual Channel
Capabilities: [250] Latency Tolerance Reporting
Capabilities: [128] Power Budgeting
Capabilities: [420] Advanced Error Reporting
Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024
Capabilities: [900] #19
Kernel driver in use: nvidia
Kernel modules: nvidiafb, nouveau, nvidia_430_drm, nvidia_430
GPU driver 확인
$ ubuntu-drivers devices == /sys/devices/pci0000:00/0000:00:03.0/0000:02:00.0 == vendor : NVIDIA Corporation modalias : pci:v000010DEd00001B80sv00001462sd00003367bc03sc00i00 driver : nvidia-418 - third-party non-free driver : nvidia-387 - third-party non-free driver : nvidia-415 - third-party free driver : nvidia-410 - third-party non-free driver : nvidia-430 - third-party free recommended ...
$ cat /proc/driver/nvidia/version NVRM version: NVIDIA UNIX x86_64 Kernel Module 430.26 Tue Jun 4 17:40:52 CDT 2019 GCC version: gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10)
Software requirements
Tutorial: GeForce GTX 1080Ti GPU NVIDIA Driver Installation in Ubuntu 18.04 (2019)
https://www.tensorflow.org/install/gpu#ubuntu_1604_cuda_10
ex> UBUNTU 16.04 , CUDA 10.0
1. Nvidia GPU driver 설치 (재설치/Upgrade)
- NVIDIA® GPU drivers —CUDA 10.0 requires 410.x or higher.
1. 설치된 이전 NVIDIA 드라이버 제거
$ sudo apt-get purge nvidia*
2. repository PPA (Personal Package Archives) 추가후, apt 업데이트
$ sudo add-apt-repository ppa:graphics-drivers/ppa $ sudo apt-get update
3. NVIDIA 관련 드라이버 확인후, 적합한 NVIDIA driver 와 setting확인툴 설치후,
적용을 위해 Reboot . (추천 버전은 nvidia-driver-418)
$ sudo apt-get install nvidia-{TAB} 으로 확인
#$ sudo apt-get install nvidia-430 nvidia-settings
$ sudo apt-get install nvidia-418 nvidia-settings
$ sudo reboot
4. 드라이버 설치 확인하고, 드라이버의 상세정보 확인
$ nvidia-smi Mon Sep 16 15:51:33 2019 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 430.26 Driver Version: 430.26 CUDA Version: 10.2 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 GeForce GTX 1080 Off | 00000000:02:00.0 Off | N/A | | 0% 46C P5 34W / 240W | 0MiB / 8118MiB | 3% Default | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+
cat /proc/driver/nvidia/gpus/{tab}/information
$ cat /proc/driver/nvidia/gpus/0000\\:02\\:00.0/information Model: GeForce GTX 1080 IRQ: 68 GPU UUID: GPU-9efb87f2-c84d-9a34-85ae-8c38b9798eb8 Video BIOS: 86.04.17.00.51 Bus Type: PCIe DMA Size: 47 bits DMA Mask: 0x7fffffffffff Bus Location: 0000:02:00.0 Device Minor: 0 Blacklisted: No
2. CUDA 관련 설치 (CUDA Toolkit/ CUPTI/cuDNN/ TensorRT)
- CUDA® Toolkit —TensorFlow supports CUDA 10.0 (TensorFlow >= 1.13.0)
https://tensorflow.rstudio.com/tools/local_gpu.html
https://www.tensorflow.org/install/install_linux#nvidia_requirements_to_run_tensorflow_with_gpu_support
현재 CUDA 설치 확인
$ dpkg -l | grep cuda-repo-ubuntu1604 ii cuda-repo-ubuntu1604 10.1.243-1 amd64 cuda repository configuration files
Downgrade cuda 10.1 to 10.0
만약 버전이 10.0이 아인면, downgrade.
TensorFlow 최신버전이 아직 CUDA 10.1을 지원하지 않는다.
https://www.tensorflow.org/install/gpu
$ sudo apt-get --purge remove "cuda*" $ sudo reboot
NVIDIA CUDA network repository installation package 설치
$ sudo apt-get install gnupg-curl $ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.0.130-1_amd64.deb $ sudo dpkg -i cuda-repo-ubuntu1804_10.0.130-1_amd64.deb
Add HTTPS support for apt-key
$ sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub $ sudo apt-get update
NVIDIA Machine Learning network repository 설치
$ wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/nvidia-machine-learning-repo-ubuntu1604_1.0.0-1_amd64.deb $ sudo apt install ./nvidia-machine-learning-repo-ubuntu1604_1.0.0-1_amd64.deb $ sudo apt-get update
CUDA Toolkit v10.0 와 CUPTI, cuDNN v7.x 설치
CUDA Toolkit을 설치한다.
CUPTI(CUDA Profiling Tools Interface)는 CUDA Toolkit 설치시 같이 설치되고, Nvidia Driver도 설치된다.
cuDNN(CUDA® Deep Neural Network library)
$ sudo apt-get install --no-install-recommends \\
cuda-10-0 \\
libcudnn7=7.6.2.24-1+cuda10.0 \\
libcudnn7-dev=7.6.2.24-1+cuda10.0
TensorRT 설치
위에서 libcudnn7 가 제대로 설치된 후, TensorRT설치한다.
$ sudo apt-get install --no-install-recommends \\
libnvinfer5=5.1.5-1+cuda10.0 \\
libnvinfer-dev=5.1.5-1+cuda10.0
환경변수 정의 – ENVIRONMENT VARIABLES
- CUDA HOME
- CUDA binary 와 CUPTI 의 binary 경로
- PATH 추가
기본
$ sudo apt-get install --no-install-recommends \\
libnvinfer5=5.1.5-1+cuda10.0 \\
libnvinfer-dev=5.1.5-1+cuda10.0
RStudio Server – MULTI-USER
기본적으로 bash startup file (/etc/profile) 에 정의하여 모든 사용자에 동일하게 적용해야하지만,
RStudio Server를 활용하는 경우, RStudioServer가 R 세션에 대해 system profile 스크립트를 실행하지 않기 때문에, RStudio 설정파일(/etc/rstudio/rserver.conf )에 라이브러리경로를 정의한다.
$ sudo vi /etc/rstudio/rserver.conf # Server Configuration File rsession-ld-library-path=/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64
CUDA Home과 PATH 는 /usr/lib/R/etc/Rprofile.site 활용
$ sudo vi /usr/lib/R/etc/Rprofile.site
Sys.setenv(CUDA_HOME="/usr/local/cuda")
Sys.setenv(PATH=paste(Sys.getenv("PATH"), "/usr/local/cuda/bin", sep = ":"))
* TensorFlow 설치 for all user multi-user installation 섹션 참고
SINGLE-USER
기본적으로 bash startup file ( ~/.profile) 에 정의한고, 적용을 위해 reboot한다.
(물론 ~/.bash_profile 이나 ~/.bash_login 이 있으면 ~/.profile을 읽지 않기 때문에 해당 파일이 있는지도 확인해야한다. ~/.bashrc 류는 Terminal 세션만 적용되지만, ~/.profile은 desktop application(RStudio)까지 읽어서 사용하므로 ~/.profile을 사용한다.)
$ sudo vi ~/.profile # Server Configuration File rsession-ld-library-path=/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64
CUDA Home과 PATH 는 ~/.Rprofile 활용
$ sudo vi ~/.Rprofile
Sys.setenv(CUDA_HOME="/usr/local/cuda")
Sys.setenv(PATH=paste(Sys.getenv("PATH"), "/usr/local/cuda/bin", sep = ":"))
* <참고> 설치 packages
cuda-10-0
cuda-command-line-tools-10-0
cuda-compiler-10-0
cuda-cublas-10-0
cuda-cublas-dev-10-0
cuda-cudart-10-0
cuda-cudart-dev-10-0
cuda-cufft-10-0
cuda-cufft-dev-10-0
cuda-cuobjdump-10-0
cuda-cupti-10-0
cuda-curand-10-0
cuda-curand-dev-10-0
cuda-cusolver-10-0
cuda-cusolver-dev-10-0
cuda-cusparse-10-0
cuda-cusparse-dev-10-0
cuda-demo-suite-10-0
cuda-documentation-10-0
cuda-driver-dev-10-0
cuda-drivers 418.87.00-1
cuda-gdb-10-0
cuda-gpu-library-advisor-10-0
cuda-libraries-10-0
cuda-libraries-dev-10-0
cuda-license-10-0
cuda-memcheck-10-0
cuda-misc-headers-10-0
cuda-npp-10-0
cuda-npp-dev-10-0
cuda-nsight-10-0
cuda-nsight-compute-10-0
cuda-nvcc-10-0
cuda-nvdisasm-10-0
cuda-nvgraph-10-0
cuda-nvgraph-dev-10-0
cuda-nvjpeg-10-0
cuda-nvjpeg-dev-10-0
cuda-nvml-dev-10-0
cuda-nvprof-10-0
cuda-nvprune-10-0
cuda-nvrtc-10-0
cuda-nvrtc-dev-10-0
cuda-nvtx-10-0
cuda-nvvp-10-0
cuda-runtime-10-0
cuda-samples-10-0
cuda-toolkit-10-0
cuda-tools-10-0
cuda-visual-tools-10-0
libcuda1-418
nvidia-418
nvidia-418-dev
nvidia-modprobe
nvidia-opencl-icd-418
nvidia-settings
libcudnn7 7.6.2.24-1+cuda10.0
libcudnn7-dev 7.6.2.24-1+cuda10.0
