CUDA 환경 설정 (rstudio)

Published by onesixx on 19-02-0319-02-03

https://tensorflow.rstudio.com/installation/gpu/local_gpu/

UBNUTU 설정

Rstuio Server사용여부 및 Sigle-user multi-user여부에 따라 설정해야 하는 것이 달라진다.

CUDA 라이브러리 설치과정에서
CUDA binaries경로를 PATH 와 LD_LIBRARY_PATH에추가
CUDA_HOME 환경변수를 설정

$ echo $CUDA_HOME
$ echo $LD_LIBRARY_PATH

LD_LIBRARY_PATH수정

sudo vi /etc/rstudio/rserver.conf

rsession-ld-library-path=/usr/local/cuda/lib64

CUDA_HOME 과 PATH 변수 설정

vi /usr/lib/R/etc/Rprofile.site

Sys.setenv(CUDA_HOME=”/usr/local/cuda”)
Sys.setenv(PATH=paste(Sys.getenv(“PATH”), “/usr/local/cuda/bin”, sep=”:”))

GPU사용률

~$➜ nvidia-smi -l 1
~$➜ watch -n 5 nvidia-smi -a --display=utilization

….참고….

sudo service –status-all

sudo service rstudio-server start

sudo service dbus start

> tf$config$experimental$list_physical_devices()
2020-11-02 10:57:07.343212: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-11-02 10:57:07.903418: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties: 
pciBusID: 0000:02:00.0 name: GeForce GTX 1080 computeCapability: 6.1
coreClock: 1.797GHz coreCount: 20 deviceMemorySize: 7.93GiB deviceMemoryBandwidth: 298.32GiB/s
2020-11-02 10:57:07.903872: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-11-02 10:57:07.904109: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10'; dlerror: libcublas.so.10: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/R/lib::/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server
2020-11-02 10:57:07.904311: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10'; dlerror: libcufft.so.10: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/R/lib::/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server
2020-11-02 10:57:07.904508: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10'; dlerror: libcurand.so.10: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/R/lib::/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server
2020-11-02 10:57:07.904707: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10'; dlerror: libcusolver.so.10: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/R/lib::/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server
2020-11-02 10:57:07.904902: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10'; dlerror: libcusparse.so.10: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/R/lib::/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server
2020-11-02 10:57:07.905103: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/R/lib::/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server
2020-11-02 10:57:07.905116: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1592] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
[[1]]
PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')

[[2]]
PhysicalDevice(name='/physical_device:XLA_CPU:0', device_type='XLA_CPU')

[[3]]
PhysicalDevice(name='/physical_device:XLA_GPU:0', device_type='XLA_GPU')

> conda_list()
          name                                                                   python
1  r-miniconda                   /home/oschung_skcc/.local/share/r-miniconda/bin/python
2 r-reticulate /home/oschung_skcc/.local/share/r-miniconda/envs/r-reticulate/bin/python
3     sixxBase     /home/oschung_skcc/.local/share/r-miniconda/envs/sixxBase/bin/python
4       sixxDL       /home/oschung_skcc/.local/share/r-miniconda/envs/sixxDL/bin/python
>   use_backend(backend="tensorflow")
> tensorflow::tf_version()
NULL
> tensorflow::tf_version()
2020-11-04 02:28:19.864661: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer.so.6
2020-11-04 02:28:19.865351: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvrtc.so.10.2: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/R/lib:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:::/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64
2020-11-04 02:28:19.865376: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
[1] ‘2.1’
> tf$config$experimental$list_physical_devices()
2020-11-04 02:28:25.564664: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-11-04 02:28:26.122880: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties: 
pciBusID: 0000:02:00.0 name: GeForce GTX 1080 computeCapability: 6.1
coreClock: 1.797GHz coreCount: 20 deviceMemorySize: 7.93GiB deviceMemoryBandwidth: 298.32GiB/s
2020-11-04 02:28:26.143734: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-11-04 02:28:26.143880: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-11-04 02:28:26.148213: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-11-04 02:28:26.148998: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-11-04 02:28:26.153880: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-11-04 02:28:26.156279: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-11-04 02:28:26.156380: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-11-04 02:28:26.159961: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
[[1]]
PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')

[[2]]
PhysicalDevice(name='/physical_device:XLA_CPU:0', device_type='XLA_CPU')

[[3]]
PhysicalDevice(name='/physical_device:XLA_GPU:0', device_type='XLA_GPU')

[[4]]
PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')

> tf$test$is_gpu_available()
2020-11-04 02:28:31.897573: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-11-04 02:28:31.914118: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2394505000 Hz
2020-11-04 02:28:31.918201: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x56534d095a60 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-11-04 02:28:31.918256: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-11-04 02:28:32.044116: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x56534d0fb290 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-11-04 02:28:32.044179: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce GTX 1080, Compute Capability 6.1
2020-11-04 02:28:32.046312: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties: 
pciBusID: 0000:02:00.0 name: GeForce GTX 1080 computeCapability: 6.1
coreClock: 1.797GHz coreCount: 20 deviceMemorySize: 7.93GiB deviceMemoryBandwidth: 298.32GiB/s
2020-11-04 02:28:32.046423: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-11-04 02:28:32.046466: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-11-04 02:28:32.046515: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-11-04 02:28:32.046559: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-11-04 02:28:32.046602: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-11-04 02:28:32.046644: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-11-04 02:28:32.046688: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-11-04 02:28:32.050342: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-11-04 02:28:32.050440: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-11-04 02:28:32.054336: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1096] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-11-04 02:28:32.054386: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102]      0 
2020-11-04 02:28:32.054415: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] 0:   N 
2020-11-04 02:28:32.058115: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1241] Created TensorFlow device (/device:GPU:0 with 7605 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:02:00.0, compute capability: 6.1)
[1] TRUE

I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA

https://m.blog.naver.com/complusblog/221237740617

W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library ‘libcublas.so.10’; dlerror: libcublas.so.10: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib/R/lib::/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server

sudo ln -s /usr/local/cuda-10.0 /usr/local/cuda

CUDA 환경 설정 (rstudio)

https://tensorflow.rstudio.com/installation/gpu/local_gpu/

LD_LIBRARY_PATH수정

CUDA_HOME 과 PATH 변수 설정

GPU사용률

I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA

onesixx

DeepLearning

epoch, batch size, iteration의 의미

DeepLearning

Kitty

DeepLearning

ligthgbm ex

CUDA 환경 설정 (rstudio)

https://tensorflow.rstudio.com/installation/gpu/local_gpu/

LD_LIBRARY_PATH수정

CUDA_HOME 과 PATH 변수 설정

GPU사용률

I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA

onesixx

Related Posts

DeepLearning

epoch, batch size, iteration의 의미

DeepLearning

Kitty

DeepLearning

ligthgbm ex