Shakerato

[Tensorflow]from tensorflow.python.client import device_libprint(device_lib.list_local_devices()) [Keras]from keras import backend as KK.tensorflow_backend._get_available_gpus() [Pytorch]import torchtorch.cuda.get_device_name(0) # number of gpu (cuda:0)torch.cuda.get_device_name(1) # number of gpu (cuda:1) Be careful with the above codes! The code for checking device(GPU) takes your GPU memory a..

Limited GPU Memory GPU usually has lesser device memory than host memoryThe latest high-end GPU (such as NVIDIA GPU P100)12–16 GB device memoryHost system memory256GBTrend for deep learning models is to have a “deeper and wider” architectureEspecially, RNN needs a lot of memory 1. First Solution: distributed Deep LearningSource: M. Cho et al., "PowerAI DDL", 2017PowerAI DDL provides a unified in..