zoukankan      html  css  js  c++  java
  • 【转】Ubuntu 16.04安装配置TensorFlow GPU版本

    之前摸爬滚打总是各种坑,今天参考这篇文章终于解决了,甚是鸡冻(≧▽≦)/,电脑不知道怎么的,安装不了16.04,就安装15.10再升级到16.04

    requirements:

    • Ubuntu 16.04
    • python 2.7
    • Flask
    • tensorflow GPU 版本

    安装nvidia driver

    经过不断踩坑的安装,终于google到了靠谱的方法,首先检查你的NVIDIA VGA card model

    sudo lshw -numeric -C display

    NVIDIA-DISPLAYCARD
    可以看到你的显卡信息,比如我的就是 product: GM107M [GeForce GTX 950M] [10DE:139A],然后去NVDIA driver search page搜索你的显卡需要的驱动型号,页面如下:
    gtx-search

    下面是我的电脑对应的驱动版本

    LINUX X64 (AMD64/EM64T) DISPLAY DRIVER
    
    Version:    375.20
    Release Date:   2016.11.18
    Operating System:   Linux 64-bit
    Language:   English (US)
    File Size:  72.37 MB

    从搜索的结果页面看到,我的驱动版本应该是375.20,为了再次确认一遍,你还可以使用这个命令查看你可以使用的驱动:

    ubuntu-drivers devices
     

    结果显示和搜索到的驱动版本一样,推荐也是375

    == /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0 ==
    vendor   : NVIDIA Corporation
    model    : GM107M [GeForce GTX 950M]
    modalias : pci:v000010DEd0000139Asv000017AAsd0000380Bbc03sc02i00
    driver   : nvidia-367 - third-party free
    driver   : nvidia-375 - third-party free recommended
    driver   : nvidia-364 - third-party free
    driver   : nvidia-358 - third-party free
    driver   : xserver-xorg-video-nouveau - distro free builtin
    driver   : nvidia-370 - third-party free
    
    == cpu-microcode.py ==
    driver   : intel-microcode - distro non-free
    

    好了,终于可以安装对应的驱动了,使用以下命令

    version: 375
    sudo apt-get install nvidia-375
    
    //你自己的版本
    //version : xxx
    //sudo apt-get install nvidia-xxx

    什么,安装很慢,找不到包?更换一下软件源,这个自己google怎么更换,最简单的就是图形界面里面找到System->settings->Software&Updates,然后换一下源,比如阿里云或者中科大(我突然不能链接中科大镜像了,真实坑),然后再执行一下命令

    sudo apt-get install mesa-common-dev
    sudo apt-get install freeglut3-dev
     

    安装完成之后,重启电脑,驱动应该就完成了!你可以在dashboard上搜索nvidia,看到像 NVIDIA X Server Settings的东西,就说明安装驱动成功了,接下来就是安装cuda8了
    NVIDIA-DashBoard
    NVIDIA X Server Settings

    安装cuda8

    首先也是去下载cuda toolkit 8.0,可以自己注册一个账号。
    CUDA8
    一定要选择runfile.下载完成之后,执行

    sudo sh cuda_8.0.44_linux.run --override
     

    然后就进入安装过程,开始都是End User License Agreement,你可以CTRL +C 跳过,然后accept,下面就是安装的交互界面,开始的Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 367.48?选择n,因为你已经安装驱动了。

    Using more to view the EULA.
    End User License Agreement
    --------------------------
    
    
    Preface
    -------
    
    The following contains specific license terms and conditions
    for four separate NVIDIA products. By accepting this
    agreement, you agree to comply with all the terms and
    conditions applicable to the specific product(s) included
    herein.
    
    
    NVIDIA CUDA Toolkit
    
    
    Description
    
    The NVIDIA CUDA Toolkit provides command-line and graphical
    tools for building, debugging and optimizing the performance
    of applications accelerated by NVIDIA GPUs, runtime and math
    libraries, and documentation including programming guides,
    user manuals, and API references. The NVIDIA CUDA Toolkit
    License Agreement is available in Chapter 1.
    
    
    Default Install Location of CUDA Toolkit
    
    Windows platform:
    
    Do you accept the previously read EULA?
    accept/decline/quit: accept
    
    Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 367.48?
    (y)es/(n)o/(q)uit: n
    
    Install the CUDA 8.0 Toolkit?
    (y)es/(n)o/(q)uit: y
    
    Enter Toolkit Location
     [ default is /usr/local/cuda-8.0 ]:  
    
    Do you want to install a symbolic link at /usr/local/cuda?
    (y)es/(n)o/(q)uit: y
    
    Install the CUDA 8.0 Samples?
    (y)es/(n)o/(q)uit: y 
    
    Enter CUDA Samples Location
     [ default is /home/kinny ]: 
    
    Installing the CUDA Toolkit in /usr/local/cuda-8.0 ...
    Missing recommended library: libXmu.so
    
    Installing the CUDA Samples in /home/kinny ...
    Copying samples to /home/kinny/NVIDIA_CUDA-8.0_Samples now...
    Finished copying samples.
    
    ===========
    = Summary =
    ===========
    
    Driver:   Not Selected
    Toolkit:  Installed in /usr/local/cuda-8.0
    Samples:  Installed in /home/kinny, but missing recommended libraries
    
    Please make sure that
     -   PATH includes /usr/local/cuda-8.0/bin
     -   LD_LIBRARY_PATH includes /usr/local/cuda-8.0/lib64, or, add /usr/local/cuda-8.0/lib64 to /etc/ld.so.conf and run ldconfig as root
    
    To uninstall the CUDA Toolkit, run the uninstall script in /usr/local/cuda-8.0/bin
    
    Please see CUDA_Installation_Guide_Linux.pdf in /usr/local/cuda-8.0/doc/pdf for detailed information on setting up CUDA.
    
    ***WARNING: Incomplete installation! This installation did not install the CUDA Driver. A driver of version at least 361.00 is required for CUDA 8.0 functionality to work.
    To install the driver using this installer, run the following command, replacing <CudaInstaller> with the name of this run file:
        sudo <CudaInstaller>.run -silent -driver
    
    Logfile is /tmp/cuda_install_17494.log

    配置cuda环境变量

    export PATH="$PATH:/usr/local/cuda-8.0/bin"
    export LD_LIBRARY_PATH="/usr/local/cuda-8.0/lib64"
    
    nvidia-smi
     

    结果出现以下输出,说明配置成功
    nvidia-smi

    安装深度学习库cuDNN

    首先��载cuDNN5.1,直接下载是非常慢的,必须走代理,我用的是终端下载的方法,注意前提是你已经注册为开发者了!

    proxychains wget https://developer.nvidia.com/compute/machine-learning/cudnn/secure/v5.1/prod/8.0/cudnn-8.0-linux-x64-v5.1-tgz
    
    这个会被forbidden,因为没有认证,开发者需要认证才能下载,你先用chrome下载,然后到show all里面去copy真实的下载地址
    proxychains wget http://developer.download.nvidia.com/compute/machine-learning/cudnn/secure/v5.1/prod/8.0/cudnn-8.0-linux-x64-v5.1.tgz?autho=1479703345_7fbb517b03361780b45a2c43277bb9ac&file=cudnn-8.0-linux-x64-v5.1.tgz
    
    这次成功了!!速度还可以!不过下载下来的文件名字有问题,修改成cudnn-8.0-linux-x64-v5.1.tgz就可以了
    
    然后是解压
    tar xvzf cudnn-8.0-linux-x64-v5.1.tgz
    
    然后将库和头文件copy到cuda目录(一定是你自己安装的目录如/usr/local/cuda-8.0),不过正确安装的话,ubuntu一般就会有软链接/usr/local/cuda -> /usr/local/cuda-8.0/
    sudo cp cuda/include/cudnn.h /usr/local/cuda/include
    sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
    sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn*

    安装tensorflow gpu enable python 2.7 版本

    sudo pip install tensorflow-gpu
    export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.11.0-cp27-none-linux_x86_64.whl
    sudo pip install --upgrade $TF_BINARY_URL
    
    
    验证
    $python 
    Python 2.7.12 (default, Jul  1 2016, 15:12:24) 
    [GCC 5.4.0 20160609] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import tensorflow
    I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so locally
    I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so locally
    I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so locally
    I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
    I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so locally
    >>> quit()
    大功告成!
    

    错误

    1.libcudart.so.8.0: cannot open shared object file: No such file or directory

    ======================================================================================

    【如果每次开启都显示此错误,则需要打开变量文件设置变量】

    1. 打开终端并输入:

      sudo gedit ~/.bashrc

    2. 输入用户密码。这时输入的密码是不可见的。

    3. 前面的步骤会打开.bashrc文件,在其末尾添加:

      export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64"
      export CUDA_HOME=/usr/local/cuda
    4. 使其立即生效,在终端执行:

      source ~/.bashrc

      或者重启电脑即可。

    =============================================================================================================

    kinny@kinny-Lenovo-XiaoXin:~/Study/tensorflow-0.11.0rc0/tensorflow/models/image/mnist$ python convolutional.py 
    Traceback (most recent call last):
      File "convolutional.py", line 34, in <module>
        import tensorflow as tf
      File "/usr/local/lib/python2.7/dist-packages/tensorflow/__init__.py", line 23, in <module>
        from tensorflow.python import *
      File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/__init__.py", line 49, in <module>
        from tensorflow.python import pywrap_tensorflow
      File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 28, in <module>
        _pywrap_tensorflow = swig_import_helper()
      File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 24, in swig_import_helper
        _mod = imp.load_module('_pywrap_tensorflow', fp, pathname, description)
    ImportError: libcudart.so.8.0: cannot open shared object file: No such file or directory

    方法是设置环境变量,把以前设置的cuda环境变量改成一下这样,这个是tensorflow官网上要求的环境变量;{}

    export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64"
    export CUDA_HOME=/usr/local/cuda
     

    2.TypeError: run() got an unexpected keyword argument ‘argv’

    Traceback (most recent call last):
      File "convolutional.py", line 339, in <module>
        tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
    TypeError: run() got an unexpected keyword argument 'argv'

    方法是把main里面的argv参数去掉

    使用python 虚拟环境

    使用gpu版本运行mnist例子非常慢,基本卡死在数据下载和读取上了!为了比较gpu和cpu的性能,使用虚拟环境安装了tensorflow的cpu版本;

    sudo apt-get install python-pip python-dev python-virtualenv
    
    mkdir py2virtualenv
    virtualenv --system-site-packages ~/py2virtualenv/tensorflowcpu
    source ~/py2virtualenv/tensorflowcpu/bin/activate
    export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.11.0-cp27-none-linux_x86_64.whl
    pip install --upgrade $TF_BINARY_URL
    

    原来cpu版本数据读取和下载很快!cpu适合做IO和简单逻辑运算和加减,但是gpu不行,gpu不适合做高IO和加减法,但是在做矩阵运算表现十分强悍,我在把mnist数据集下载到本地后,分别使用cpu版本和gpu版本跑tensorflow/tensorflow/models/image/mnist/convolutional.py,结果显示:

    //cpu版本
    Step 8100 (epoch 9.43), 130.6 ms
    Minibatch loss: 1.630, learning rate: 0.006302
    Minibatch error: 0.0%
    Validation error: 0.8%
    平均每 100 次 130.64ms 左右
    
    real  19m5.685s
    user  67m33.720s
    sys 0m12.340s
    
    //gpu版本
    Step 8100 (epoch 9.43), 23.2 ms
    Minibatch loss: 1.634, learning rate: 0.006302
    Minibatch error: 0.0%
    Validation error: 0.9%
    平均每 100 次 23.2ms 左右
    
    real  3m28.296s
    user  2m45.888s
    sys 0m29.064s

    GPU在矩阵密集运算方面完虐cpu,大概是6倍。我的是GTX 950M,不知道现在的GTX 1080M是什么情况。

    Caffe 深度学习入门教程  http://www.linuxidc.com/Linux/2016-11/136774.htm

    Ubuntu 16.04下Matlab2014a+Anaconda2+OpenCV3.1+Caffe安装 http://www.linuxidc.com/Linux/2016-07/132860.htm

    Ubuntu 16.04系统下CUDA7.5配置Caffe教程 http://www.linuxidc.com/Linux/2016-07/132859.htm

    Caffe在Ubuntu 14.04 64bit 下的安装 http://www.linuxidc.com/Linux/2015-07/120449.htm

    深度学习框架Caffe在Ubuntu下编译安装  http://www.linuxidc.com/Linux/2016-07/133225.htm

    Caffe + Ubuntu 14.04 64bit + CUDA 6.5 配置说明  http://www.linuxidc.com/Linux/2015-04/116444.htm

    Ubuntu 16.04上安装Caffe http://www.linuxidc.com/Linux/2016-08/134585.htm

    Caffe配置简明教程 ( Ubuntu 14.04 / CUDA 7.5 / cuDNN 5.1 / OpenCV 3.1 )  http://www.linuxidc.com/Linux/2016-09/135016.htm

    Ubuntu 16.04上安装Caffe(CPU only)  http://www.linuxidc.com/Linux/2016-09/135034.htm

    更多Ubuntu相关信息见Ubuntu 专题页面 http://www.linuxidc.com/topicnews.aspx?tid=2

    本文原文链接地址http://www.linuxidc.com/Linux/2016-11/137561.htm

  • 相关阅读:
    vim 的配置文件
    linux bash 的自动补全
    linux ping 命令
    linux 安装ifconfig
    dos exist 命令
    linux 用户的添加,组的添加,以及查看
    if else 的.bat 文件
    For 的.bat文件
    rmdir 的.bat文件
    dos set 命令
  • 原文地址:https://www.cnblogs.com/xia-Autumn/p/6228911.html
Copyright © 2011-2022 走看看