zoukankan      html  css  js  c++  java
  • [深度学习] centos7上搭建基于Anaconda3的caffe+pycaffe环境(python3.6)

    本文记录从零开始在CentOS7.x系统上搭建Caffe深度学习平台,并配置pycaffe环境。(由于在虚拟机上搭建,所以为CPU_ONLY模式)

    1.选择CentOS7 mini版镜像安装虚拟机

    镜像版本:CentOS-7-x86_64-Minimal-1511.iso

    安装好虚拟机后,配置网卡,本文采用桥接模式接入局域网,能够正常访问互联网。

    2.更新yum源

    安装wget、net-tools等常用工具:

    yum install wget net-tools -y

    更换国内yum源:

    wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
    yum clean all
    yum makecache
    yum update -y

    安装epel-release源:

    yum install epel-release -y

    3.安装依赖

    按照caffe官方文档,安装依赖包:

    yum install protobuf-devel leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel -y
    yum install gflags-devel glog-devel lmdb-devel -y
    yum install openblas-devel atlas-devel -y

    安装其他需要的工具包:

    yum install git gcc gcc-c++ bzip2 unzip -y

    4.安装Anaconda3

    从Anaconda Install Archive中下载:Anaconda3-5.2.0-Linux-x86_64.sh(对应python3.6)

    安装Anaconda,安装目录在/usr/local/anaconda3

    bash Anaconda3-5.2.0-Linux-x86_64.sh

    根据提示,依次Enter、yes、输入安装位置,等待安装完成。结束不选择不添加.bashrc,不安装VSCode。

    使用/usr/local/anaconda3/bin/pip安装protobuf==3.6.0和protobuf-py3

    cd /usr/local/anaconda3/bin
    ./pip install protobuf==3.6.0
    ./pip install protobuf-py3

    5.下载caffe

    选择从github上下载caffe,本文下载的是caffe的zip格式包,名为master,解压:

    unzip ~/master

    进入caffe-master目录:

    cd ~/caffe-master

    将配置文件复制一份:

    cp Makefile.config.example Makefile.config

    修改配置文件Makefile.config:

    CPU_ONLY := 1
    BLAS := open
    BLAS_INCLUDE := /usr/include/openblas ANACONDA_HOME := /usr/local/anaconda3 PYTHON_INCLUDE := $(ANACONDA_HOME)/include $(ANACONDA_HOME)/include/python3.6m $(ANACONDA_HOME)/lib/python3.6/site-packages/numpy/core/include PYTHON_LIBRARIES := boost_python36 python3.6m PYTHON_LIB := $(ANACONDA_HOME)/lib

    6.添加环境变量

    将各个动态库加入环境变量:

    vi /etc/profile

    在末尾添加:

    LD_LIBRARY_PATH=/usr/lib:/usr/lib64:/lib:/lib64:/usr/local/lib
    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/anaconda3/lib

    生效:

    source /etc/profile

    7.编译caffe

    进入~/caffe-master目录:

    make all -j8
    make test
    make runtest

    8.编译libboost_python36.so(编译pycaffe时依赖)

    下载 boost_1_70_0.tar.gz载/root下。

    要编译libboost_python36版本,需要将/usr/bin下的python软连接指向Anaconda3中的python

    cd /usr/bin
    mv python python.bak
    ln -s /usr/local/anaconda3/bin/python3.6 python

    注意,此时尽量不要使用yum,因为yum默认使用python2.7,修改了python软连接会导致报错,解决方法如下:

    vi /usr/bin/yum

    将第一行的#!/usr/bin/python 修改为:

    #!/usr/bin/python2.7

    同样修改/usr/libexec/urlgrabber-ext-down文件。

    解压boost压缩包:

    tar zxvf ~/boost_1_70_0.tar.gz

    进入解压后的boost_1_70_0目录:

    ./bootstrap.sh --with-python=/usr/local/anaconda3/bin/python3 --with-python-version=3.6 --with-python-root=/usr/local/anaconda3/lib/python3.6
    ./b2 cflags='-fPIC' cxxflags='-fPIC' --with-python include="/usr/local/anaconda3/include/python3.6m/"
    ./b2 install

    9.编译pycaffe

    进入~/caffe-master目录

    make pycaffe

    10.添加环境变量

    vi /etc/profile

    添加:

    export PYTHONPATH=/root/caffe-master/python:$PYTHONPATH
    export PYTHONPATH=/usr/local/anaconda3/lib/python3.6/site-packages:$PYTHONPATH

    11.python导入caffe

    运行python3.6.5

    import caffe

    此时会报错:

    >>> import caffe
    Failed to include caffe_pb2, things might go wrong!
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/root/caffe-master/python/caffe/__init__.py", line 4, in <module>
        from .proto.caffe_pb2 import TRAIN, TEST
      File "/root/caffe-master/python/caffe/proto/caffe_pb2.py", line 4, in <module>
        from google.protobuf.internal import enum_type_wrapper
    ModuleNotFoundError: No module named 'google'

    这是需要更新protobuf的版本,从2.5.0版本更新为2.6.1。

    12.升级protobuf

    下载protobuf-2.6.1.tar.gz在/root下

    卸载protobuf-2.5.0版本:

    yum remove protobuf -y

    这里的yum可能会报错:

    [root@leocaffe lib]# yum remove protobuf
      File "/usr/bin/yum", line 30
        except KeyboardInterrupt, e:
                                ^
    SyntaxError: invalid syntax

    这是因为我们将python软连接指向了python3.6,我们可以使用前面所讲的解决错误的方法,修改/usr/bin/yum和/usr/libexec/urlgrabber-ext-down文件(参照8.编译libboost_python36.so

    解压:

    cd ~
    tar zxvf ~/protobuf-2.6.1.tar.gz

    执行:

    cd /usr/local
    mkdir protobuf
    cd ~/protobuf-2.6.1
    ./configure --prefix=/usr/local/protobuf

    然后编译:

    cd ~/protobuf-2.6.1
    make
    make install

    13.添加protobuf环境变量

    vi /etc/profile

    添加以下内容:

    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/protobuf/lib/
    export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/protobuf/lib/
    export PATH=$PATH:/usr/local/protobuf/bin/
    export C_INCLUDE_PATH=$C_INCLUDE_PATH:/usr/local/protobuf/include/
    export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/usr/local/protobuf/include/
    export PKG_CONFIG_PATH=/usr/local/protobuf/lib/pkgconfig/

    检查以下安装是否成功:

    protoc --version
    [root@leocaffe lib]# protoc --version
    libprotoc 2.6.1

    打印版本号说明安装成功。

    14.python再次导入caffe

     为保险,可以重新对caffe进行编译:

    cd ~/caffe-master
    make clean
    make all -j8
    make pycaffe

    然后导入caffe:

    import caffe

    成功:

    [root@leocaffe bin]# ./python
    Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56) 
    [GCC 7.2.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import caffe
    >>> 

    总结:

    从本文可以看出,部署caffe平台坑还是挺多,主要集中在所依赖的动态库,环境变量以及软件版本问题。特别注意boost的编译安装、protobuf的版本更新、以及profile环境变量的修改。

  • 相关阅读:
    mybatis 使用缓存策略
    mybatis 使用事务处理
    mybatis 使用接口绑定
    mybatis 配置文件全解
    mybatis mapper映射文件全解
    mybatis中使用log4j
    初次使用Mybatis
    Servlet 实现文件上传与下载
    log4j v2版本的配置和使用
    Servlet 转发请求与重定向,以及路径问题
  • 原文地址:https://www.cnblogs.com/leokale-zz/p/11643822.html
Copyright © 2011-2022 走看看