zoukankan      html  css  js  c++  java
  • 编译gpu集群版caffe

    在这个版本安装之前,要先装好opencv,openmpi等。

    下载地址:https://github.com/yjxiong/caffe.git

    我的opencv是2.4.12版本

    编译是用了:

    cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -DCUDA_CUDA_LIBRARY=/usr/local/cuda/lib64/stubs/libcuda.so -D CUDA_ARCH_BIN=5.2 -D CUDA_ARCH_PTX="" -D WITH_CUDA=ON -D WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_OPENGL=ON -D ENABLE_FAST_MATH=1 -D CUDA_FAST_MATH=1 -D WITH_CUBLAS=1 -D WITH_NVCUVID:BOOL="1" .

    caffe的编译是:

    cmake -DUSE_MPI=ON -DMPI_CXX_COMPILER=/data/dog123/openmpi/bin/mpicxx ..

    ----------------------------------------------------------------------------------------

    (还是写完整些比较好)

    到你要存放是目录下,使用命名(git clone https://github.com/yjxiong/caffe.git)下载软件包。

    将Makefile.config.example 另存一份名为Makefile.config

    修改Makefile.config,最终的样子如下:

    ## Refer to http://caffe.berkeleyvision.org/installation.html
    # Contributions simplifying and improving our build system are welcome!
    
    # cuDNN acceleration switch (uncomment to build with cuDNN).
     USE_CUDNN := 1
    
    # CPU-only switch (uncomment to build without GPU support).
    # CPU_ONLY := 1
    
    # To customize your choice of compiler, uncomment and set the following.
    # N.B. the default for Linux is g++ and the default for OSX is clang++
    # CUSTOM_CXX := g++
    
    # CUDA directory contains bin/ and lib/ directories that we need.
    CUDA_DIR := /usr/local/cuda
    # On Ubuntu 14.04, if cuda tools are installed via
    # "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
    # CUDA_DIR := /usr
    
    # CUDA architecture setting: going with all of them.
    # For CUDA < 6.0, comment the *_50 lines for compatibility.
    CUDA_ARCH := -gencode arch=compute_20,code=sm_20 
    		-gencode arch=compute_20,code=sm_21 
    		-gencode arch=compute_30,code=sm_30 
    		-gencode arch=compute_35,code=sm_35 
    		-gencode arch=compute_50,code=sm_50 
    		-gencode arch=compute_50,code=compute_50
    
    # BLAS choice:
    # atlas for ATLAS (default)
    # mkl for MKL
    # open for OpenBlas
    BLAS := atlas
    # Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
    # Leave commented to accept the defaults for your choice of BLAS
    # (which should work)!
    # BLAS_INCLUDE := /path/to/your/blas
    # BLAS_LIB := /path/to/your/blas
    
    # Homebrew puts openblas in a directory that is not on the standard search path
    # BLAS_INCLUDE := $(shell brew --prefix openblas)/include
    # BLAS_LIB := $(shell brew --prefix openblas)/lib
    
    # This is required only if you will compile the matlab interface.
    # MATLAB directory should contain the mex binary in /bin.
     MATLAB_DIR := /usr/local/MATLAB/R2014a
    # MATLAB_DIR := /Applications/MATLAB_R2012b.app
    
    # NOTE: this is required only if you will compile the python interface.
    # We need to be able to find Python.h and numpy/arrayobject.h.
    PYTHON_INCLUDE := /usr/include/python2.7 
    		/usr/lib/python2.7/dist-packages/numpy/core/include
    # Anaconda Python distribution is quite popular. Include path:
    # Verify anaconda location, sometimes it's in root.
    # ANACONDA_HOME := $(HOME)/anaconda
    # PYTHON_INCLUDE := $(ANACONDA_HOME)/include 
    		# $(ANACONDA_HOME)/include/python2.7 
    		# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include 
    
    # We need to be able to find libpythonX.X.so or .dylib.
    PYTHON_LIB := /usr/lib
    # PYTHON_LIB := $(ANACONDA_HOME)/lib
    
    # Homebrew installs numpy in a non standard path (keg only)
    # PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
    # PYTHON_LIB += $(shell brew --prefix numpy)/lib
    
    # Uncomment to support layers written in Python (will link against Python libs)
     WITH_PYTHON_LAYER := 1
    
    # Whatever else you find you need goes here.
    INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
    LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
    
    # If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
    # INCLUDE_DIRS += $(shell brew --prefix)/include
    # LIBRARY_DIRS += $(shell brew --prefix)/lib
    
    # Uncomment to use `pkg-config` to specify OpenCV library paths.
    # (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
    # USE_PKG_CONFIG := 1
    
    BUILD_DIR := build
    DISTRIBUTE_DIR := distribute
    
    # Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
    # DEBUG := 1
    
    # The ID of the GPU that 'make runtest' will use to run unit tests.
    TEST_GPUID := 0
    
    # enable pretty build (comment to see full commands)
    Q ?= @
    

      (红色部分是要核对下的)

    然后在caffe目录下执行如下命令:

    创建build文件夹并进入:

    mkdir build

    cd build

    编译:

    cmake -DUSE_MPI=ON -DMPI_CXX_COMPILER=/data/dog123/openmpi/bin/mpicxx ..

    编译的结果是:

    og@asus:/data/dog123/caffe/build$ cmake -DUSE_MPI=ON -DMPI_CXX_COMPILER=/data/dog123/openmpi/bin/mpicxx ..
    -- The C compiler identification is GNU 4.7.3
    -- The CXX compiler identification is GNU 4.7.3
    -- Check for working C compiler: /usr/bin/cc
    -- Check for working C compiler: /usr/bin/cc -- works
    -- Detecting C compiler ABI info
    -- Detecting C compiler ABI info - done
    -- Check for working CXX compiler: /usr/bin/c++
    -- Check for working CXX compiler: /usr/bin/c++ -- works
    -- Detecting CXX compiler ABI info
    -- Detecting CXX compiler ABI info - done
    -- Boost version: 1.64.0
    -- Found the following Boost libraries:
    --   system
    --   thread
    -- Looking for include file pthread.h
    -- Looking for include file pthread.h - found
    -- Looking for pthread_create
    -- Looking for pthread_create - not found
    -- Looking for pthread_create in pthreads
    -- Looking for pthread_create in pthreads - not found
    -- Looking for pthread_create in pthread
    -- Looking for pthread_create in pthread - found
    -- Found Threads: TRUE  
    -- Found GFlags: /usr/include  
    -- Found gflags  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so)
    -- Found Glog: /usr/include  
    -- Found glog    (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so)
    -- Found PROTOBUF: /usr/lib/x86_64-linux-gnu/libprotobuf.so  
    -- Found PROTOBUF Compiler: /usr/bin/protoc
    -- Found HDF5: /usr/lib/x86_64-linux-gnu/libhdf5_hl.so;/usr/lib/x86_64-linux-gnu/libhdf5.so  
    -- Found LMDB: /usr/include  
    -- Found lmdb    (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so)
    -- Found LevelDB: /usr/include  
    -- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so)
    -- Found Snappy: /usr/include  
    -- Found Snappy  (include: /usr/include, library: /usr/lib/libsnappy.so)
    -- CUDA detected: 8.0
    -- Found cuDNN (include: /usr/local/cuda/include, library: /usr/local/cuda/lib64/libcudnn.so)
    -- Added CUDA NVCC flags for: sm_52
    -- OpenCV found (/usr/local/share/OpenCV)
    -- Found Atlas: /usr/include  
    -- Found Atlas (include: /usr/include, library: /usr/lib/libatlas.so)
    -- Found PythonInterp: /usr/bin/python2.7 (found suitable version "2.7.6", minimum required is "2.7") 
    -- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found suitable version "2.7.6", minimum required is "2.7") 
    -- Found NumPy: /usr/local/lib/python2.7/dist-packages/numpy/core/include (found suitable version "1.12.1", minimum required is "1.7.1") 
    -- NumPy ver. 1.12.1 found (include: /usr/local/lib/python2.7/dist-packages/numpy/core/include)
    -- Boost version: 1.64.0
    -- Found the following Boost libraries:
    --   python
    -- Found Doxygen: /usr/bin/doxygen (found version "1.8.6") 
    -- Found MPI_C: /data/dog123/openmpi/lib/libmpi.so  
    -- Found MPI_CXX: /data/dog123/openmpi/lib/libmpi.so  
    -- Detected Doxygen OUTPUT_DIRECTORY: ./doxygen/
    -- Found Git: /usr/bin/git (found version "1.9.1") 
    -- 
    -- ******************* Caffe Configuration Summary *******************
    -- General:
    --   Version           :   <TODO> (Caffe doesn't declare its version in headers)
    --   Git               :   v0.9999-1628-gfd7458e
    --   System            :   Linux
    --   C++ compiler      :   /usr/bin/c++
    --   Release CXX flags :   -O3 -DNDEBUG -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
    --   Debug CXX flags   :   -g -fPIC -Wall -Wno-sign-compare -Wno-uninitialized
    --   Build type        :   Release
    -- 
    --   BUILD_SHARED_LIBS :   ON
    --   BUILD_python      :   ON
    --   BUILD_matlab      :   OFF
    --   BUILD_docs        :   ON
    --   CPU_ONLY          :   OFF
    -- 
    -- Dependencies:
    --   BLAS              :   Yes (Atlas)
    --   Boost             :   Yes (ver. 1.64)
    --   glog              :   Yes
    --   gflags            :   Yes
    --   protobuf          :   Yes (ver. 2.5.0)
    --   lmdb              :   Yes (ver. 0.9.10)
    --   Snappy            :   Yes (ver. 1.1.0)
    --   LevelDB           :   Yes (ver. 1.15)
    --   OpenCV            :   Yes (ver. 2.4.12)
    --   CUDA              :   Yes (ver. 8.0)
    -- 
    -- NVIDIA CUDA:
    --   Target GPU(s)     :   Auto
    --   GPU arch(s)       :   sm_52
    --   cuDNN             :   Yes
    -- 
    -- Python:
    --   Interpreter       :   /usr/bin/python2.7 (ver. 2.7.6)
    --   Libraries         :   /usr/lib/x86_64-linux-gnu/libpython2.7.so (ver 2.7.6)
    --   NumPy             :   /usr/local/lib/python2.7/dist-packages/numpy/core/include (ver 1.12.1)
    -- 
    -- Documentaion:
    --   Doxygen           :   /usr/bin/doxygen (1.8.6)
    --   config_file       :   /data/dog123/caffe/.Doxyfile
    -- 
    -- Install:
    --   Install path      :   /data/dog123/caffe/build/install
    -- 
    -- Configuring done
    -- Generating done
    -- Build files have been written to: /data/dog123/caffe/build
    dog@asus:/data/dog123/caffe/build$ 
    

      

    安装:

    make all -j8 (j8 是为了加快安装速度,可以去掉)

    sudo make install (注意 sudo权限)

    最后就是测试:

    make runtest (我这里有2个test不过,但是我还没找到原因(因为没看到错误在哪,都在输出的前面覆盖了),因为装好多遍都有2个不过,所以先将就。也就是这样,感觉自己跟一个炸弹绑在一起,我不知道它什么时候会不爽然后炸我1炸,哈哈哈哈哈)

    最后就是python和matlab接口。

    这2者都是caffe装之前就装好了的。

    编译python接口:

    添加环境变量:

    vi ~/.bashrc

    写入:

    export PYTHONPATH=/your/path/caffe/python:$PYTHONPATH

    保存,退出,执行sourc使文件生效:
    source ~/.bashrc

    接着在caffe目录下:

    sudo make pycaffe

    如果报错,点这里。一般再执行一遍上面命令即可。

    最后就是:输入命令:

    python

    import caffe 

    没报错就是成功了。

    编译matlab接口:

    同理,在~/.bashrc中添加环境变量:

    export PATH=$PATH:/usr/local/MATLAB/R2014a/bin

    然后在caffe目录下执行:

    sudo make matcaffe

    没报错的话,就用下面命令测试下:

    make matcaffe

    如果报错,就点这里

    嗯,就这些。

  • 相关阅读:
    vim 多窗口编辑
    opengl笔记——旋转,一段代码的理解
    用条件变量实现事件等待器的正确与错误做法
    opengl笔记—— glMultMatrixf() 区别 glLoadMatrixf()
    Mule与其它web应用服务器的区别
    海量数据相似度计算之simhash短文本查找
    关于协方差矩阵的理解
    C++ STL中的常用容器浅谈
    唐-诗:《枫桥夜泊》
    唐-诗:《肚桑干》
  • 原文地址:https://www.cnblogs.com/beihaidao/p/6866342.html
Copyright © 2011-2022 走看看