搭建一套基于docker的开发训练环境
步骤如下:
一、安装宿主机docker环境
参考该链接:NVidia-Docker2安装与常用命令
二、下载ubuntu18.04的docker基本镜像
dockerhub链接:https://hub.docker.com/_/ubuntu,在命令行使用docker pull ubuntu:18.04下载
三、安装sudo权限
apt-get update,如果不成功,需要更换源:/etc/apt/sources.list
源链接可参考该链接:ubuntu替换国内源(清华,阿里)
注意区分18.04和16.04,本人亲测阿里云可以正常更新,但清华源无法下载。替换方法,由于当前镜像下没有vi,vim,也无法安装通过sudo安装,可以先采用映射文件夹到docker的方式,拷贝一个准备好的sources.list进来。
四、安装显卡驱动
1、NVidia driver:
在docker终端输入ubuntu-drivers devices,查看显示配置信息,正常显示如下:
root@e25881d8e29d:/# ubuntu-drivers devices == /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0 == modalias : pci:v000010DEd00001F36sv0000103Csd0000860Fbc03sc00i00 vendor : NVIDIA Corporation driver : nvidia-driver-450-server - distro non-free driver : nvidia-driver-440-server - distro non-free driver : nvidia-driver-460 - distro non-free recommended driver : nvidia-driver-450 - distro non-free driver : nvidia-driver-418-server - distro non-free driver : xserver-xorg-video-nouveau - distro free builtin == /sys/devices/pci0000:00/0000:00:1d.5/0000:70:00.0 == modalias : pci:v00008086d00002723sv00008086sd00000088bc02sc80i00 vendor : Intel Corporation driver : backport-iwlwifi-dkms - distro free
如果提示
bash: ubutnu-drivers: command not found
,则安装:sudo apt-get install ubuntu-drivers-common
安装显卡驱动:sudo apt-get install nvidia-driver-450
查看驱动版本:cat /proc/driver/nvidia/version
root@e25881d8e29d:/workspace# cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 450.102.04 Tue Dec 29 06:51:23 UTC 2020
GCC version: gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)
2、Cuda安装
ls /dev/nvidia*,显示如下:
/dev/nvidia-modeset /dev/nvidia-uvm /dev/nvidia-uvm-tools /dev/nvidia0 /dev/nvidiactl /dev/nvidia-caps: nvidia-cap1 nvidia-cap2
#install CUDA
sudo dpkg -i cuda-repo-ubuntu1804-10-2-local-10.2.89-440.33.01_1.0-1_amd64.deb
sudo apt-key add /var/cuda-repo-10-2-local-10.2.89-440.33.01/7fa2af80.pub
sudo apt-get update
sudo apt-get -y install cuda
安装完之后,需要添加环境变量
sudo gedit ~/.bashrc
export LD_LIBRARY_PATH=/usr/local/cuda/lib
export PATH=$PATH:/usr/local/cuda/bin
然后,输入以下命令使配置的环境变量立即生效:
source ~/.bashrc
查看当前cuda是否成功安装
nvcc -V
root@e25881d8e29d:/usr/local/cuda-10.2/lib64# nvcc -V nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2019 NVIDIA Corporation Built on Wed_Oct_23_19:24:38_PDT_2019 Cuda compilation tools, release 10.2, V10.2.89
出现以上信息,说明cuda10.2即已安装完成。
五、安装软件包
1、安装miniconda
先下载包,放入docker相应映射目录:Miniconda3-latest-Linux-x86_64
bash Miniconda3-latest-Linux-x86_64,安装过程中需要注意,有一个选项:是否需要每次启动conda?输入no,其它都默认yes就可以了
安装完后,需要设置环境变量: sudo vim ~/.bashrc
在文件最后加上:export PATH="/miniconda/bin":$PATH
source ~/.bashrc 使修改生效
sudo update-alternatives --install /usr/bin/python3 python3 /miniconda3/bin/python3.8 1 设置python3.8为默认python位置
输入python,正常显示即说明配置成功
2、pytorch,torchvision
先下载好安装文件,然后使用pip安装
pip install torch-1.6.0-cp38-cp38-manylinux1_x86_64.whl
安装过程中会同步安装feature(future-0.18.2-py3-none-any.whl),numpy(numpy-1.19.5-cp38-cp38-manylinux2010_x86_64.whl)
pip install torchvision-0.7.0-cp38-cp38-manylinux1_x86_64.whl
安装过程中会同步安装Pillow(Pillow-8.1.0-cp38-cp38-manylinux1_x86_64.whl)
3、使用pycharm连接docker
配置部分可参考该链接:使用docker来配置pycharm开发和训练环境
4、安装opencv
pip install opencv-python
5、安装yacs,skimage
pip install yacs,scikit-image,matplotlib, scipy
6、安装apex
git clone https://github.com/NVIDIA/apex
cd apex
python3 setup.py install
7、安装Caffe
参考该链接:Caffe环境安装