2020系统综合实践 第7次实践作业 11组

目录

• 1.在树莓派中安装opencv库
  • 1.1 安装依赖
  • 1.2 下载OpenCV源码
  • 1.3 安装pip
  • 1.4 安装Python虚拟机
  • 1.5 编译OpenCV
  • 1.6 安装OpenCV
• 2.使用opencv和python控制树莓派的摄像头
• 3.利用树莓派的摄像头实现人脸识别
  • facerec_on_raspberry_pi.py
  • facerec_from_webcam_faster.py
• 4.结合微服务的进阶任务
  • 部署opencv的docker容器
  • 在容器中跑通示例代码facerec_on_raspberry_pi.py
  • 选做:在容器中跑通facerec_from_webcam_faster.py
• 5.实验记录
  • 问题解决
  • 小组成员及分工
  • 在线协作

1.在树莓派中安装opencv库

1.1 安装依赖

sudo apt-get update && sudo apt-get upgrade
sudo apt-get install build-essential cmake pkg-config
sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev
sudo apt-get install libgtk2.0-dev libgtk-3-dev
sudo apt-get install libatlas-base-dev gfortran
sudo apt-get install python2.7-dev python3-dev

1.2 下载OpenCV源码

• 查看老师博客时发现旧版本问题较多，使用较新版本

cd ~
wget -O opencv.zip https://github.com/Itseez/opencv/archive/4.1.2.zip
unzip opencv.zip
wget -O opencv_contrib.zip https://github.com/Itseez/opencv_contrib/archive/4.1.2.zip
unzip opencv_contrib.zip

1.3 安装pip

wget https://bootstrap.pypa.io/get-pip.py
sudo python get-pip.py
sudo python3 get-pip.py

1.4 安装Python虚拟机

sudo pip install virtualenv virtualenvwrapper
sudo rm -rf ~/.cache/pip

• 配置~/.profile

sudo nano ~/.profile
export WORKON_HOME=$HOME/.virtualenvs 
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3 
source /usr/local/bin/virtualenvwrapper.sh 
export VIRTUALENVWRAPPER_VIRTUALENV=/usr/local/bin/virtualenv
export VIRTUALENVWRAPPER_ENV_BIN_DIR=bin 

• 使配置生效

source ~/.profile

• 使用python3安装虚拟机

mkvirtualenv cv -p python3

• 使配置生效并进入虚拟机，每次重新进入虚拟机都要键入

source ~/.profile && workon cv

• 安装numpy

pip install numpy

1.5 编译OpenCV

cd ~/opencv-4.1.2/
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE 
    -D CMAKE_INSTALL_PREFIX=/usr/local 
    -D INSTALL_PYTHON_EXAMPLES=ON 
    -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-4.1.2/modules 
    -D BUILD_EXAMPLES=ON ..

• 查看版本

• 编译前增大交换空间CONF_SWAPSIZE=1024

sudo nano /etc/dphys-swapfile #虚拟机中sudo才可以修改

• 开始编译

sudo /etc/init.d/dphys-swapfile stop 
sudo /etc/init.d/dphys-swapfile start
make -j4 #开始编译

1.6 安装OpenCV

sudo make install
sudo ldconfig

• 查看

ls -l /usr/local/lib/python2.7/dist-packages/
ls -l /usr/local/lib/python3.7/site-packages/
cd ~/.virtualenvs/cv/lib/python3.7/site-packages/ 
ls
ln -s /usr/local/lib/python3.7/site-packages/cv2 cv2

• 验证安装结果

source ~/.profile 
workon cv 
python 

---

2.使用opencv和python控制树莓派的摄像头

参考教程：还是可以参考 Adrian Rosebrock的 Accessing the Raspberry Pi Camera with OpenCV and Python；跑通教程的示例代码（有可能要调整里面的参数）

• 激活cv虚拟环境

source ~/.profile
workon cv

• 安装picamera

pip install "picamera[array]"

• 使用Python和OpenCV访问Raspberry Pi的单个映像
  打开一个新文件，命名test_image.py，然后插入以下示例代码：

# import the necessary packages
from picamera.array import PiRGBArray
from picamera import PiCamera
import time
import cv2
# initialize the camera and grab a reference to the raw camera capture
camera = PiCamera()
rawCapture = PiRGBArray(camera)
# allow the camera to warmup
time.sleep(0.1)
# grab an image from the camera
camera.capture(rawCapture, format="bgr")
image = rawCapture.array
# display the image on screen and wait for a keypress
cv2.imshow("Image", image)
cv2.waitKey(0)

• 运行py文件

python test_image.py

• 使用Python和OpenCV访问Raspberry Pi的视频流
  打开一个新文件，命名test_video.py，然后插入以下示例代码：

# import the necessary packages
from picamera.array import PiRGBArray
from picamera import PiCamera
import time
import cv2
# initialize the camera and grab a reference to the raw camera capture
camera = PiCamera()
camera.resolution = (640, 480)
camera.framerate = 32
rawCapture = PiRGBArray(camera, size=(640, 480))
# allow the camera to warmup
time.sleep(0.1)
# capture frames from the camera
for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
	# grab the raw NumPy array representing the image, then initialize the timestamp
	# and occupied/unoccupied text
	image = frame.array
	# show the frame
	cv2.imshow("Frame", image)
	key = cv2.waitKey(1) & 0xFF
	# clear the stream in preparation for the next frame
	rawCapture.truncate(0)
	# if the `q` key was pressed, break from the loop
	if key == ord("q"):
		break

• 运行py文件

python test_video.py

---

3.利用树莓派的摄像头实现人脸识别

人脸识别有开源的 python库face_recognition，这当中有很多示例代码，要求跑通 face_recognition的示例代码 facerec_on_raspberry_pi.py以及 facerec_from_webcam_faster.py

• 安装所需库

pip install dlib
pip install face_recognition
pip install numpy

• 文件夹内放入相关文件

facerec_on_raspberry_pi.py

import face_recognition
import picamera
import numpy as np

# Get a reference to the Raspberry Pi camera.
# If this fails, make sure you have a camera connected to the RPi and that you
# enabled your camera in raspi-config and rebooted first.
camera = picamera.PiCamera()
camera.resolution = (320, 240)
output = np.empty((240, 320, 3), dtype=np.uint8)

# Load a sample picture and learn how to recognize it.
print("Loading known face image(s)")
obama_image = face_recognition.load_image_file("obama_small.jpg")
obama_face_encoding = face_recognition.face_encodings(obama_image)[0]

# Initialize some variables
face_locations = []
face_encodings = []

while True:
    print("Capturing image.")
    # Grab a single frame of video from the RPi camera as a numpy array
    camera.capture(output, format="rgb")

    # Find all the faces and face encodings in the current frame of video
    face_locations = face_recognition.face_locations(output)
    print("Found {} faces in image.".format(len(face_locations)))
    face_encodings = face_recognition.face_encodings(output, face_locations)

    # Loop over each face found in the frame to see if it's someone we know.
    for face_encoding in face_encodings:
        # See if the face is a match for the known face(s)
        match = face_recognition.compare_faces([obama_face_encoding], face_encoding)
        name = "<Unknown Person>"

        if match[0]:
            name = "Barack Obama"

        print("I see someone named {}!".format(name))

• 运行代码，将摄像头对准照片可以看到识别成功

facerec_from_webcam_faster.py

import face_recognition
import cv2
import numpy as np

# Get a reference to webcam #0 (the default one)
video_capture = cv2.VideoCapture(0)

# Load a sample picture and learn how to recognize it.
obama_image = face_recognition.load_image_file("obama.jpg")
obama_face_encoding = face_recognition.face_encodings(obama_image)[0]

# Load a second sample picture and learn how to recognize it.
biden_image = face_recognition.load_image_file("biden.jpg")
biden_face_encoding = face_recognition.face_encodings(biden_image)[0]

# Create arrays of known face encodings and their names
known_face_encodings = [
    obama_face_encoding,
    biden_face_encoding
]
known_face_names = [
    "Barack Obama",
    "Joe Biden"
]

# Initialize some variables
face_locations = []
face_encodings = []
face_names = []
process_this_frame = True

while True:
    # Grab a single frame of video
    ret, frame = video_capture.read()

    # Resize frame of video to 1/4 size for faster face recognition processing
    small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)

    # Convert the image from BGR color (which OpenCV uses) to RGB color (which face_recognition uses)
    rgb_small_frame = small_frame[:, :, ::-1]

    # Only process every other frame of video to save time
    if process_this_frame:
        # Find all the faces and face encodings in the current frame of video
        face_locations = face_recognition.face_locations(rgb_small_frame)
        face_encodings = face_recognition.face_encodings(rgb_small_frame, face_locations)

        face_names = []
        for face_encoding in face_encodings:
            # See if the face is a match for the known face(s)
            matches = face_recognition.compare_faces(known_face_encodings, face_encoding)
            name = "Unknown"

            # # If a match was found in known_face_encodings, just use the first one.
            # if True in matches:
            #     first_match_index = matches.index(True)
            #     name = known_face_names[first_match_index]

            # Or instead, use the known face with the smallest distance to the new face
            face_distances = face_recognition.face_distance(known_face_encodings, face_encoding)
            best_match_index = np.argmin(face_distances)
            if matches[best_match_index]:
                name = known_face_names[best_match_index]

            face_names.append(name)

    process_this_frame = not process_this_frame


    # Display the results
    for (top, right, bottom, left), name in zip(face_locations, face_names):
        # Scale back up face locations since the frame we detected in was scaled to 1/4 size
        top *= 4
        right *= 4
        bottom *= 4
        left *= 4

        # Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)

        # Draw a label with a name below the face
        cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (0, 0, 255), cv2.FILLED)
        font = cv2.FONT_HERSHEY_DUPLEX
        cv2.putText(frame, name, (left + 6, bottom - 6), font, 1.0, (255, 255, 255), 1)

    # Display the resulting image
    cv2.imshow('Video', frame)

    # Hit 'q' on the keyboard to quit!
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release handle to the webcam
video_capture.release()
cv2.destroyAllWindows()

• 识别成功

---

4.结合微服务的进阶任务

使用微服务，部署 opencv的 docker容器（要能够支持 arm），并在 opencv的 docker容器中跑通（3）的示例代码 facerec_on_raspberry_pi.py
选做：在 opencv的 docker容器中跑通步骤（3）的示例代码facerec_from_webcam_faster.py

部署opencv的docker容器

• 脚本安装docker

sudo curl -sSL https://get.docker.com | sh

• 加用户到docker组，重新登陆生效

sudo usermod -aG docker pi
exit && ssh pi@raspiberry

• 查看版本

docker --version

• 配置docker的镜像加速

sudo nano /etc/docker/daemon.json

• 重启生效

service docker restart

• 拉取arm可用的docker镜像

docker pull sixsq/opencv-python

• 进入容器并安装所需库

docker run -it sixsq/opencv-python /bin/bash
pip install "picamera[array]" dlib face_recognition

• 安装成功后退出容器并commit

docker commit [22ee2e36d2e2] face_recognition_opencv

• 建立dockerfile目录

• Dockerfile

FROM face_recognition_opencv
RUN mkdir /myapp
WORKDIR /myapp
COPY myapp .

• build镜像

docker build -t ex7_opencv .

在容器中跑通示例代码facerec_on_raspberry_pi.py

• 进入容器运行代码

docker run -it --device=/dev/vchiq --device=/dev/video0 --name ex7_opencv ex7_opencv
python3 facerec_on_raspberry_pi.py

选做:在容器中跑通facerec_from_webcam_faster.py

• 在Windows系统中安装Xming和Putty
• 检查树莓派的ssh配置中的X11是否开启

cat /etc/ssh/sshd_config

• 打开putty，connection->SSH->Auth->X11，勾起Enable X11 forwarding

• 使用Putty的ssh登录树莓派查看DISPLAY环境变量值

printenv

• 编写run.sh

#sudo apt-get install x11-xserver-utils
xhost +
sudo docker run -it --rm 
	-v /tmp/.X11-unix:/tmp/.X11-unix 
	-e DISPLAY=$DISPLAY 
	-e QT_X11_NO_MITSHM=1 
  	--device=/dev/vchiq 
	--device=/dev/video0 
	--name ex7_opencvgui3 
	facerc_gui 
	python3 facerec_from_webcam_faster.py

• 打开终端运行run.sh

sh run.sh

---

5.实验记录

记录遇到的问题和解决方法，提供小组成员名单、分工、各自贡献以及在线协作的图片

问题解决

问题① 配置~/.profile时报错

解决： 参考此篇

问题② 安装numpy时报错

解决： 网络问题，多尝试几次就成功了

问题③ 编译OpenCV时-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-4.1.2/modules 里版本为3.3.0，然后报错

解决： 删除build文件，修改版本重来一次

问题④ pip安装face_recognition超时

解决： 下载whl，用winscp导入到树莓派进行离线安装

• winscp登陆：
  
• 离线安装
  

问题⑤ 运行facerec_from_webcam_faster.py报错

解决： 是由于之前运行第一个例子后Ctrl+Z关闭了摄像头

问题⑥ 在容器内pip安装face_recognition再次超时
解决： 同样离线安装，但要先拷贝whl到容器中：

问题⑦ 使用Putty的ssh登录树莓派后找不到DISPLAY环境变量值
解决： 参考

小组成员及分工

姓名	学号	分工
叶艳玲	031702208	实际操作 问题解决
王星雨	031702212	博客撰写 问题解决
李享	031702509	资料整理 问题解决

在线协作

通过屏幕分享的方式直播操作，共同解决问题