nginx和Docker的打怪升级

需求问题

最近做的一个项目有个需求是http与webscoket服务共存在一个nginx.conf文件中，最开始做的配置如下：

upstream app_server {
    # fail_timeout=0 means we always retry an upstream even if it failed
    # to return a good HTTP response

    # for UNIX domain socket setups
    # server unix:/tmp/gunicorn.sock fail_timeout=0;
    # for a TCP configuration
    server 10.6.14.200:8000 fail_timeout=0;
}

server {
    listen 80;
    # gzip config
    gzip on;
    gzip_min_length 1k;
    gzip_comp_level 9;
    gzip_types text/plain text/css text/javascript application/json application/javascript application/x-javascript application/xml;
    gzip_vary on;
    gzip_disable "MSIE [1-6]\.";

    root /usr/share/nginx/html;

    location / {
        try_files $uri $uri/ /index.html;
    }

    location /websocket {
        proxy_pass http://app_server;

        proxy_http_version 1.1;
        proxy_read_timeout 360s;   
        proxy_redirect off;   
        proxy_set_header Upgrade $http_upgrade; 
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host:$server_port;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header REMOTE-HOST $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

以上配置，http 协议的请求是没有问题的，但是前端 websocket无法建立连接, 也不知道哪里出了问题。

优化策略: 既支持http又支持 ws 的配置

通过nginx官方关于WebSocket的配置得知，可以自定义变量。故配置如下，就可以做到既支持 ws 请求，又支持 http请求。

upstream app_server {
    # fail_timeout=0 means we always retry an upstream even if it failed
    # to return a good HTTP response

    # for UNIX domain socket setups
    # server unix:/tmp/gunicorn.sock fail_timeout=0;
    # for a TCP configuration
    server 10.6.14.200:8000 fail_timeout=0;
}

map $http_upgrade $connection_upgrade {
        default          keep-alive;  # 默认为keep-alive 可以支持 一般http请求
        'websocket'      upgrade;  # 如果为websocket 则为 upgrade 可升级的。
    }

server {
    listen 80;
    # gzip config
    gzip on;
    gzip_min_length 1k;
    gzip_comp_level 9;
    gzip_types text/plain text/css text/javascript application/json application/javascript application/x-javascript application/xml;
    gzip_vary on;
    gzip_disable "MSIE [1-6]\.";

    root /usr/share/nginx/html;

    location / {
        try_files $uri $uri/ /index.html;
    }

    location /websocket {
        proxy_pass http://app_server;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade; # 此处配置 上面定义的变量
        proxy_set_header Connection $connection_upgrade;
    }
}

优化再升级：通过环境变量配置后端host和port，启动时动态连接后端

正常情况下，nginx是不支持直接读取环境变量的，官方说明如下：

Using environment variables in nginx configuration:

Out-of-the-box, Nginx doesn't support using environment variables inside most configuration blocks.

But envsubst may be used as a workaround if you need to generate your nginx configuration dynamically before nginx starts.

envsubst这个命令可以进行传参动态生成模板，所以算是曲线读取环境变量

# nginx.conf.template
upstream app_server {
    # fail_timeout=0 means we always retry an upstream even if it failed
    # to return a good HTTP response

    # for UNIX domain socket setups
    # server unix:/tmp/gunicorn.sock fail_timeout=0;
    # for a TCP configuration
    server ${BACKEND_HOST}:${BACKEND_PORT} fail_timeout=0;
}

map $http_upgrade $connection_upgrade {
        default          keep-alive;  # 默认为keep-alive 可以支持 一般http请求
        'websocket'      upgrade;  # 如果为websocket 则为 upgrade 可升级的。
    }

server {
    listen 80;
    # gzip config
    gzip on;
    gzip_min_length 1k;
    gzip_comp_level 9;
    gzip_types text/plain text/css text/javascript application/json application/javascript application/x-javascript application/xml;
    gzip_vary on;
    gzip_disable "MSIE [1-6]\.";

    root /usr/share/nginx/html;

    location / {
        try_files $uri $uri/ /index.html;
    }

    location /websocket {
        proxy_pass http://app_server;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade; # 此处配置 上面定义的变量
        proxy_set_header Connection $connection_upgrade;
    }
}

通过设置环境变量BACKEND_HOST和BACKEND_PORT并执行envsubst '${BACKEND_HOST} ${BACKEND_PORT}' < nginx.conf.template > default.conf即可生成带有环境变量的配置文件，然后再启动nginx。

以上配置可以在打包镜像时进行，所以前端目录树如下：

├── dist
├── ./docker-entrypoint.sh
├── ./Dockerfile
└── ./nginx.conf.template

# docker-entrypoint.sh

#!/usr/bin/env bash
set -eu

# shellcheck disable=SC2016
envsubst '${BACKEND_HOST} ${BACKEND_PORT}' < /etc/nginx/conf.d/default.conf.template > /etc/nginx/conf.d/default.conf

nginx -g "daemon off;"

# Dockerfile

FROM nginx:latest

COPY nginx.conf.template /etc/nginx/conf.d/default.conf.template

COPY docker-entrypoint.sh /

COPY dist /usr/share/nginx/html

EXPOSE 80

CMD ["/docker-entrypoint.sh"]

# nginx.conf.template

upstream app_server {
    # fail_timeout=0 means we always retry an upstream even if it failed
    # to return a good HTTP response

    # for UNIX domain socket setups
    # server unix:/tmp/gunicorn.sock fail_timeout=0;
    # for a TCP configuration
    server ${BACKEND_HOST}:${BACKEND_PORT} fail_timeout=0;
}

map $http_upgrade $connection_upgrade {
        default          keep-alive;  # 默认为keep-alive 可以支持 一般http请求
        'websocket'      upgrade;  # 如果为websocket 则为 upgrade 可升级的。
    }

server {
    listen 80;
    # gzip config
    gzip on;
    gzip_min_length 1k;
    gzip_comp_level 9;
    gzip_types text/plain text/css text/javascript application/json application/javascript application/x-javascript application/xml;
    gzip_vary on;
    gzip_disable "MSIE [1-6]\.";

    root /usr/share/nginx/html;

    location / {
        try_files $uri $uri/ /index.html;
    }

    location /websocket {
        proxy_pass http://app_server;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade; # 此处配置 上面定义的变量
        proxy_set_header Connection $connection_upgrade;
    }
}

• dist目录即为前端项目打包后的文件目录

• 构建镜像命令为docker build -t iamgeName:Tag[] .

• 运行镜像命令为docker run -d --restart=always -p 10086:80 -e BACKEND_HOST=10.6.14.200 -e BACKEND_PORT=8000 --name containerName imageName:Tag[]

以上内容一般适用于单镜像运行，对于使用docker-compose或者k8s可以直接通过内部机制进行访问会更加方便，就可以省去这些步骤啦