Nginx核心知识100讲学习笔记（陶辉）详解HTTP模块（接受请求模块|正则表达式|冲突合并）

一、冲突的配置指令以谁为准？

1、配置块的嵌套

main
http {
		upstream { … }
		split_clients {…}
		map {…}
		geo {…}
		server {
				if () {…}
				location {
						limit_except {…}
				}
				location {
						location {
						}
				}
		}
		server {
		}
}

2、指令的Context

Syntax:  log_format name [escape=default|json|none] string ...;
Default:  log_format combined "...";
Context:  http

Syntax:
access_log path [format [buffer=size] [gzip[=level]] [flush=time] [if=condition]];
access_log off;
Default:  access_log logs/access.log combined;
Context:  http, server, location, if in location, limit_excep

3、值指令：存储配置项的值（可以合并）

root
access_log
gzip

4、动作类指令：指定行为（不可以合并）

rewrite
proxy_pass

生效阶段

server_rewrite 阶段
rewrite 阶段
content 阶段

5、存储值的指令集成规则：向上覆盖

server {
	listen 8080;
	root /home/geek/nginx/html;
	access_log logs/geek.access.log main;
	location /test {
		root /home/geek/nginx/test;
		access_log logs/access.test.log main;
	}
	location /dlib {
		alias dlib/;
	}
	location / {
	}
}

子配置不存在时，直接使用父配置块；子配置存在时，直接覆盖父配置块

二、HTTP模块合并配置的实现

1、指令在哪个块下生效？

就是11个阶段的那个阶段

2、指令允许出现在那些块下？

    { ngx_string("valid_referers"),
      NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_1MORE,
......

3、在server块内生效、从http向server合并指令：

• char *(*merge_srv_conf)(ngx_conf_t *cf, void *prev, void *conf);

任何一个nginx模块中都有一个结构体叫：ngx_module_t 这个结构体提供的所有配置指令都在

ngx_module_t  ngx_http_referer_module = {
    NGX_MODULE_V1,
    &ngx_http_referer_module_ctx,          /* module context */
    ngx_http_referer_commands,             /* module directives */
    NGX_HTTP_MODULE,                       /* module type */
    NULL,                                  /* init master */
    NULL,                                  /* init module */
    NULL,                                  /* init process */
    NULL,                                  /* init thread */
    NULL,                                  /* exit thread */
    NULL,                                  /* exit process */
    NULL,                                  /* exit master */
    NGX_MODULE_V1_PADDING
};

这个结构体提供的所有配置指令都在都在ngx_command_t的数组中每一个元素就是一条指令

static ngx_command_t  ngx_http_referer_commands[] = {

    { ngx_string("valid_referers"),
      NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_1MORE,
      ngx_http_valid_referers,
      NGX_HTTP_LOC_CONF_OFFSET,
      0,
      NULL },

    { ngx_string("referer_hash_max_size"),
      NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
      ngx_conf_set_num_slot,
      NGX_HTTP_LOC_CONF_OFFSET,
      offsetof(ngx_http_referer_conf_t, referer_hash_max_size),
      NULL },

    { ngx_string("referer_hash_bucket_size"),
      NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
      ngx_conf_set_num_slot,
      NGX_HTTP_LOC_CONF_OFFSET,
      offsetof(ngx_http_referer_conf_t, referer_hash_bucket_size),
      NULL },

      ngx_null_command
};

定义了8个可回调方法

static ngx_http_module_t  ngx_http_referer_module_ctx = {
    ngx_http_referer_add_variables,        /* preconfiguration */
    NULL,                                  /* postconfiguration */

    NULL,                                  /* create main configuration */
    NULL,                                  /* init main configuration */

    NULL,                                  /* create server configuration */
    NULL,                                  /* merge server configuration */

    ngx_http_referer_create_conf,          /* create location configuration */
    ngx_http_referer_merge_conf            /* merge location configuration */
};

4、配置缓存在内存

• char *(*merge_loc_conf)(ngx_conf_t *cf, void *prev, void *conf);

重点看合并函数

static char *
ngx_http_referer_merge_conf(ngx_conf_t *cf, void *parent, void *child)
{
    ngx_http_referer_conf_t *prev = parent;
    ngx_http_referer_conf_t *conf = child;

    ngx_uint_t                 n;
    ngx_hash_init_t            hash;
    ngx_http_server_name_t    *sn;
    ngx_http_core_srv_conf_t  *cscf;

    if (conf->keys == NULL) {
        conf->hash = prev->hash;

#if (NGX_PCRE)
        ngx_conf_merge_ptr_value(conf->regex, prev->regex, NULL);
        ngx_conf_merge_ptr_value(conf->server_name_regex,
                                 prev->server_name_regex, NULL);
#endif
        ngx_conf_merge_value(conf->no_referer, prev->no_referer, 0);
        ngx_conf_merge_value(conf->blocked_referer, prev->blocked_referer, 0);
        ngx_conf_merge_uint_value(conf->referer_hash_max_size,
                                  prev->referer_hash_max_size, 2048);
        ngx_conf_merge_uint_value(conf->referer_hash_bucket_size,
                                  prev->referer_hash_bucket_size, 64);

        return NGX_CONF_OK;
    }

    if (conf->server_names == 1) {
        cscf = ngx_http_conf_get_module_srv_conf(cf, ngx_http_core_module);

        sn = cscf->server_names.elts;
        for (n = 0; n < cscf->server_names.nelts; n++) {

#if (NGX_PCRE)
            if (sn[n].regex) {

                if (ngx_http_add_regex_server_name(cf, conf, sn[n].regex)
                    != NGX_OK)
                {
                    return NGX_CONF_ERROR;
                }

                continue;
            }
#endif

            if (ngx_http_add_referer(cf, conf->keys, &sn[n].name, NULL)
                != NGX_OK)
            {
                return NGX_CONF_ERROR;
            }
        }
    }

    if ((conf->no_referer == 1 || conf->blocked_referer == 1)
        && conf->keys->keys.nelts == 0
        && conf->keys->dns_wc_head.nelts == 0
        && conf->keys->dns_wc_tail.nelts == 0)
    {
        ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                      "the "none" or "blocked" referers are specified "
                      "in the "valid_referers" directive "
                      "without any valid referer");
        return NGX_CONF_ERROR;
    }

    ngx_conf_merge_uint_value(conf->referer_hash_max_size,
                              prev->referer_hash_max_size, 2048);
    ngx_conf_merge_uint_value(conf->referer_hash_bucket_size,
                              prev->referer_hash_bucket_size, 64);
    conf->referer_hash_bucket_size = ngx_align(conf->referer_hash_bucket_size,
                                               ngx_cacheline_size);

    hash.key = ngx_hash_key_lc;
    hash.max_size = conf->referer_hash_max_size;
    hash.bucket_size = conf->referer_hash_bucket_size;
    hash.name = "referer_hash";
    hash.pool = cf->pool;

三、Listen 指令

语法解析

Syntax:

listen address[:port] [default_server] [ssl] [http2 | spdy] [proxy_protocol] [setfib=number] [fastopen=number] [backlog=number] [rcvbuf=size]
[sndbuf=size] [accept_filter=filter] [deferred] [bind] [ipv6only=on|off] [reuseport] [so_keepalive=on|off|[keepidle]:[keepintvl]:[keepcnt]];
listen port [default_server] [ssl] [http2 | spdy] [proxy_protocol] [setfib=number] [fastopen=number] [backlog=number] [rcvbuf=size] [sndbuf=size]
[accept_filter=filter] [deferred] [bind] [ipv6only=on|off] [reuseport] [so_keepalive=on|off|[keepidle]:[keepintvl]:[keepcnt]];
listen unix:path [default_server] [ssl] [http2 | spdy] [proxy_protocol] [backlog=number] [rcvbuf=size] [sndbuf=size] [accept_filter=filter] [deferred]
[bind] [so_keepalive=on|off|[keepidle]:[keepintvl]:[keepcnt]];

Default:  listen *:80 | *:8000;
Context:  server

示例

listen unix:/var/run/nginx.sock;
listen 127.0.0.1:8000;
listen 127.0.0.1;
listen 8000;
listen *:8000;
listen localhost:8000 bind;
listen [::]:8000 ipv6only=on;
listen [::1];

四、接收请求事件模块

1、图解过程

 2、过程中的疑难点

两种内存池

分配连接内存池：connection_pool_size:512
请求内存池

如果60s没收到请求就超时：client_header_timeout: 60s

把用户的data读取到我的用户态、这样就要分配内存：client_header_buffer_size:1k；这个很大并不合适、如果超过1k会有什么变化？

五、接收请求http模块

1、图解接收过程

 2、过程中的疑难点

1、请求内存池会分配多大？

基本是连接内存池的8倍、因为请求的上下文存在业务、如果分配的太小的话会不停的扩充会影响性能

2、分配大内存：

解决了什么问题：解决有一些url太长了、最大会是多大？
并不是分配32k、先分配一个8k，把刚刚的内容拷贝到我们的第一部分、剩下的7k再去接收剩下的url如果还没有接收完、按照以上迭代、最多分配32k

large_client_header_buffers:4 8k

3、什么叫表示URL?

nginx会有很多变量、这些变量它并不是复制一份、而是c语言的指针指向我们接收的请求行、所以nginx的性能才如此强大

这里的large_client_header_buffers:4 8k和解析请求是复用的也就是解析请求用了16k 那么head就只剩下16k

 3、过大的请求头部

Syntax:  client_header_buffer_size size;
Default:  client_header_buffer_size 1k;
Context:  http, server

Syntax:  large_client_header_buffers number size;
Default:  large_client_header_buffers 4 8k;
Context:  http, serve

六、正则表达式

1、元字符

2、重复

3、正则表达式【和（）】

：取消原字符的特殊含义

原始url：/admin/website/article/35/change/uploads/party/5.jpg
转换后的：/static/uploads/party/5.jpg

（）分组与取值

匹配原始url的正则表达式：
/^/admin/website/article/(d+)/change/uploads/(w+)/(w+).(png|jpg|gif|jpeg|bmp)$/
rewrite^/admin/website/solution/(d+)/change/uploads/(.*).(png|jpg|gif|jpeg|bmp)$/static/uploads/$2/$3.$4 last;

4、正则表达式的验证pcre2test

安装

wget file:///C:/Users/lah/Downloads/pcre2-10.35.tar.gz
tar xf pcre2-10.35.tar.gz 
cd pcre2-10.35
./configure --enable-utf8
make && make install
echo $?

测试验证：

[root@ceph-client ~]# pcre2test
PCRE2 version 10.35 2020-05-09
re> /^/admin/website/article/(d+)/change/uploads/(w+)/(w+).(png|jpg|gif|jpeg|b
mp)$/ > 
data> /admin/website/article/35/change/uploads/party/5.jpg
0: /admin/website/article/35/change/uploads/party/5.jpg
1: 35
2: party
3: 5
4: jpg

注意事项：

1、在nginx中使用不需要添加/但是在pcre2test中需要加

2、不能带点、所以这里的.我用了转义符号

.(png|jpg|gif|jpeg|bmp)$/