zoukankan      html  css  js  c++  java
  • Nginx 负载均衡的Cache缓存批量清理的操作记录

    1)nginx.conf配置

    [root@inner-lb01 ~]# cat /data/nginx/conf/nginx.conf
    user  www;
    worker_processes  8;
     
    #error_log  logs/error.log;
    #error_log  logs/error.log  notice;
    #error_log  logs/error.log  info;
     
    #pid        logs/nginx.pid;
     
     
    events {
        worker_connections  65535;
    }
     
     
    http {
        include       mime.types;
        default_type  application/octet-stream;
        charset utf-8;
           
        ######
        ## set access log format
        ######
        log_format  main  '$remote_addr $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '$http_user_agent $http_x_forwarded_for $request_time $upstream_response_time $upstream_addr $upstream_status';
     
        #######
        ## http setting
        #######
        sendfile       on;
        tcp_nopush     on;
        tcp_nodelay    on;
        keepalive_timeout  65;
        proxy_cache_path /var/www/cache levels=1:2 keys_zone=mycache:20m max_size=2048m inactive=60m;
        proxy_temp_path /var/www/cache/tmp;
     
        fastcgi_connect_timeout 3000;
        fastcgi_send_timeout 3000;
        fastcgi_read_timeout 3000;
        fastcgi_buffer_size 256k;
        fastcgi_buffers 8 256k;
        fastcgi_busy_buffers_size 256k;
        fastcgi_temp_file_write_size 256k;
        fastcgi_intercept_errors on;
     
        #
        client_header_timeout 600s;
        client_body_timeout 600s;
       # client_max_body_size 50m;
        client_max_body_size 100m;
        client_body_buffer_size 256k;
     
        gzip  on;
        gzip_min_length  1k;
        gzip_buffers     4 16k;
        gzip_http_version 1.1;
        gzip_comp_level 9;
        gzip_types       text/plain application/x-javascript text/css application/xml text/javascript application/x-httpd-php;
        gzip_vary on;
     
        ## includes vhosts
        include vhosts/*.conf;
    }
    

    针对上面的配置,注意下面两点:
    1. log_format如上的配置后,nginx的log日志中就能显示客户端访问的来源ip了。如下:
    [root@inner-lb01 ~]# tail -10 /data/nginx/logs/bs7001.wangshibo.cn-access.log

    172.16.51.163 - [21/Nov/2017:17:01:46 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.033 0.033 172.16.51.161:7001 200
    172.16.51.163 - [21/Nov/2017:17:01:46 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.021 0.021 172.16.51.161:7001 200
    172.16.51.163 - [21/Nov/2017:17:01:52 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.028 0.028 172.16.51.161:7001 200
    172.16.51.163 - [21/Nov/2017:17:01:52 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.022 0.022 172.16.51.161:7001 200
    172.16.51.163 - [21/Nov/2017:17:01:58 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.033 0.033 172.16.51.161:7001 200
    172.16.51.163 - [21/Nov/2017:17:01:58 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.021 0.021 172.16.51.161:7001 200
    172.16.51.163 - [21/Nov/2017:17:02:04 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.034 0.034 172.16.51.161:7001 200
    172.16.51.163 - [21/Nov/2017:17:02:04 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.021 0.021 172.16.51.161:7001 200
    172.16.51.163 - [21/Nov/2017:17:02:10 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.031 0.031 172.16.51.161:7001 200
    172.16.51.163 - [21/Nov/2017:17:02:10 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.022 0.022 172.16.51.161:7001 200

    2. nginx的静态缓存配置:
    proxy_cache_path /var/www/cache          表示定义缓存存储目录,手动创建;
    levels=1:2          缓存级别,表示缓存目录的第一级目录是1个字符,第二级目录是2个字符;
    keys_zone=mycache:20m      内核中建立用于缓存缓存数据源数据的空间,查找缓存的时候,先从这个内核空间中找到,缓存数据的源数据,然后再到对应目录中查找缓存;
    max_size=2048m        缓存空间最大值;
    inactive=60m         缓存的数据,60分钟内没有被访问过就删除
    proxy_temp_path /var/www/cache/tmp         表示创建缓存的时候可能生成一些临时文件存放的位置,自动创建

    2)vhosts虚拟目录的upstream配置

    [root@inner-lb01 ~]# cat /data/nginx/conf/vhosts/bs7001.wangshibo.cn.conf 
    upstream bs7001 {
          ip_hash;
          server 172.16.51.161:7001 max_fails=3 fail_timeout=15s;
          server 172.16.51.162:7001 max_fails=3 fail_timeout=15s;
    }
              
      server {
          listen      80;
          server_name bs7001.wangshibo.cn;
        
          access_log  /data/nginx/logs/bs7001.wangshibo.cn-access.log main;
          error_log  /data/nginx/logs/bs7001.wangshibo.cn-error.log;
        
     location / {
             proxy_pass http://bs7001;
             proxy_redirect off ;
             proxy_set_header Host $host;
             proxy_set_header X-Real-IP $remote_addr;
             proxy_set_header REMOTE-HOST $remote_addr;
             proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
             proxy_connect_timeout 300;
             proxy_send_timeout 300;
             proxy_read_timeout 600;
             proxy_buffer_size 256k;
             proxy_buffers 4 256k;
             proxy_busy_buffers_size 256k;
             proxy_temp_file_write_size 256k;
             proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504 http_404;
             proxy_max_temp_file_size 128m;
             proxy_cache mycache;
             proxy_cache_valid 200 302 1h; 
             proxy_cache_valid 301 1d;
             proxy_cache_valid any 1m;
            }
    }

    由上面的配置需要注意:
    proxy_cache mycache;            表示内存缓存源数据空间名字,nginx.conf文件里已经定义
    proxy_cache_valid 200 302 1h;       表示200、302状态的页面资源缓存1小时
    proxy_cache_valid 301 1d;        表示301状态的页面资源缓存1天
    proxy_cache_valid any 1m;       他缓存1分钟

     3)清理cache缓存

    [root@inner-lb01 ~]# ll /var/www/cache/
    total 68
    drwx------ 27 www www  4096 Nov 21 16:24 0
    drwx------ 24 www www  4096 Nov 21 16:24 1
    drwx------ 27 www www  4096 Nov 21 16:24 2
    drwx------ 21 www www  4096 Nov 21 17:18 3
    drwx------ 35 www www  4096 Nov 21 16:24 4
    drwx------ 31 www www  4096 Nov 21 16:24 5
    drwx------ 34 www www  4096 Nov 21 14:38 6
    drwx------ 28 www www  4096 Nov 21 14:38 7
    drwx------ 26 www www  4096 Nov 21 16:25 8
    drwx------ 23 www www  4096 Nov 21 16:24 9
    drwx------ 25 www www  4096 Nov 21 16:13 a
    drwx------ 25 www www  4096 Nov 21 14:58 b
    drwx------ 21 www www  4096 Nov 21 16:24 c
    drwx------ 18 www www  4096 Nov 21 16:12 d
    drwx------ 18 www www  4096 Nov 21 16:25 e
    drwx------ 20 www www  4096 Nov 21 16:24 f
    drwx------  2 www root 4096 Nov 21 17:18 tmp
    

    编写批量清理缓存的脚本

    [root@inner-lb01 ~]# cat /opt/nginx_cache_clean.sh 
    #!/bin/bash  
    #Email:wangshibo@wangshibo.com
    #Auto Clean Nginx Proxy_Cache Shell Scripts  
    #Aunthor:wanghsibo
    #Date:2017-11-20 
    echo -e "
    
    "  
    echo -n -e "e[35;1m请输入Nginx Proxy_cache缓存的具体路径(友情提示:可以使用Tab补全功能哦!)e[0me[34;5m:e[0m"  
    read -e path  
    CACHE_DIR=$path  
    echo -e "e[32;1m----------------------------------------------------------------e[0m"  
    echo -e "e[32;1m----------------------------------------------------------------e[0m"  
    echo -n -e "e[32;1m请输入你要删除的动作
    1.按文件类型删除	2.按具体文件名删除	3.按文件目录删除
    :"  
    read action  
         case $action in  
    1)  
    echo -e "e[32;1m----------------------------------------------------------------e[0m"  
    echo -e "e[32;1m----------------------------------------------------------------e[0m"  
    echo -n -e "e[34;1m 请输入你要删除的缓存文件类型(可以输入多个参数空格隔开)e[0me[34;5m:e[0m"  
    read -a FILE  
    for i in `echo ${FILE[*]}|sed 's/ /
    /g'`  
    do  
    grep -r -a  .$i ${CACHE_DIR}| awk 'BEGIN {FS=":"} {print $1}'  > /tmp/cache_list.txt  
     for j in `cat /tmp/cache_list.txt`  
    do  
       rm  -rf  $j  
       echo "$i  $j 删除成功!"  
     done  
    done  
    ;;  
    2)  
    echo -e "e[32;1m----------------------------------------------------------------e[0m"  
    echo -e "e[32;1m----------------------------------------------------------------e[0m"  
    echo -n -e "e[33;1m 请输入你要删除的缓存文件具体名称(可以输入多个参数空格隔开)e[0me[34;5m:e[0m"  
    read -a FILE  
    for i in `echo ${FILE[*]}|sed 's/ /
    /g'`  
    do  
    grep -r -a  $i ${CACHE_DIR}| awk 'BEGIN {FS=":"} {print $1}'  > /tmp/cache_list.txt  
     for j in `cat /tmp/cache_list.txt`  
    do  
       rm  -rf  $j  
       echo "$i  $j 删除成功!"  
     done  
    done  
    ;;  
    3)  
    echo -e "e[32;1m----------------------------------------------------------------e[0m"  
    echo -e "e[32;1m----------------------------------------------------------------e[0m"  
    echo -n -e "e[33;1m支持的模式有:
    1.清除网站store目录下的所有缓存:test.dd.com/data/upload/shop/store
    2.清除网站shop下的所有缓存:test.dd.com/data/upload/shope[0m
    "  
    echo -n -e "e[34;1m 请输入你要删除的缓存文件具体目录e[0me[34;5m:e[0m"  
    read -a FILE  
    for i in `echo ${FILE[*]}|sed 's/ /
    /g'`  
    do  
    grep -r -a  "$i" ${CACHE_DIR}| awk 'BEGIN {FS=":"} {print $1}'  > /tmp/cache_list.txt  
     for j in `cat /tmp/cache_list.txt`  
    do  
       rm  -rf  $j  
       echo "$i  $j 删除成功!"  
     done  
    done  
    ;;  
    *)  
    echo "输入错误,请重新输入"  
    ;;  
    esac

    执行脚本:

    [root@inner-lb01 ~]# chmod 755 /opt/nginx_cache_clean.sh
    [root@inner-lb01 ~]# sh /opt/nginx_cache_clean.sh
    
    
    
    请输入Nginx Proxy_cache缓存的具体路径(友情提示:可以使用Tab补全功能哦!):/var/www/cache/
    ----------------------------------------------------------------
    ----------------------------------------------------------------
    请输入你要删除的动作
    1.按文件类型删除   2.按具体文件名删除  3.按文件目录删除
    :1
    ----------------------------------------------------------------
    ----------------------------------------------------------------
     请输入你要删除的缓存文件类型(可以输入多个参数空格隔开):html htm js css jpg gif png jpeg bmp flv swf ico
    html  /var/www/cache/6/5e/75685391e510dde1671cb198c113e5e6 删除成功!
    html  /var/www/cache/3/b7/49e71a93813bcedd9e0c679bc20c1b73 删除成功!
    html  /var/www/cache/3/b7/49e71a93813bcedd9e0c679bc20c1b73 删除成功!
    html  /var/www/cache/8/86/ee4a14c4cc9e12503f9e8bef123df868 删除成功!
    html  /var/www/cache/8/53/a9e2ef17aba2c92aac71b36a50329538 删除成功!
    html  /var/www/cache/8/53/a9e2ef17aba2c92aac71b36a50329538 删除成功!
    html  /var/www/cache/8/53/a9e2ef17aba2c92aac71b36a50329538 删除成功!
    html  /var/www/cache/8/53/a9e2ef17aba2c92aac71b36a50329538 删除成功!
    js  /var/www/cache/f/3c/6a74478ecb76c1235ef47766e1b333cf 删除成功!
    js  /var/www/cache/f/3c/6a74478ecb76c1235ef47766e1b333cf 删除成功!
    js  /var/www/cache/1/2b/eeabd84e3ed8cc4bec61f608e2ab32b1 删除成功!
    js  /var/www/cache/6/98/e38773f35e9e18fe4449396105faf986 删除成功!
    js  /var/www/cache/e/d6/5c7f20342601e4cee752424175e35d6e 删除成功!
    js  /var/www/cache/8/f9/ec9fa02ace81a82d98baa04b3d810f98 删除成功!
    js  /var/www/cache/7/62/4ad7e6a1ea04cf484a911250e5af5627 删除成功!
    js  /var/www/cache/7/62/4ad7e6a1ea04cf484a911250e5af5627 删除成功!
    js  /var/www/cache/c/a7/1edf94c22612d6e7f2cb2c0b289dba7c 删除成功!
    js  /var/www/cache/a/2e/6673665a2ec9fae7640f61c121ae42ea 删除成功!
    css  /var/www/cache/7/45/26ddc4b8c5085be9757b90cd4a36a457 删除成功!
    css  /var/www/cache/7/45/26ddc4b8c5085be9757b90cd4a36a457 删除成功!
    css  /var/www/cache/7/7f/cb39e58e71c9fb5a1ba77107288167f7 删除成功!
    css  /var/www/cache/7/7f/cb39e58e71c9fb5a1ba77107288167f7 删除成功!
    css  /var/www/cache/a/45/9270775ac808bde171eda5f39547845a 删除成功!
    css  /var/www/cache/a/45/9270775ac808bde171eda5f39547845a 删除成功!
    css  /var/www/cache/a/45/9270775ac808bde171eda5f39547845a 删除成功!
    css  /var/www/cache/2/40/ab83b9cdeec1955375d28f6e74b6e402 删除成功!
    css  /var/www/cache/2/40/ab83b9cdeec1955375d28f6e74b6e402 删除成功!
    png  /var/www/cache/f/a0/eaa602c9070ef04101a5c148d63b4a0f 删除成功!
    png  /var/www/cache/f/a0/eaa602c9070ef04101a5c148d63b4a0f 删除成功!
    png  /var/www/cache/f/a0/eaa602c9070ef04101a5c148d63b4a0f 删除成功!
    png  /var/www/cache/f/a0/eaa602c9070ef04101a5c148d63b4a0f 删除成功!
    png  /var/www/cache/f/a0/eaa602c9070ef04101a5c148d63b4a0f 删除成功!
    png  /var/www/cache/1/f7/3ba2c5ee13c4a086ed885221cb1a9f71 删除成功!
    png  /var/www/cache/1/f7/3ba2c5ee13c4a086ed885221cb1a9f71 删除成功!
    png  /var/www/cache/6/f3/5e00a9011498e811a075031373928f36 删除成功!
    png  /var/www/cache/6/f3/5e00a9011498e811a075031373928f36 删除成功!
    png  /var/www/cache/8/70/03a64ee54acbe99ce81d2e907edec708 删除成功!

    上面脚本执行后,会提示输入cache的缓存目录,然后选择删除缓存文件的条件(这里我选择"按文件类型删除"),选择了删除html 、htm、js、css、jpg 、gif、 png 、jpeg 、bmp 、flv、 swf 、ico这12中文件格式的缓存文件。

    或者直接使用find命令查找缓存目录下的文件,直接将文件全部删除

    # find /var/www/cache -type f|xargs rm -f 

    -------------------Nginx禁用html、js、css缓存-----------------

    在本地开发的时候,经常会碰到缓存引起的莫名其妙的问题,最暴力的方式就是清掉浏览器的缓存,或者使用Ctrl + F5,Shift + F5强制刷新页面。
    有时候按了好几下,缓存还是清不掉,只能暂时禁用浏览器静态资源缓存了,配置如下:
    location ~.*.(js|css|html|png|jpg)$
    {
        add_header Cache-Control no-cache;
    }
    
    或者
    location /js
    {
        add_header Cache-Control no-cache;
    }
    
    现在,按F5就行了!
  • 相关阅读:
    leetcode33. Search in Rotated Sorted Array
    pycharm 设置sublime text3 monokai主题
    django class Meta
    leetcode30, Substring With Concatenation Of All Words
    Sublime text3修改tab键为缩进为四个空格,
    sublime text3 python打开图像的问题
    安装上imesupport输入法依然不跟随的解决办法,
    sublime text3 的插件冲突弃用问题,
    sublime text3 BracketHighlighter括号匹配的设置
    windows 下wget的使用
  • 原文地址:https://www.cnblogs.com/kevingrace/p/5840950.html
Copyright © 2011-2022 走看看