zoukankan      html  css  js  c++  java
  • Nginx 添加防爬虫

    include agent_deny.conf;

    conf下添加

    #禁止Scrapy|curl等工具的抓取

    if ($http_user_agent ~* (Scrapy|Curl|HttpClient))

    {

        return 403;

    }

    #禁止指定UA及UA为空的访问

    if ($http_user_agent ~ "FeedDemon|JikeSpider|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Java|Feedly|UniversalFeedParser|ApacheBench|Microsoft URL Control|Swiftbot|ZmEu|oBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|YisouSpider|HttpClient|MJ12bot|heritrix|EasouSpider|Ezooms|^$" )

    {

        return 403;            

    }

    #禁止非GET|HEAD|POST方式的抓取

    if ($request_method !~ ^(GET|HEAD|POST)$)

    {

      return 403;

    }

    纵有白头俱老意,奈何缘浅路芊芊.
  • 相关阅读:
    css gridlayout
    css position 属性
    简单的登陆界面
    introduce
    Event flow
    constructor和object的区别
    10th week (1)
    编程语言的历史和发展
    正则表达式的验证匹配
    js正则替换
  • 原文地址:https://www.cnblogs.com/hanby/p/14253017.html
Copyright © 2011-2022 走看看