zoukankan      html  css  js  c++  java
  • 一次scrapy失败的提示信息:由于连接方在一段时间后没有正确答复或连接的主机没有反 应,连接尝试失败

    2017-10-31 19:09:26 [scrapy.extensions.logstats] INFO: Crawled 8096 pages (at 67 pages/min), scraped 7720 items (at 62 items/min)
    2017-10-31 19:09:27 [scrapy.downloadermiddlewares.retry] DEBUG: Gave up retrying <GET http://collection.sina.com.cn/auction/zjgd/2017-10-19/doc-ifymzqpq2471870.shtml> (failed 3 times): TCP connection timed out: 10060: 由于连接方在一段时间后没有正确答复或连接的主机没有反应,连接尝试失败。.
    2017-10-31 19:09:27 [scrapy.core.scraper] ERROR: Error downloading <GET http://collection.sina.com.cn/auction/zjgd/2017-10-19/doc-ifymzqpq2471870.shtml>: TCP connection timed out: 10060: 由于连接方在一段时间后没有正确答复或连接的主机没有反 应,连接尝试失败。.
    2017-10-31 19:09:27 [scrapy.core.engine] INFO: Closing spider (finished)
    2017-10-31 19:09:27 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
    {'downloader/exception_count': 126,
    'downloader/exception_type_count/twisted.internet.error.DNSLookupError': 1,
    'downloader/exception_type_count/twisted.internet.error.TCPTimedOutError': 125,
    'downloader/request_bytes': 3278403,
    'downloader/request_count': 8298,
    'downloader/request_method_count/GET': 8298,
    'downloader/response_bytes': 269632727,
    'downloader/response_count': 8172,
    'downloader/response_status_count/200': 8153,
    'downloader/response_status_count/301': 5,
    'downloader/response_status_count/302': 1,
    'downloader/response_status_count/404': 13,
    'dupefilter/filtered': 7040,
    'finish_reason': 'finished',
    'finish_time': datetime.datetime(2017, 10, 31, 11, 9, 27, 889696),
    'httperror/response_ignored_count': 4,
    'httperror/response_ignored_status_count/404': 4,
    'item_scraped_count': 7720,
    'log_count/DEBUG': 16020,
    'log_count/ERROR': 197,
    'log_count/INFO': 27,
    'request_depth_max': 2,
    'response_received_count': 8096,
    'retry/count': 121,
    'retry/max_reached': 5,
    'retry/reason_count/twisted.internet.error.DNSLookupError': 1,
    'retry/reason_count/twisted.internet.error.TCPTimedOutError': 120,
    'scheduler/dequeued': 8274,
    'scheduler/dequeued/memory': 8274,
    'scheduler/enqueued': 8274,
    'scheduler/enqueued/memory': 8274,
    'start_time': datetime.datetime(2017, 10, 31, 10, 53, 26, 916526)}
    2017-10-31 19:09:27 [scrapy.core.engine] INFO: Spider closed (finished)

  • 相关阅读:
    hive中named_struct构造和使用
    hive非分区表导入csv数据
    hive常用的日期函数
    hive mapreduce reducer 调优
    Hive Analytics Functions row_number rank over partition by
    skip-grant-tables
    VC中的字符串转换宏
    CMAKE的使用
    VSCODE C/C++配置
    libcurl,多线程,gzip,共享DNS
  • 原文地址:https://www.cnblogs.com/anqiang1995/p/7763192.html
Copyright © 2011-2022 走看看