zoukankan      html  css  js  c++  java
  • 安装scrapy框架

    安装scrapy框架之前,需要安装几个必备库

    ps.分享个python库下载地址:https://www.lfd.uci.edu/~gohlke/pythonlibs/

    0、wheel(有了这个库之后可以本地安装pyhton库)

    1、lxml

    2、pyOpenSSL

    3、pywin32

    4、twisted

    anaconda可以使用自带的conda install xxxx来安装组件

    必备组件安装完成后,pip install scrapy 或者conda install scrapy

    安装完成后,test一下:

    命令行输入scrapy

    (C:ProgramDataAnaconda3) C:Userswangguoqiang>Scrapy
    Scrapy 1.5.1 - no active project
    
    Usage:
    scrapy <command> [options] [args]
    
    Available commands:
    bench Run quick benchmark test
    fetch Fetch a URL using the Scrapy downloader
    genspider Generate new spider using pre-defined templates
    runspider Run a self-contained spider (without creating a project)
    settings Get settings values
    shell Interactive scraping console
    startproject Create new project
    version Print Scrapy version
    view Open URL in browser, as seen by Scrapy
    
    [ more ] More commands available when run from project directory
    
    Use "scrapy <command> -h" to see more info about a command

    显示如下即代表安装成功

    继续输入Scrapy startproject hello

    (C:ProgramDataAnaconda3) C:Userswangguoqiang>Scrapy startproject hello
    New Scrapy project 'hello', using template directory 'c:\programdata\anaconda
    \lib\site-packages\scrapy\templates\project', created in:
    C:Userswangguoqianghello
    
    You can start your first spider with:
    cd hello
    scrapy genspider example example.com

    cd 进入创建的hello项目中

    输入scrapy genspider baidu www.baidu.com

    (C:ProgramDataAnaconda3) C:Userswangguoqiang>cd hello
    
    (C:ProgramDataAnaconda3) C:Userswangguoqianghello>scrapy genspider baidu ww
    w.baidu.com
    Created spider 'baidu' using template 'basic' in module:
      hello.spiders.baidu

    输入

    scrapy crawl baidu

  • 相关阅读:
    169_Majority Element
    171_Excel Sheet Column Number
    217_Contains Duplicate
    242_Valid Anagram
    【findIndex】根据数组对象某一元素的id,找到此元素在数组所在的位置
    【鼠标右击组件】v-contextmenu
    【 拖拽组件】基于 Sortable.js 的 Vue 拖拽组件。 访问地址:Vue.Draggable
    【vue-markdown编辑器】vue-markdown 组件github地址
    mac下如何制作windows启动盘
    【vue webstorm】WebStorm Vue代码格式错误
  • 原文地址:https://www.cnblogs.com/wang666/p/9467829.html
Copyright © 2011-2022 走看看