zoukankan      html  css  js  c++  java
  • 安装scrapy框架

    安装scrapy框架之前,需要安装几个必备库

    ps.分享个python库下载地址:https://www.lfd.uci.edu/~gohlke/pythonlibs/

    0、wheel(有了这个库之后可以本地安装pyhton库)

    1、lxml

    2、pyOpenSSL

    3、pywin32

    4、twisted

    anaconda可以使用自带的conda install xxxx来安装组件

    必备组件安装完成后,pip install scrapy 或者conda install scrapy

    安装完成后,test一下:

    命令行输入scrapy

    (C:ProgramDataAnaconda3) C:Userswangguoqiang>Scrapy
    Scrapy 1.5.1 - no active project
    
    Usage:
    scrapy <command> [options] [args]
    
    Available commands:
    bench Run quick benchmark test
    fetch Fetch a URL using the Scrapy downloader
    genspider Generate new spider using pre-defined templates
    runspider Run a self-contained spider (without creating a project)
    settings Get settings values
    shell Interactive scraping console
    startproject Create new project
    version Print Scrapy version
    view Open URL in browser, as seen by Scrapy
    
    [ more ] More commands available when run from project directory
    
    Use "scrapy <command> -h" to see more info about a command

    显示如下即代表安装成功

    继续输入Scrapy startproject hello

    (C:ProgramDataAnaconda3) C:Userswangguoqiang>Scrapy startproject hello
    New Scrapy project 'hello', using template directory 'c:\programdata\anaconda
    \lib\site-packages\scrapy\templates\project', created in:
    C:Userswangguoqianghello
    
    You can start your first spider with:
    cd hello
    scrapy genspider example example.com

    cd 进入创建的hello项目中

    输入scrapy genspider baidu www.baidu.com

    (C:ProgramDataAnaconda3) C:Userswangguoqiang>cd hello
    
    (C:ProgramDataAnaconda3) C:Userswangguoqianghello>scrapy genspider baidu ww
    w.baidu.com
    Created spider 'baidu' using template 'basic' in module:
      hello.spiders.baidu

    输入

    scrapy crawl baidu

  • 相关阅读:
    Base64字符串在传递过程中加号被转成空格
    IIS无法启动,应用程序池自动关闭,应用程序池XXXX将被自动禁用 解决方案之一
    我要写程序我要写程序
    react native项目改名(仅针对android)
    webpack之html-webpack-plugin的使用
    webpack之React开发环境配置
    webpack之webpack-dev-server的使用
    html2canvas截图空白问题
    webpack源码分析——参数初始化
    webpack源码分析——配置调试环境
  • 原文地址:https://www.cnblogs.com/wang666/p/9467829.html
Copyright © 2011-2022 走看看