zoukankan      html  css  js  c++  java
  • scrapy常用命令(持续) | Commonly used Scrapy command list (con't)

    以下命令都是在CMD中运行,首先把路径定位到项目文件夹

    ------------------------------------------------------------------------------------------

    1. 创建一个scrapy project

    scrapy startproject project_name

    --------------------------------------------------------------------------------------------

    ------------------------------------------------------------------------------------------

    2. 自动生成spider代码命令

    首先进入项目目录,然后敲入如下命令

    scrapy genspider spider_name start_url

    ------------------------------------------------------------------------------------------

    --------------------------------------------------------------------------------------

    3. 列出项目中所有spider

    scrapy list

    返回项目中所有爬虫的名字

    -------------------------------------------------------------------------------------------

    ---------------------------------------------------------------------------------------

    4. 运行一个scrapy project,并且保存结果到csv文件

    scrapy crawl spider_name -o file.csv

    会通过yield命令生成iteration generator,把数据保存到file.csv文件中

    ---------------------------------------------------------------------------------------------

    English Version

    The Below commands will run at CMD which included python path in system env path. 

    Before run the commands, navigate to the directory where you want to create Scrapy project. 

    ------------------------------------------------------------------------------------------

    1. Createa a Scrapy project

    scrapy startproject project_name

    --------------------------------------------------------------------------------------------

    ------------------------------------------------------------------------------------------

    2. Create a spider. It will create a spider sub-class in your source code based on template.

    Navigete to project directory, then key in below command

    scrapy genspider spider_name start_url

    ------------------------------------------------------------------------------------------

    --------------------------------------------------------------------------------------

    3. List all spiders in your project

    scrapy list

    It will return all spiders with spider name

    -------------------------------------------------------------------------------------------

    ---------------------------------------------------------------------------------------

    4. Run a Scrapy project and save the result set into a CSV file.

    scrapy crawl spider_name -o file.csv -t csv

    Scrapy framework will make a iteration generator thru yield command in code and save your data into file.csv

    ---------------------------------------------------------------------------------------------

  • 相关阅读:
    linux开放防火墙端口方法
    Node.js 构建TCP服务
    React高阶组件
    在create-react-app 中启用装饰器语法
    Redux
    微信小程序组件与组件之间的通讯
    微信小程序网络请求
    微信小程序的页面导航跳转
    sqoop2启动job报错
    sqoop2启动client异常
  • 原文地址:https://www.cnblogs.com/wtang/p/7342211.html
Copyright © 2011-2022 走看看