zoukankan      html  css  js  c++  java
  • scrapy常用命令(持续) | Commonly used Scrapy command list (con't)

    以下命令都是在CMD中运行,首先把路径定位到项目文件夹

    ------------------------------------------------------------------------------------------

    1. 创建一个scrapy project

    scrapy startproject project_name

    --------------------------------------------------------------------------------------------

    ------------------------------------------------------------------------------------------

    2. 自动生成spider代码命令

    首先进入项目目录,然后敲入如下命令

    scrapy genspider spider_name start_url

    ------------------------------------------------------------------------------------------

    --------------------------------------------------------------------------------------

    3. 列出项目中所有spider

    scrapy list

    返回项目中所有爬虫的名字

    -------------------------------------------------------------------------------------------

    ---------------------------------------------------------------------------------------

    4. 运行一个scrapy project,并且保存结果到csv文件

    scrapy crawl spider_name -o file.csv

    会通过yield命令生成iteration generator,把数据保存到file.csv文件中

    ---------------------------------------------------------------------------------------------

    English Version

    The Below commands will run at CMD which included python path in system env path. 

    Before run the commands, navigate to the directory where you want to create Scrapy project. 

    ------------------------------------------------------------------------------------------

    1. Createa a Scrapy project

    scrapy startproject project_name

    --------------------------------------------------------------------------------------------

    ------------------------------------------------------------------------------------------

    2. Create a spider. It will create a spider sub-class in your source code based on template.

    Navigete to project directory, then key in below command

    scrapy genspider spider_name start_url

    ------------------------------------------------------------------------------------------

    --------------------------------------------------------------------------------------

    3. List all spiders in your project

    scrapy list

    It will return all spiders with spider name

    -------------------------------------------------------------------------------------------

    ---------------------------------------------------------------------------------------

    4. Run a Scrapy project and save the result set into a CSV file.

    scrapy crawl spider_name -o file.csv -t csv

    Scrapy framework will make a iteration generator thru yield command in code and save your data into file.csv

    ---------------------------------------------------------------------------------------------

  • 相关阅读:
    Flask学习目录
    Android笔记
    PyQt学习目录
    学习目录
    Python学习目录一
    sqlserver STUFF & STR
    PowerShell+Jenkins,实现项目的自动化部署
    DotNet Core Backend development
    .Net Core 类注入单元测试
    团队项目之团队展示&选题
  • 原文地址:https://www.cnblogs.com/wtang/p/7342211.html
Copyright © 2011-2022 走看看