zoukankan      html  css  js  c++  java
  • scrapy 请求传参

    class MovieSpider(scrapy.Spider):
        name = 'movie'
        allowed_domains = ['www.id97.com']
        start_urls = ['http://www.id97.com/']
    
        def parse(self, response):
            div_list = response.xpath('//div[@class="col-xs-1-5 movie-item"]')
    
            for div in div_list:
                item = MovieproItem()
                item['name'] = div.xpath('.//h1/a/text()').extract_first()
                item['score'] = div.xpath('.//h1/em/text()').extract_first()
                #xpath(string(.))表示提取当前节点下所有子节点中的数据值(.)表示当前节点
                item['kind'] = div.xpath('.//div[@class="otherinfo"]').xpath('string(.)').extract_first()
                item['detail_url'] = div.xpath('./div/a/@href').extract_first()
                #请求二级详情页面,解析二级页面中的相应内容,通过meta参数进行Request的数据传递
                yield scrapy.Request(url=item['detail_url'],callback=self.parse_detail,meta={'item':item})
    
        def parse_detail(self,response):
            #通过response获取item
            item = response.meta['item']
            item['actor'] = response.xpath('//div[@class="row"]//table/tr[1]/a/text()').extract_first()
            item['time'] = response.xpath('//div[@class="row"]//table/tr[7]/td[2]/text()').extract_first()
            item['long'] = response.xpath('//div[@class="row"]//table/tr[8]/td[2]/text()').extract_first()
            #提交item到管道
            yield item
  • 相关阅读:
    假如
    天涯 好文章
    Widget中PendingIntent和RemoteViews用法
    申请android google map API key
    初步学习:Eclipse3.6开发C/C++程序
    Android getWidth和getMeasuredWidth的正解
    intent intentfilter
    mapview 生成apikey
    ViewConfiguration.getScaledTouchSlop () 用法
    Android JNI知识简介
  • 原文地址:https://www.cnblogs.com/ls1997/p/10874556.html
Copyright © 2011-2022 走看看