zoukankan      html  css  js  c++  java
  • 爬虫,第八次实战xpath之(下载网站的视频播放链接)

    import requests
    from lxml import etree
    
    headers = {'User-Agen':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4098.3 Safari/537.36'}
    video_list = []
    for i in range(4,19):
        url = 'http://www.d9dyy.com/html/dl/list_%s.html'%i
        page_text = requests.get(url=url,headers=headers).text
        tree = etree.HTML(page_text)
        page_li = tree.xpath('/html/body/div[2]/div[2]/ul/li')
        for li in page_li:
                li_url = li.xpath('./a/@href')[0]
                # print(li_url)
                real_url = 'www.d9dyy.com'+li_url
                video_list.append(real_url)
                for video_li in video_list:
                    video_li = 'http://'+video_li
                    video_text = requests.get(url=video_li,headers=headers).text
                    video_tree = etree.HTML(video_text)
                    video_player_url = video_tree.xpath('/html/body/div[2]/div[1]/div[1]/iframe/@src')[0]
                    # print(video_player_url)
                    fp = open('网站影片播放链接.txt','w',encoding='UTF-8')
                    fp.write(video_player_url)
                    print('辛辛苦苦')

    中间有点小不足,太懒了,没把名字给写上去。爬虫慢慢娴熟了,后面要学习更精湛的技术了,就是反反反。。。爬虫技术。哈哈哈

  • 相关阅读:
    wepy框架构建小程序(1)
    百度地图2
    百度地图1
    VS Code 用户自定义代码片段(React)
    JS MarcoTasks MicroTasks
    JS位运算和遍历
    VueX源码分析(5)
    VueX源码分析(4)
    tensorflow 自带的实现函数翻转的函数
    namedtuple
  • 原文地址:https://www.cnblogs.com/sucanji/p/10858046.html
Copyright © 2011-2022 走看看