zoukankan      html  css  js  c++  java
  • 爬取校园新闻首页的新闻

    1. 用requests库和BeautifulSoup库,爬取校园新闻首页新闻的标题、链接、正文。

    2. 分析字符串,获取每篇新闻的发布时间,作者,来源,摄影等信息。

    3. 将其中的发布时间由str转换成datetime类型。

    import requests
    
    from bs4 import BeautifulSoup
    from datetime import datetime
    
    url="http://news.gzcc.cn/html/xiaoyuanxinwen/"
    res=requests.get(url)
    res.encoding="utf-8"
    
    soup=BeautifulSoup(res.text,"html.parser")
    for news in soup.select("li"):
        if len(news.select(".news-list-title")) > 0:
            print(news.select(".news-list-title"))
    
    for news in soup.select("li"):
        if len(news.select(".news-list-title")) > 0:
            t=news.select('.news-list-title')[0].text
            dt=news.select('.news-list-info')[0].contents[0].text
            a = news.select('a')[0].attrs['href']
            print(t,dt,a,'
    ')
    for news in soup.select("li"):
        if len(news.select(".news-list-title")) > 0:
            a = news.select('a')[0].attrs['href']#新闻链接
            t = news.select('.news-list-title')[0].text#标题
            res1=requests.get(a)
            res1.encoding='utf-8'
            soup1=BeautifulSoup(res1.text,'html.parser')
            dt=soup1.select('.show-info')[0].text#时间
            print(t,'
    ',dt,'
    ')
            print(soup1.select('#content')[0].text)#内容
            t1=dt.lstrip('发布时间:')[:19]
            i=dt.find('摄影:')
            s=dt[dt.find('来源:'):].split()[0].lstrip('来源:')
            a = dt[dt.find('作者:'):].split()[0].lstrip('作者:')
            if i>0:
                p = dt[dt.find('摄影:'):].split()[0].lstrip('摄影:')
                print(t1,a,s,p)
            break
    str = '2018-03-30 17:10:12 '
    datetime.strptime(str,'%Y-%m-%d %H:%M:%S ')
    print('
    
    ',str)
  • 相关阅读:
    Security headers quick reference Learn more about headers that can keep your site safe and quickly look up the most important details.
    Missing dollar riddle
    Where Did the Other Dollar Go, Jeff?
    proteus 与 keil 联调
    cisco router nat
    router dhcp and dns listen
    配置802.1x在交换机的端口验证设置
    ASAv931安装&初始化及ASDM管理
    S5700与Cisco ACS做802.1x认证
    playwright
  • 原文地址:https://www.cnblogs.com/gg8-8/p/8698233.html
Copyright © 2011-2022 走看看