zoukankan      html  css  js  c++  java
  • 从0开始学爬虫7之BeautifulSoup模块的简单介绍

    参考文档:

    https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/

    # 安装 beautifulsoup4

    (pytools) D:pythonpytools>pip install beautifulsoup4

    # coding=utf-8
    
    from bs4 import BeautifulSoup as bs
    import re
    
    
    html_doc = """
    <html><head><title>The Dormouse's story</title></head>
    
    <p class="title"><b>The Dormouse's story</b></p>
    
    <p class="story">Once upon a time there were three little sisters; and their names were
    <a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>,
    <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
    <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
    and they lived at the bottom of a well.</p>
    
    <p class="story">...</p>
    """
    soup = bs(html_doc, "html.parser")
    
    # print(soup.prettify())
    
    # print(soup.title.string)
    # print(soup.a)
    # print(soup.find(id='link2'))
    
    # print(soup.find(id='link2').string)
    
    # print(soup.find(id='link2').get_text())
    
    # 获取所有a标签的内容
    # for link in soup.find_all('a'):
    #     print(link.get_text())
    
    # 使用string获取不到p标签的内容
    # print(soup.find("p", {"class": "story"}).get_text())
    
    # 查找所有以b开头的标签
    # for tag in soup.find_all(re.compile("^b")):
    #     print(tag.get_text())
    
    # 查找所有的a标签中href以 http://example.com开头的数据,其中的.号代表任意字符,如果需要 表示点号本身可以用转义   href=re.compile(r"^http://example.com/")
    data = soup.find_all("a", href=re.compile(r"^http://example.com/"))
    print(data)
  • 相关阅读:
    Microsoft.NET User Group
    白话MVP 和 MVVM 【转】
    高效的二分法TOP MAX/TOP MIN分页存贮过程
    策略模式5
    说说我们项目组的例行会议
    合格的项目经理
    说说我们的招聘和面试
    web安全问题汇总
    ASP.NET中常用的优化性能方法
    说说我们安排的培训
  • 原文地址:https://www.cnblogs.com/reblue520/p/11200073.html
Copyright © 2011-2022 走看看