zoukankan      html  css  js  c++  java
  • 从0开始学爬虫7之BeautifulSoup模块的简单介绍

    参考文档:

    https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/

    # 安装 beautifulsoup4

    (pytools) D:pythonpytools>pip install beautifulsoup4

    # coding=utf-8
    
    from bs4 import BeautifulSoup as bs
    import re
    
    
    html_doc = """
    <html><head><title>The Dormouse's story</title></head>
    
    <p class="title"><b>The Dormouse's story</b></p>
    
    <p class="story">Once upon a time there were three little sisters; and their names were
    <a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>,
    <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
    <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
    and they lived at the bottom of a well.</p>
    
    <p class="story">...</p>
    """
    soup = bs(html_doc, "html.parser")
    
    # print(soup.prettify())
    
    # print(soup.title.string)
    # print(soup.a)
    # print(soup.find(id='link2'))
    
    # print(soup.find(id='link2').string)
    
    # print(soup.find(id='link2').get_text())
    
    # 获取所有a标签的内容
    # for link in soup.find_all('a'):
    #     print(link.get_text())
    
    # 使用string获取不到p标签的内容
    # print(soup.find("p", {"class": "story"}).get_text())
    
    # 查找所有以b开头的标签
    # for tag in soup.find_all(re.compile("^b")):
    #     print(tag.get_text())
    
    # 查找所有的a标签中href以 http://example.com开头的数据,其中的.号代表任意字符,如果需要 表示点号本身可以用转义   href=re.compile(r"^http://example.com/")
    data = soup.find_all("a", href=re.compile(r"^http://example.com/"))
    print(data)
  • 相关阅读:
    面试题整理
    Node.js核心模块-stream流
    Node.js核心模块-crypto加密
    Node.js核心模块-assert
    Node.js全局对象-process
    nodemon
    随笔
    python学习笔记(十四): unittest
    python学习笔记(十三): 多线程多进程
    python学习笔记(十二):发送邮件
  • 原文地址:https://www.cnblogs.com/reblue520/p/11200073.html
Copyright © 2011-2022 走看看