1 安装
pip3 install beautifulsoup4
小测:
import requests r = requests.get("http://python123.io/ws/demo.html") from bs4 import BeautifulSoup soup = BeautifulSoup(r.text, 'html.parser') print(soup.prettify())
3 beautiful soup基本元素
<html> <body> <p class=“title”> … </p> </body> </html> 标签树 Beautiful Soup库是解析、遍历、维护“标签树”的功能库
Beautiful Soup库的引用
Beautiful Soup库,也叫beautifulsoup4 或 bs4
约定引用方式如下,即主要是用BeautifulSoup类
from bs4 import BeautifulSoup
import bs4
from bs4 import BeautifulSoup soup = BeautifulSoup("<html>data</html>", "html.parser") soup2 = BeautifulSoup(open("D://demo.html"), "html.parser") BeautifulSoup对应一个HTML/XML文档的全部内容
演示:
2 基于bs4库的HTML内容遍历方法
import requests
r = requests.get("http://python123.io/ws/demo.html")
demo = r.text
demo
Out[26]: '<html><head><title>This is a python demo page</title></head> <body> <p class="title"><b>The demo python introduces several python courses.</b></p> <p class="course">Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses: <a href="http://www.icourse163.org/course/BIT-268001" class="py1" id="link1">Basic Python</a> and <a href="http://www.icourse163.org/course/BIT-1001870001" class="py2" id="link2">Advanced Python</a>.</p> </body></html>'
HTML基本格式
<>...</>构成了所属关系
<html> <head> <title> This is a python demo page </title> </head> <body> <p class="title"> <b> The demo python introduces several python courses. </b> </p> <p class="course"> Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses: <a class="py1" href="http://www.icourse163.org/course/BIT-268001" id="link1"> Basic Python </a> and <a class="py2" href="http://www.icourse163.org/course/BIT-1001870001" id="link2"> Advanced Python </a> . </p> </body> </html>
标签树的下行遍历
soup = BeautifulSoup(demo, 'html.parser') soup.head Out[33]: <head><title>This is a python demo page</title></head> soup.head.contents Out[34]: [<title>This is a python demo page</title>] soup.body.contents Out[35]: [' ', <p class="title"><b>The demo python introduces several python courses.</b></p>, ' ', <p class="course">Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses: <a class="py1" href="http://www.icourse163.org/course/BIT-268001" id="link1">Basic Python</a> and <a class="py2" href="http://www.icourse163.org/course/BIT-1001870001" id="link2">Advanced Python</a>.</p>, ' '] len(soup.body.contents) Out[36]: 5 soup.body.contents[1] Out[37]: <p class="title"><b>The demo python introduces several python courses.</b></p>