软工第四周学习博客

本周我们学习了利用Python对网页数据的爬取，从而实现了对上周疫情数据的更新，Python爬虫代码如下：

# coding=utf-8
import pymysql
import requests
import json
url = "https://c.m.163.com/ug/api/wuhan/app/data/list-total?t=316765403234"
header = {"user-agent": "Mozilla/5.0 (Linux; Android 8.0.0; Pixel 2 XL Build/OPD1.170816.004) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Mobile Safari/537.36"}
response = requests.get(url, headers = header )
mydir = json.loads(response.content.decode())
# 数据库的连接
db = pymysql.connect(host='localhost', port=3306, user='root', passwd='123456', db='payiqing', charset='utf8')
#使用cursor方法创建一个游标
cursor = db.cursor()
# mydir['data']['areaTree'][0]['children']再往后是省份
# print(mydir['data']['areaTree'][0]['children'][33])
for item in mydir['data']['areaTree'][0]['children']:
    name = item['name']
    confirm = item['total']['confirm']
    severe = item['total']['severe']
    heal = item['total']['heal']
    dead = item['total']['dead']
    suspect = item['total']['suspect']
    idcode = item['id']
    lastUpdateTime = item['lastUpdateTime']
    # 插入数据
    sql = "insert into pythontest(id,name,confirm,suspect,heal,dead,severe,idcode,lastupdatetime) values ({},'{}','{}','{}','{}','{}','{}','{}','{}');".format(0,name,confirm,suspect,heal,dead,severe,idcode,lastUpdateTime)
    print(sql)
    cursor.execute(sql)
    db.commit()

　　通过用Python将数据更新到数据库中，然后再利用上周的代码，将数据在web界面进行可视化。从而实现了数据的可更行