zoukankan      html  css  js  c++  java
  • Python爬取疫情数据并存入mysql中

    import time, json, requests
    import pymysql
    
    url='https://view.inews.qq.com/g2/getOnsInfo?name=disease_h5&&callback=&_=%d'%int(time.time()*1000)
    data = json.loads(requests.get(url=url).json()['data'])
    lis=[]
    for m in range(len(data['areaTree'][0]['children'])):
        for n in range(len(data['areaTree'][0]['children'][m]['children'])):
            info={}
            info['pronvice']=data['areaTree'][0]['children'][m]['name']#省份
            info['city']=data['areaTree'][0]['children'][m]['children'][n]['name']#城市   len(data['areaTree'][0]['children'][0]['children'])
            info['total_confirm']=data['areaTree'][0]['children'][m]['children'][n]['total']['confirm']
            info['total_suspect']=data['areaTree'][0]['children'][m]['children'][n]['total']['suspect']
            info['total_dead']=data['areaTree'][0]['children'][m]['children'][n]['total']['dead']
            info['total_heal']=data['areaTree'][0]['children'][m]['children'][n]['total']['heal']
            info['today_confirm']=data['areaTree'][0]['children'][m]['children'][n]['today']['confirm']
            info['today_suspect']=data['areaTree'][0]['children'][m]['children'][n]['today']['suspect']
            info['today_dead']=data['areaTree'][0]['children'][m]['children'][n]['today']['dead']
            info['today_heal']=data['areaTree'][0]['children'][m]['children'][n]['today']['heal']
            lis.append(info)
    print(lis[0])
    print(len(lis))
    
    db =  pymysql.connect(host='127.0.0.1', port=3306, user='root', password='root', db='data')
    
    cursor = db.cursor()
    for i in range(len(lis)):
        cols = ", ".join('`{}`'.format(k) for k in lis[i].keys())
        print(cols)  # '`name`, `age`'
    
        val_cols = ', '.join('%({})s'.format(k) for k in lis[i].keys())
        print(val_cols)  # '%(name)s, %(age)s'
    
        sql = "insert into yiqing(%s) values(%s)"
        res_sql = sql % (cols, val_cols)
        print(res_sql)  # 'insert into users(`name`, `age`) values(%(name)s, %(age)s)'
    
        cursor.execute(res_sql, lis[i])  # 将字典a传入
        db.commit()
    

     结果

  • 相关阅读:
    eclipse中文乱码问题解决方案
    修改Tomcat的JDK目录
    Tomcat 5.5 修改服务器的侦听端口
    HTML DOM教程 27HTML DOM Button 对象
    HTML DOM教程 24HTML DOM Frameset 对象
    Navicat for MySQL v8.0.27 的注册码
    HTML DOM教程 25HTML DOM IFrame 对象
    Tomcat 5.5 的下载和安装
    android manifest相关属性
    ubuntu10.04 下 eclipse 小结
  • 原文地址:https://www.cnblogs.com/1061321925wu/p/12286648.html
Copyright © 2011-2022 走看看