zoukankan      html  css  js  c++  java
  • Python爬取疫情数据并存入mysql中

    import time, json, requests
    import pymysql
    
    url='https://view.inews.qq.com/g2/getOnsInfo?name=disease_h5&&callback=&_=%d'%int(time.time()*1000)
    data = json.loads(requests.get(url=url).json()['data'])
    lis=[]
    for m in range(len(data['areaTree'][0]['children'])):
        for n in range(len(data['areaTree'][0]['children'][m]['children'])):
            info={}
            info['pronvice']=data['areaTree'][0]['children'][m]['name']#省份
            info['city']=data['areaTree'][0]['children'][m]['children'][n]['name']#城市   len(data['areaTree'][0]['children'][0]['children'])
            info['total_confirm']=data['areaTree'][0]['children'][m]['children'][n]['total']['confirm']
            info['total_suspect']=data['areaTree'][0]['children'][m]['children'][n]['total']['suspect']
            info['total_dead']=data['areaTree'][0]['children'][m]['children'][n]['total']['dead']
            info['total_heal']=data['areaTree'][0]['children'][m]['children'][n]['total']['heal']
            info['today_confirm']=data['areaTree'][0]['children'][m]['children'][n]['today']['confirm']
            info['today_suspect']=data['areaTree'][0]['children'][m]['children'][n]['today']['suspect']
            info['today_dead']=data['areaTree'][0]['children'][m]['children'][n]['today']['dead']
            info['today_heal']=data['areaTree'][0]['children'][m]['children'][n]['today']['heal']
            lis.append(info)
    print(lis[0])
    print(len(lis))
    
    db =  pymysql.connect(host='127.0.0.1', port=3306, user='root', password='root', db='data')
    
    cursor = db.cursor()
    for i in range(len(lis)):
        cols = ", ".join('`{}`'.format(k) for k in lis[i].keys())
        print(cols)  # '`name`, `age`'
    
        val_cols = ', '.join('%({})s'.format(k) for k in lis[i].keys())
        print(val_cols)  # '%(name)s, %(age)s'
    
        sql = "insert into yiqing(%s) values(%s)"
        res_sql = sql % (cols, val_cols)
        print(res_sql)  # 'insert into users(`name`, `age`) values(%(name)s, %(age)s)'
    
        cursor.execute(res_sql, lis[i])  # 将字典a传入
        db.commit()
    

     结果

  • 相关阅读:
    js(数组篇02)
    js 按键
    安装Java Decompiler
    移动端H5地图离线瓦片方案(1)(2)
    Visual Studio 2012 智能提示功能消失解决办法
    CSS 基础:文本和字体(4)<思维导图>
    CSS 基础:定位元素(3)<思维导图>
    CSS 基础:CSS 工作原理(2)<思维导图>
    CSS 基础:HTML 标记与文档结构(1)<思维导图>
    java定时器的几种用法[转]
  • 原文地址:https://www.cnblogs.com/1061321925wu/p/12286648.html
Copyright © 2011-2022 走看看