zoukankan      html  css  js  c++  java
  • Python爬取疫情数据并存入mysql中

    import time, json, requests
    import pymysql
    
    url='https://view.inews.qq.com/g2/getOnsInfo?name=disease_h5&&callback=&_=%d'%int(time.time()*1000)
    data = json.loads(requests.get(url=url).json()['data'])
    lis=[]
    for m in range(len(data['areaTree'][0]['children'])):
        for n in range(len(data['areaTree'][0]['children'][m]['children'])):
            info={}
            info['pronvice']=data['areaTree'][0]['children'][m]['name']#省份
            info['city']=data['areaTree'][0]['children'][m]['children'][n]['name']#城市   len(data['areaTree'][0]['children'][0]['children'])
            info['total_confirm']=data['areaTree'][0]['children'][m]['children'][n]['total']['confirm']
            info['total_suspect']=data['areaTree'][0]['children'][m]['children'][n]['total']['suspect']
            info['total_dead']=data['areaTree'][0]['children'][m]['children'][n]['total']['dead']
            info['total_heal']=data['areaTree'][0]['children'][m]['children'][n]['total']['heal']
            info['today_confirm']=data['areaTree'][0]['children'][m]['children'][n]['today']['confirm']
            info['today_suspect']=data['areaTree'][0]['children'][m]['children'][n]['today']['suspect']
            info['today_dead']=data['areaTree'][0]['children'][m]['children'][n]['today']['dead']
            info['today_heal']=data['areaTree'][0]['children'][m]['children'][n]['today']['heal']
            lis.append(info)
    print(lis[0])
    print(len(lis))
    
    db =  pymysql.connect(host='127.0.0.1', port=3306, user='root', password='root', db='data')
    
    cursor = db.cursor()
    for i in range(len(lis)):
        cols = ", ".join('`{}`'.format(k) for k in lis[i].keys())
        print(cols)  # '`name`, `age`'
    
        val_cols = ', '.join('%({})s'.format(k) for k in lis[i].keys())
        print(val_cols)  # '%(name)s, %(age)s'
    
        sql = "insert into yiqing(%s) values(%s)"
        res_sql = sql % (cols, val_cols)
        print(res_sql)  # 'insert into users(`name`, `age`) values(%(name)s, %(age)s)'
    
        cursor.execute(res_sql, lis[i])  # 将字典a传入
        db.commit()
    

     结果

  • 相关阅读:
    easy-ui的data-options用法
    my_note
    定时器
    abp安装
    微信小程序
    几个免费的ui 后台
    abp创建实体的方法
    winform 开源项目
    func委托
    for update 锁行和锁表
  • 原文地址:https://www.cnblogs.com/1061321925wu/p/12286648.html
Copyright © 2011-2022 走看看