zoukankan      html  css  js  c++  java
  • 【Python之路】第十九篇--Python操作MySQL

    本篇对于Python操作MySQL主要使用两种方式:

    • 原生模块 pymsql

    • ORM框架 SQLAchemy

    pymsql

    pymsql是Python中操作MySQL的模块,其使用方法和MySQLdb几乎相同。

    下载安装

    pip3 install pymysql

    使用操作

    1、执行SQL

    # 创建连接
    conn = pymysql.connect(host='127.0.0.1', port=3306, user='root', passwd='123456', db='db1',charset='utf8')
    # 创建游标
    cursor = conn.cursor()
    
    # 执行SQL,并返回受影响行数
    effect_row = cursor.execute("update hosts set host = '1.1.1.2' where nid > %s", (1,))
      
    # 执行SQL,并返回受影响行数
    effect_row = cursor.executemany("insert into hosts(host,color_id)values(%s,%s)", [("1.1.1.11",1),("1.1.1.11",2)])
    
    # 提交,不然无法保存新建或者修改的数据
    conn.commit()
      
    # 关闭游标
    cursor.close()
    # 关闭连接
    conn.close()

    增,删,改需要执行 conn.commit()

    2、获取新创建数据自增ID

    import pymysql
      
    conn = pymysql.connect(host='127.0.0.1', port=3306, user='root', passwd='123', db='t1')
    cursor = conn.cursor()
    cursor.executemany("insert into hosts(host,color_id)values(%s,%s)", [("1.1.1.11",1),("1.1.1.11",2)])
    conn.commit()
    cursor.close()
    conn.close()
      
    # 获取最新自增ID  => 如果插入多条,只能拿到最后一条id
    new_id = cursor.lastrowid

    3、获取查询数据

    import pymysql
      
    conn = pymysql.connect(host='127.0.0.1', port=3306, user='root', passwd='123', db='t1')
    cursor = conn.cursor()
    cursor.execute("select * from hosts")
      
    # 获取第一行数据
    row_1 = cursor.fetchone()
    # => 再次执行:cursor.fetchone() 获得下一条数据,没有时为None
    
    # 获取前n行数据
    # row_2 = cursor.fetchmany(n)
    # ==> 执行了n次fetchone()
    
    # 获取所有数据
    # row_3 = cursor.fetchall()
      
    conn.commit()
    cursor.close()
    conn.close()

    注:在fetch数据时按照顺序进行,可以使用cursor.scroll(num,mode)来移动游标位置,如:

    • cursor.scroll(-1,mode='relative')  # 相对当前位置移动

    • cursor.scroll(2,mode='absolute') # 相对绝对位置移动

    4、fetch数据类型

      关于默认获取的数据是元祖类型,如果想要或者字典类型的数据,即:

    import pymysql
      
    conn = pymysql.connect(host='127.0.0.1', port=3306, user='root', passwd='123456', db='t1')
      
    # 游标设置为字典类型
    cursor = conn.cursor(cursor=pymysql.cursors.DictCursor)
    
    row = cursor.execute("select * from user")
      
    result = cursor.fetchone()
    print(result)
    
    conn.commit()
    cursor.close()
    conn.close()

    补充:

    1.SQL注入

    由于字符串拼接出现注入
    "select name from user where name='%s' and password ='%s' " %(username,password)
    pymysql 提供了转义功能:
    "select name from user where name=%s and password =%s ",( username,password ) 

    SQLAchemy

    SQLAlchemy是Python编程语言下的一款ORM框架,该框架建立在数据库API之上,使用关系对象映射进行数据库操作,

    简言之便是:将对象转换成SQL,然后使用数据API执行SQL并获取执行结果。

    ORM:

    ORM框架的作用就是把数据库表的一行记录与一个对象互相做自动转换。 正确使用ORM的前提是了解关系数据库的原理。

    ORM就是把数据库表的行与相应的对象建立关联,互相转换。 由于关系数据库的多个表还可以用外键实现一对多、多对多等关联,

    相应地, ORM框架也可以提供两个对象之间的一对多、多对多等功能。

    SQLAlchemy:

    本身无法操作数据库,其必须以pymsql等第三方插件,

    Dialect用于和数据API进行交流,根据配置文件的不同调用不同的数据库API,从而实现对数据库的操作,如:

    MySQL-Python
        mysql+mysqldb://<user>:<password>@<host>[:<port>]/<dbname>
      
    pymysql
        mysql+pymysql://<username>:<password>@<host>/<dbname>[?<options>]
      
    MySQL-Connector
        mysql+mysqlconnector://<user>:<password>@<host>[:<port>]/<dbname>
      
    cx_Oracle
        oracle+cx_oracle://user:pass@host:port/dbname[?key=value&key=value...]
      
    更多详见:http://docs.sqlalchemy.org/en/latest/dialects/index.html
    
    #!/usr/bin/env python
    # -*-coding:utf-8 -*-
    
    from sqlalchemy import create_engine,and_,or_,func,Table
    from sqlalchemy.ext.declarative import declarative_base
    from sqlalchemy import Column, Integer, String,ForeignKey
    from  sqlalchemy.orm import sessionmaker,relationship
    
    engine = create_engine("mysql+pymysql://root:123456@127.0.0.1:3306/t1?charset=utf8", max_overflow=5)
    
    Base = declarative_base()
    
    
    def init_db():
        Base.metadata.create_all(engine)   
    
    def drop_db():
        Base.metadata.drop_all(engine)    

    一、底层处理

    使用 Engine/ConnectionPooling/Dialect 进行数据库操作,Engine使用ConnectionPooling连接数据库,然后再通过Dialect执行SQL语句。

    #!/usr/bin/env python
    # -*- coding:utf-8 -*-
    from sqlalchemy import create_engine
     
     
    engine = create_engine("mysql+pymysql://root:123@127.0.0.1:3306/t1", max_overflow=5)
     
    # 执行SQL
    # cur = engine.execute(
    #     "INSERT INTO hosts (host, color_id) VALUES ('1.1.1.22', 3)"
    # )
     
    # 新插入行自增ID
    # cur.lastrowid
     
    # 执行SQL
    # cur = engine.execute(
    #     "INSERT INTO hosts (host, color_id) VALUES(%s, %s)",[('1.1.1.22', 3),('1.1.1.221', 3),]
    # )
     
     
    # 执行SQL
    # cur = engine.execute(
    #     "INSERT INTO hosts (host, color_id) VALUES (%(host)s, %(color_id)s)",
    #     host='1.1.1.99', color_id=3
    # )
     
    # 执行SQL
    # cur = engine.execute('select * from hosts')
    # 获取第一行数据
    # cur.fetchone()
    # 获取第n行数据
    # cur.fetchmany(3)
    # 获取所有数据
    # cur.fetchall()
    View Code

    二、ORM功能使用

    使用 ORM/Schema Type/SQL Expression Language/Engine/ConnectionPooling/Dialect 所有组件对数据进行操作。

    根据类创建对象,对象转换成SQL,执行SQL。处理中文数据时,在连接数据库时要加上   ?charset=utf8

    1.创建表

    #!/usr/bin/env python
    # -*- coding:utf-8 -*-
    from sqlalchemy.ext.declarative import declarative_base
    from sqlalchemy import Column, Integer, String, ForeignKey, UniqueConstraint, Index
    from sqlalchemy.orm import sessionmaker, relationship
    from sqlalchemy import create_engine
    
    engine = create_engine("mysql+pymysql://root:123@127.0.0.1:3306/t1", max_overflow=5)
    
    Base = declarative_base()
    
    # 创建单表
    class Users(Base):
    
        __tablename__ = 'users'
    
        id = Column(Integer, primary_key=True)
        name = Column(String(32))
        extra = Column(String(16))
    
        # 设置索引:
        __table_args__ = (
        UniqueConstraint('id', 'name', name='uix_id_name'),
            Index('ix_id_name', 'name', 'extra'),
        )
    
        # 输出Users对象时,调用:
        def __repr__(self):
            return "%s-%s-%s" % (self.id, self.name , self.extra)
    
    
    def init_db():
        Base.metadata.create_all(engine)   #创建表 
    
    def drop_db():
        Base.metadata.drop_all(engine)    #删除表 
    View Code

    一对多

    # 一对多
    class Favor(Base):
        __tablename__ = 'favor'
        nid = Column(Integer, primary_key=True)
        caption = Column(String(50), default='red', unique=True)
    
    
    class Person(Base):
        __tablename__ = 'person'
        nid = Column(Integer, primary_key=True)
        name = Column(String(32), index=True, nullable=True)
        favor_id = Column(Integer, ForeignKey("favor.nid"))   # 外键
    View Code

    多对多

    # 多对多
    class Group(Base):
        __tablename__ = 'group'
        id = Column(Integer, primary_key=True)
        name = Column(String(64), unique=True, nullable=False)
        port = Column(Integer, default=22)
    
    
    class Server(Base):
        __tablename__ = 'server'
    
        id = Column(Integer, primary_key=True, autoincrement=True)
        hostname = Column(String(64), unique=True, nullable=False)
    
    
    class ServerToGroup(Base):
        __tablename__ = 'servertogroup'
        nid = Column(Integer, primary_key=True, autoincrement=True)
        server_id = Column(Integer, ForeignKey('server.id'))
        group_id = Column(Integer, ForeignKey('group.id'))
    View Code

    2.操作表

    #!/usr/bin/env python
    # -*- coding:utf-8 -*-
    from sqlalchemy.ext.declarative import declarative_base
    from sqlalchemy import Column, Integer, String, ForeignKey, UniqueConstraint, Index
    from sqlalchemy.orm import sessionmaker, relationship
    from sqlalchemy import create_engine
    
    engine = create_engine("mysql+pymysql://root:123@127.0.0.1:3306/t1", max_overflow=5)
    
    #生成一个SQLORM基类
    Base = declarative_base()
    
    # 创建单表
    class Users(Base):
        __tablename__ = 'users'
        id = Column(Integer, primary_key=True)
        name = Column(String(32))
        extra = Column(String(16))
    
        __table_args__ = (
        UniqueConstraint('id', 'name', name='uix_id_name'),
            Index('ix_id_name', 'name', 'extra'),
        )
    
        def __repr__(self):
            return "%s-%s" %(self.id, self.name)
    
    # 一对多
    class Favor(Base):
        __tablename__ = 'favor'
        nid = Column(Integer, primary_key=True)
        caption = Column(String(50), default='red', unique=True)
    
        def __repr__(self):
            return "%s-%s" %(self.nid, self.caption)
    
    class Person(Base):
        __tablename__ = 'person'
        nid = Column(Integer, primary_key=True)
        name = Column(String(32), index=True, nullable=True)
        favor_id = Column(Integer, ForeignKey("favor.nid"))
        # 与生成表结构无关,仅用于查询方便
        favor = relationship("Favor", backref='pers')
    
    # 多对多
    class ServerToGroup(Base):
        __tablename__ = 'servertogroup'
        nid = Column(Integer, primary_key=True, autoincrement=True)
        server_id = Column(Integer, ForeignKey('server.id'))
        group_id = Column(Integer, ForeignKey('group.id'))
        group = relationship("Group", backref='s2g')
        server = relationship("Server", backref='s2g')
    
    class Group(Base):
        __tablename__ = 'group'
        id = Column(Integer, primary_key=True)
        name = Column(String(64), unique=True, nullable=False)
        port = Column(Integer, default=22)
        # group = relationship('Group',secondary=ServerToGroup,backref='host_list')
    
    
    class Server(Base):
        __tablename__ = 'server'
    
        id = Column(Integer, primary_key=True, autoincrement=True)
        hostname = Column(String(64), unique=True, nullable=False)
    
    
    
    def init_db():
        Base.metadata.create_all(engine)
    
    
    def drop_db():
        Base.metadata.drop_all(engine)
    
    # 这两行触发sessionmaker类下的__call__方法,return得到 Session实例,赋给变量session,
    # 所以session可以调用Session类下的add,add_all等方法
    
    Session = sessionmaker(bind=engine)
    session = Session()
    View Code

    .增

    obj = Users(name="alex0", extra='sb')
    session.add(obj)
    session.add_all([
        Users(name="alex1", extra='sb'),
        Users(name="alex2", extra='sb'),
    ])
    
    session.commit()
    View Code

    .删

    session.query(Users).filter(Users.id > 2).delete()
    session.commit()
    View Code

    .改

    session.query(Users).filter(Users.id > 2).update({"name" : "099"})
    session.query(Users).filter(Users.id > 2).update({Users.name: Users.name + "099"}, synchronize_session=False)
    session.query(Users).filter(Users.id > 2).update({"num": Users.num + 1}, synchronize_session="evaluate")
    session.commit()
    View Code

    .查

    ret = session.query(Users).all()
    ret = session.query(Users.name, Users.extra).all()
    ret = session.query(Users).filter_by(name='alex').all()
    ret = session.query(Users).filter_by(name='alex').first()
    View Code

    3.更多查询方法:

    #!/usr/bin/env python
    # -*-coding:utf-8 -*-
    
    
    from sqlalchemy import create_engine,and_,or_,func,Table
    from sqlalchemy.ext.declarative import declarative_base
    from sqlalchemy import Column, Integer, String,ForeignKey
    from  sqlalchemy.orm import sessionmaker,relationship
    
    
    engine = create_engine('mysql+pymysql://root@127.0.0.1:3307/day40?charset=utf8')
    
    Base = declarative_base()
    
    class Man_To_Woman(Base):
    
        __tablename__ = 'man_to_woman'
        nid = Column(Integer,primary_key=True)
        man_id = Column(Integer,ForeignKey('man.nid'))
        woman_id = Column(Integer,ForeignKey('woman.nid'))
    
    class Man(Base):
    
        __tablename__ = 'man'
        nid = Column(Integer,primary_key=True)
        name = Column(String(20),nullable=False)
        woman = relationship("Woman", secondary=Man_To_Woman.__table__)
    
    class Woman(Base):
    
        __tablename__ = 'woman'
        nid = Column(Integer,primary_key=True)
        name = Column(String(20),nullable=False)
        man = relationship("Man",secondary=Man_To_Woman.__table__)
    
    
    
    Base.metadata.create_all(engine)
    
    MySession = sessionmaker(engine)
    session = MySession()
    事例: 表结构

    1.filter_by( ... )  填写键值对方式

    ret = session.query(Man).filter_by(name='alex').first()
    print(ret.nid,ret.name)

    2.filter    填写条件判断

    ret = session.query(Man).filter(Man.name=='eric').first()
    
    ret = session.query(Man).filter(Man.name=='eric' , Man.nid > 0).first()
    
    row = session.query(Man).filter(Man.nid.between(1,4)).all()
    
    ret = session.query(Users).filter(Users.id.in_(session.query(Users.id).filter_by(name='eric'))).all()

    3.and_  or_  条件判断

    from sqlalchemy import and_, or_
    
    ret = session.query(Man).filter(and_(Man.name == 'eric',Man.nid == 2)).first()
    ret = session.query(Man).filter(or_(Man.name == 'eric',Man.nid == 2)).first()

    4.~   取反

    ret = session.query(Man).filter(Man.nid.in_([2,3])).first()
    ret = session.query(Man).filter(~Man.nid.in_([2,3])).first()

    5.like + %   通配符

    ret = session.query(Man).filter(Man.name.like('%x')).first()
    ret = session.query(Man).filter(~Man.name.like('%x')).first()

    6.切片  限制 ( 序号,前闭后开 )

    row = session.query(Man)[1:3]
    for ret in row:
        print(ret.nid, ret.name)
    
    row = session.query(Man).limit(3).offset(1)

    7.order_by   排序

    row = session.query(Man).order_by(Man.nid.desc()).all()
    row = session.query(Man).order_by(Man.nid.asc()).all()

    8.group_by  分组

    row = session.query(func.count('*')).select_from(Man).all()
    row = session.query(func.count('*')).filter(Man_To_Woman.nid > 1).all()
    row = session.query(func.count('*')).select_from(Man_To_Woman).group_by(Man_To_Woman.man_id).all()
    row = session.query(func.count('*')).select_from(Man_To_Woman).group_by(Man_To_Woman.man_id).limit(1).all()
    
    row = session.query(Man_To_Woman).group_by(Man_To_Woman.man_id).all()
    
    ret = session.query(Users).group_by(Users.extra).all()
    ret = session.query(
        func.max(Users.id),
        func.sum(Users.id),
        func.min(Users.id)).group_by(Users.name).all()
    
    ret = session.query(
        func.max(Users.id),
        func.sum(Users.id),
        func.min(Users.id)).group_by(Users.name).having(func.min(Users.id) >2).all()

    9.join  连表

    row = session.query(Users, Favor).filter(Users.id == Favor.nid).all()
    
    ret = session.query(Son).join(Father).all()
    
    ret = session.query(Son).join(Father, isouter=True).all()

    10.union  组合

    q1 = session.query(Users.name).filter(Users.id > 2)
    q2 = session.query(Favor.caption).filter(Favor.nid < 2)
    ret = q1.union(q2).all()
    
    q1 = session.query(Users.name).filter(Users.id > 2)
    q2 = session.query(Favor.caption).filter(Favor.nid < 2)
    ret = q1.union_all(q2).all()

    更多功能参见文档,猛击这里下载PDF

    补充  Relationship:

    • 改变数据输出的方式:可以在表的类中定义一个特殊成员:__repr__, return一个自定义的由字符串拼接的数据连接方式.

    • 数据库中表关系之间除了MySQL中标准的外键(ForeignKey)之外,还可以创建一个虚拟的关系,比如 group = relationship("Group",backref='uuu'),一般此虚拟关系与foreignkey一起使用.

    relationship : 通过relatioinship 找到绑定关系的数据 !!!

    一对多,连表操作:

    class Father(Base):
    
        __tablename__ ='father'
        nid = Column(Integer,primary_key=True)
        name = Column(String(32))
        son = relationship('Son')
    
    
    class Son(Base):
    
        __tablename__ = 'son'
        nid = Column(Integer, primary_key=True)
        name = Column(String(32))
        father_id = Column(Integer,ForeignKey('father.nid'))
        father = relationship('Father')
    表结构

    正向查询:

    需求:查询Son表中所有数据,并且显示对应的Father表中的数据.

    ret = session.query(Son).all()
    for obj in ret:
        print(obj.nid,obj.name,obj.father_id,obj.father.name)

    反向查询:

    需求:查询Father表中, 属于 alvin 的所有儿子Son.

    obj = session.query(Father).filter(Father.name=='alvin').first()
    
    row = obj.son
    for ret in row:
        print(ret.nid,ret.name,ret.father.name)

    多对多,连表操作:

    #!/usr/bin/env python
    # -*-coding:utf-8 -*-
    
    from sqlalchemy import create_engine,and_,or_,func,Table
    from sqlalchemy.ext.declarative import declarative_base
    from sqlalchemy import Column, Integer, String,ForeignKey
    from  sqlalchemy.orm import sessionmaker,relationship
    
    
    engine = create_engine('mysql+pymysql://root@127.0.0.1:3307/day40?charset=utf8')
    
    Base = declarative_base()
    
    class Man_To_Woman(Base):
    
        __tablename__ = 'man_to_woman'
        nid = Column(Integer,primary_key=True)
        man_id = Column(Integer,ForeignKey('man.nid'))
        woman_id = Column(Integer,ForeignKey('woman.nid'))
    
    class Man(Base):
    
        __tablename__ = 'man'
        nid = Column(Integer,primary_key=True)
        name = Column(String(20),nullable=False)
        woman = relationship("Woman", secondary=Man_To_Woman.__table__)
    
    class Woman(Base):
    
        __tablename__ = 'woman'
        nid = Column(Integer,primary_key=True)
        name = Column(String(20),nullable=False)
        man = relationship("Man",secondary=Man_To_Woman.__table__)
    
    
    
    Base.metadata.create_all(engine)
    
    MySession = sessionmaker(engine)
    session = MySession()
    表结构

    正,反向操作: 

    1.alex的所有女人

    2.凤姐的所有男人

    man1 = session.query(Man).filter(Man.name=='alex').first()
    print(man1)
    for ret in man1.woman:
        print(ret.nid,ret.name)
    
    woman1 = session.query(Woman).filter(Woman.name=='fengjie').first()
    print(woman1)
    for ret in woman1.man:
        print(ret.nid,ret.name)

    relatioinship 语句的简写:  ,我添加到Man表中

    woman = relationship("Woman", secondary=Man_To_Woman.__table__,backref='man')

    1   关于 session.add   session.query   session.commit的顺序问题?

    在同一个会话中, insert into table (xxxx)后,可以接着 select * from xxx; 查询到刚刚插入的数据;

    只是不能在其他会话,比如我另开一个客户端去连接数据库不能查询到刚刚插入的数据。

    这个数据已经到数据库。值是数据库吧这个数据给锁了。只有插入数据的那个session可以查看到,其他的session不能查看到,可以理解提交并解锁吧。

  • 相关阅读:
    Springboot 之 自定义配置文件及读取配置文件
    SQLSERVER系统视图 sql server系统表详细说明
    MySQL Workbench建表时 PK NN UQ BIN UN ZF AI 的含义
    使用Ecplise git commit时出现"There are no stages files"
    maven添加sqlserver的jdbc驱动包
    java将XML文档转换成json格式数据
    java将XML文档转换成json格式数据
    cannot be resolved. It is indirectly referenced from required .class files
    org.codehaus.jackson.map.JsonMappingException: Can not construct instance of java.util.Date from String value '2012-12-12 12:01:01': not a valid representation (error: Can not parse date "2012-12-
    @Autowired注解和静态方法 NoClassDefFoundError could not initialize class 静态类
  • 原文地址:https://www.cnblogs.com/5poi/p/6418809.html
Copyright © 2011-2022 走看看