Mysql Programming CS 155P笔记(一) Storage Engines

What's the difference between MyISAM and InnoDB?

In deciding which data engine to use, it's important to understand how each one behaves, it's stengths, weaknesses and performance characteristics.

MyISAM is, as the acronym would indicate, designed for sequential or indexed access to data.   When adding data, it's generally appended to the end of the MyISAM data file.  When you delete rows, there will be gaps which may be later filled in by an insert of a row of the same or smaller size.  When altering a row which gets larger, the new row is appended to the end of the table and the previous locatino marked as available.   As a result, the order of rows returned in a "select *" statement can be different every time since it will generally return rows in the order they exist in the file as it scans the file sequentially.   Since all the data is stored in a single file with no real "record length", any updates or inserts on the table will lock the entire table.   This is referred to as a "table level lock" or "table locking".  Because the locks will temporarily block other statements, MyISAM aren't ideal for transactional loads or any data that is often updated or changed.  They're well suited for high read environments and especially when reading indexed data (selecting data with a key field in the where clause) like a Data Warehouse.

InnoDB balances high reliability with high performance.  It has "row level locking" meaning, you can update rows in the table while reading other rows in the table.   A select can happen at the same time an insert or update as long as the rows in the select statement are different.  Similarly, you can have multiple queries updating different rows simultaneously.

The idea of "row level locking" is a bit of a misnomer in the implementation of InnoDB.   The Engine actually stores and retrieves data in blocks.  The default block size is 16K.  So locks are actually at the "block" level rather than the "row" level.   If all your rows are exactly 16K in size then, most likely, you'll have individual rows locked as needed.  However, it's more likely you'll have multiple rows data stored in the same block so an update to 1 row may block a select on an adjacent row because it's locked as part of the data block.

You can configure the InnoDB engine block size.   It's beyond the scope of this class, but it's helpful if the InnoDB blocksize matches or is a multiple of the underlying filesystem's blocksize.   The default blocksize on Linux's EXT4 filesystem (the default and most common currently) is 4K, so for every 1 InnoDB block read/write, you'll have 4 filesystem block reads/writes.  This can have an affect on performance, especially if the filesystem doesn't allocate blocks next to eachother.  1 InnoDB block could be stored in 4 different parts of the disk/filesystem which will reduce performance.

Where does MySQL store data?

All of the data is stored in files on a filesystem.  MySQL doesn't currently support "raw" disks like Oracle and some other databases.   Each "databases" is a directory in the filesystem and each table will have 2 or 3 files in the filesystem.

MyISAM tables will have 3 files per table:

• tablename.frm: stores the table format
• tablename.myd: stores the data
• tablename.myi: stores the indexes

InnoDB tables will always have 1 or 2 files per table:

• tablename.frm: stores the table format
• tablename.ibd: stores the data and indexes if innodb_file_per_table configuation variable is set.

MySQL有多种存储引擎，MyISAM和InnoDB是其中常用的两种。这里介绍关于这两种引擎的一些基本概念（非深入介绍）。

MyISAM是MySQL的默认存储引擎，基于传统的ISAM类型，支持全文搜索，但不是事务安全的，而且不支持外键。每张MyISAM表存放在三个文件中：frm 文件存放表格定义；数据文件是MYD (MYData)；索引文件是MYI (MYIndex)。

InnoDB是事务型引擎，支持回滚、崩溃恢复能力、多版本并发控制、ACID事务，支持行级锁定（InnoDB表的行锁不是绝对的，如果在执行一个SQL语句时MySQL不能确定要扫描的范围，InnoDB表同样会锁全表，如like操作时的SQL语句），以及提供与Oracle类型一致的不加锁读取方式。InnoDB存储它的表和索引在一个表空间中，表空间可以包含数个文件。

主要区别：

• MyISAM是非事务安全型的，而InnoDB是事务安全型的。
• MyISAM锁的粒度是表级，而InnoDB支持行级锁定。
• MyISAM支持全文类型索引，而InnoDB不支持全文索引。
• MyISAM相对简单，所以在效率上要优于InnoDB，小型应用可以考虑使用MyISAM。
• MyISAM表是保存成文件的形式，在跨平台的数据转移中使用MyISAM存储会省去不少的麻烦。
• InnoDB表比MyISAM表更安全，可以在保证数据不会丢失的情况下，切换非事务表到事务表（alter table tablename type=innodb）。

应用场景：

• MyISAM管理非事务表。它提供高速存储和检索，以及全文搜索能力。如果应用中需要执行大量的SELECT查询，那么MyISAM是更好的选择。
• InnoDB用于事务处理应用程序，具有众多特性，包括ACID事务支持。如果应用中需要执行大量的INSERT或UPDATE操作，则应该使用InnoDB，这样可以提高多用户并发操作的性能。

常用命令：

　　（1）查看表的存储类型（三种）：

• show create table tablename
• show table status from  dbname  where name=tablename
• mysqlshow  -u user -p password --status dbname tablename

　　（2）修改表的存储引擎：

• alter table tablename type=InnoDB

　　（3）启动mysql数据库的命令行中添加以下参数使新发布的表都默认使用事务：

• --default-table-type=InnoDB

　　（4）临时改变默认表类型：

• set table_type=InnoDB
• show variables like 'table_type'

My favorite filesystem is ZFS.  I like it for a great many reasons, one of which is that it performs atomic writes.   It does this because it's a "copy on write" filesystem.  In other words, when a data block in ZFS is changed, it writes a new block and then marks the old one as free.  If the new block didn't complete, the old block is still in tact.  Once the new block is written, the old block is effectively erased.   So if your datbase crashes while writing an updated InnoDB block, the entire block will either be safe in the ZFS journal or safe on disk.  So we can turn off the double-write buffer.

ZFS has other features that I like for MySQL as well:

• transparent block level compression  (for the average InnoDB database can save up to 50% of the disk space)
  The assumption of compression is that CPU is faster than disk I/O.   It takes some time to compress the data, but it requires less time to write it to the disk.  Disk writes (insert/update) may take the same or less time than when the data isn't compressed, but reads are typically much faster since we have to read less data from a slow disk drive and decompression is a very fast operation.
  NOTE: InnoDB engine does offer compression, but it's generally best to do this at the filesystem level if possible.  There is a lot of overhead in InnoDB compression
• snapshots.   This is perhaps my favorite feature of ZFS and is a great value when it's used as a MySQL datadir.   ZFS can make instant and virually unlimited number of filesystem snapshots.   Since ZFS is a "copy on write" filesystem, the snapshots don't take up any space until the file data changes.  Then the live version of the filesystem continues to write new blocks and mark previous blocks as free, but the previous blocks remain in use by the snapshot.   When a snapshot is deleted, all the changed blocks that belong to it are then marked free.   So the cost of a snapshot is effectively moved form 'create' time to 'destroy' time.   Since ZFS has a "lazy reclaim" process that cleans up blocks marked free, the cost of destroying a snapshot is minimal but the space used is added back to the filesystem free space over time.

In MySQL, the best way to make a database backup is to "flush tables with read lock" then take a snapshot of the filesystem then "unlock tables;"  In a busy database with a slow snapshot mechanism (linux Logical Volume Manager [LVM] does slow snaphosts athe volume level inbetween the raw disk and the filesystem) it can make your database inaccessible for a long time.

Once you have your snapshot, simply copy the snapshot filesystem to your backup location.

On the MySQL servers I administer, I typically use ZFS to create hourly and daily snapshots.  I keep 24 hourly snapshots and 14 daily snapshots.   I then archive (via rsync) the daily snapshots to a backup server.  

In this case, if there's a sever server crash, I can recover the database from the bakup while losing <24 hours worth of data.  

关于正在进行的ZFS和数据库性能测试的信息，请参看zfs_and_databases。还可参见ZFS for Databases

概要说明

• 如果数据库针对I/O使用固定磁盘的块或记录尺寸，请将ZFS的recordsize属性设成与数据库一致。您可以在每个文件系统上做这一操作，即使是共享一个存储池的多个文件系统。
• 由于其copy-on-write设计，调小ZFS的recordsize是一种以牺牲批处理报告查询为代价而提高OLTP（On-Line Transaction Processing，联机事务处理）性能的方式。
• ZFS校验存于磁盘上的每个块（block）。这减轻了数据库层面上对校验数据的需要和额外的时间。若校验由ZFS代替了数据库的，所有的差异都可以在数据返回给应用程序之前被捕获和修正。ZFS在数据库上的性能是非常快的活动目标。保持对Solaris发行版的更新非常重要。

截至2007年7月，如下功能会对数据库性能产生影响：

• ZFS 放出多至35个并发I/O至每个顶级设备，且这可以导致服务的时间延长。
• ZFS对每一输入块进行一些多至64K的低级预取操作，这样做可令存储带宽饱和。详情请参见6437054号Bug以及这篇blog。
• 使用8K的预取并用5到10个并发I/O来帮助一些数据库的负载。对于调节这些值的做法请参看ZFS Evil Tuning Guide。这种调节需要有望在未来发行版中去除。

MySQL

参考：A look at MySQL on ZFS

• InnoDB:
• 限定ZFS ARC已在内存与动态重构（Dynamic Reconfiguration）建议中描述
• 关于对日志使用单独存储池，请参看上述Oracle事项。
• 请设置ZFS recordsize=8K|16K（注意：请在任何数据文件的创建之前做这一设置！）
• 之后请对数据和日志使用不同的路径（在mysql.conf）中设置
• MyISAM:
• 限定ZFS ARC已在内存与动态重构（Dynamic Reconfiguration）建议中描述
• 请为日志（WAL）创建独立的intent log。若您没有该功能（即您运行在Solaris 10发行版），那么请创建至少两个存储池，一个存数据，一个存日志（WAL）
• 关于对日志使用单独存储池，请参看上述Oracle事项。
• 请设置ZFS recordsize=8K|16K（注意：请在任何数据文件的创建之前做这一设置！）

请参阅一些在PostgreSQL和MySQL中用db_STRESS性能测试获得的真实结果。

Numeric:

A question might remain, what's the difference betwen a DECIMAL and a FLOATing point data type?

There's a pretty big difference which impacts both data ranges that are allowed how MySQL stores and manipulates each data type.

A DECIMAL data type is stored in a binary format and will only allow the specifc number of digits indicated.  For example if we define  a column as "wage DECIMAL(2.2)", the possible range is -99.99 to 99.99.   If someone is getting $99/hour and you want to give them a $2 raise you will have a problem.

The benefit of the DECIMAL data type is that you don't get floating point errors. 

选择小数的浮点运算时间如果要精确尽量选用DECIMAL，因为如果采用Float时数值越大产生的错误越大。

Strings

MySQL has a variety of data types to store text or binary data.

The ones that might immediately come to mind are CHAR and VARCHAR.  They both store character data but CHAR are limited to a maximum of 255 characters while VARCHAR can go up to 65,535 characters (depending on other column sizes and character set).  VARCHAR are stored as a 1 or 2 character prefix, followed by the character data.  As such, the minimum space consumed by a varchar value is 1 byte.  This would be the length (0) indicating an empty string.   Any non empty string would take up a minimum of 2 bytes.  1 for the length indicator, and 1 or more for the character data.  If the length of the string is longer than 255 bytes, then 2 bytes are required to store the length.  A string of 256 characters will, therefor, take up 258 bytes, while a string of 255 characters will take up 256 bytes. 

VARCHAR当小于256 characters时 1 bytes + 长度，当大于256 characters时 2 bytes + 长度

Date and Time

The DATETIME data type allows you to store full dates down to the second. The supported range is '1000-01-01 00:00:00' to '9999-12-31 23:59:59'.  This wouldn't be a good datatype to use to store details about the viking migration (years 400-800 if you were wondering).

DATETIME时间从1000-01-01 00:00:00' to '9999-12-31 23:59:59

Another common choice is TIMESTAMP.  A TIMESTAMP is the number of seconds since 1970-01-01 00:00:01.  Since a TIMESTAMP is a 4-byte data type, if you do some math, you'll determine that 4294967295  seconds from the beginning of 1970 is 2038-01-19 03:14:07.   So the last date we can currently store in a TIMESTAMP is early 2038.  This wouldn't be the right datatype if you were trying to figure out the date at which we might pay off the US Debt.

TIMESTAMP时间只从1970-01-01 00:00:01 - 2038-01-19 03:14:07

TIMESTAMP and DATETIME have a nother nice feature.  They can be used as an AUTO_UPDATE field.   In this case, whenever a column value in the row is updated, the TIMESTAMP field will be updated to include the CURRENT_TIMESTAMP (now()).  You can also set CURRENT_TIMESTAMP as a default value for these datatypes as well.

CREATE TABLE t1 (
  ts TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  dt DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);

使用上面的办法自动插入当前时间

Variables

SQL, like any programming language, has variables which allow us to store information in one statement and refer to it later in another statement.

• Variables are NOT global and exist only in a specific session.   When you're client disconnects, all the variables are erased.
• The @ sign is used to refer to a variable (similar to $ is some programming languages)
• Variable names are NOT case sensitive and can be a maximum of 64 characters.
• The names can be any string, if it's a quoted string, otherwise the variable name consists of alphanumeric characters, ., _, and $.
  For example, @'my-var', @"my-var", or @`my-var` are all valid, but @my-var is not.
• You initialize or change a variable using the SET keyword (SET @age=27)
• You can assign a value to a variable in a statement using the := as an assignment operator (SELECT `name`,@age := 27)
• Variables can refer to values of certain data types:
  • integer
  • decimal
  • floating-point
  • binary or nonbinary string
  • NULL



注意variables变量的定义方法

Assignment and Expressions

Assignment

While it was briefly mentioned in the previous section that you assign variables a value using the := operator (colon, equals), we didn't really understand WHERE we can assign variables in this manner.

We're used to simply using the = sign to assign variables.  In MySQL, the equals operator assignment is used in the SET SQL Statement.  You can SET  var = val.

However, we'll also often want to set variables within statements such as SELECT and UPDATE.   In these cases, we can't use the = sign since it is a comparison operator in these contexts, so we have to use something else the parser can understand.   They chose :=.

Like most programming languages, you can both read and assign a variable, like this:

如果使用Set可以使用 Set @var1=3，当变量使用在select语句中可以直接使用 :=3这样表示

SET @var1=3, @var2=2;
SELECT @var1 := @var2 := @var1 + @var2;
SELECT @var1, @var2;