扩容Linux文件系统

扩容Linux文件系统

腾讯云 云硬盘扩容

https://cloud.tencent.com/product/cbs

https://cloud.tencent.com/document/product/362/6738

普通云硬盘（HDD Cloud Storage） 容量最大为16TB
高性能云硬盘（Premium Cloud Storage） 容量最大为4TB
SSD云硬盘（SSD Cloud Storage） 容量最大为4TB
单台虚拟机最多可挂载 10 块云盘，容量达 40TB。您可以轻松搭建大容量的文件系统，用于大数据、数据仓库、日志处理等业务。

请注意，由于MBR的限制，选择任何一种方式时，请保持任意分区的大小不超过2TB（若您扩容后的空间已经大于2TB则不可选择第二种方式。

一般系统分区方案
/boot 500M
/ 30G
SWAP 8G
/data 剩下空间

---

扩容分三步
1) 扩容实体云硬盘大小

2) 扩容分区

确定文件系统分区表形式
扩容分区
3) 扩容文件系统

---

GPT分区云硬盘扩容
umount 挂载点
parted [磁盘路径]
e2fsck -f /dev/vdb1
resize2fs /dev/vdb1
mount 分区路径 挂载点

# 具体命令
umount -lf /data
parted /dev/vdb 
e2fsck -f /dev/vdb1
resize2fs /dev/vdb1
mount /dev/vdb1  /data
或
echo "/dev/vdb  /data  ext4  defaults,noatime,nodiratime   0 0" >>/etc/fstab
mount -a

新空间增加到已有分区中(GPT分区格式)

查看数据盘信息

执行命令

parted 磁盘路径 print

确认云硬盘的容量变化。如在过程中收到如下提示,请输入Fix：

 

这里扩容后的云硬盘大小为107GB，已有分区的大小为10.7GB。

卸载已挂载数据盘

执行以下命令确认该云硬盘是否还有分区已挂载：

mount | grep '磁盘路径'

这里云硬盘上有一个分区(vdb1)挂载在/data上，需要将其解挂。

使用以下命令解挂：

umount 挂载点
本例中即执行umount /data进行卸载。

注：要将云硬盘上所有分区的文件系统都解挂，如vdb1、vdb2......
再次使用mount | grep '/dev/vdb'命令来确认此硬盘上所有分区的文件系统都已解挂。

 

数据盘分区

确认云硬盘所有分区均已卸载后，执行以下命令，将原分区删除并以同样的起始偏移新建一个分区：

parted  [磁盘路径]
接下来输入unit s，将显示和操纵单位变成sector（默认为GB），输入print来查看分区信息，记住已有分区的Start值。

删除分区并新建后，Start值必须与这个相同，否则数据将会丢失。

执行以下命令删除原有分区：

rm [分区Number]
由上图可知云硬盘上有一个分区，Number号为“1”，执行rm 1，结果如下图：

输入mkpart primary [原分区起始扇区] 100%

新建一个主分区。

本例中使用mkpart primary 2048s 100%，此主分区从第2048个扇区开始（必须与删除之前的分区一致），100%表示此分区到磁盘的最末尾。

如果出现如图状态请输入Ignore：

再次输入print可发现新分区已经新建成功，输入quit，即可退出parted工具：

检查扩容后分区的文件系统

使用以下命令检查扩容后的分区：

e2fsck -f 分区路径
前述步骤中本例已新建了分区1，使用e2fsck -f /dev/vdb1进行操作。结果如下：

扩容文件系统

执行以下命令进行分区上文件系统的扩容操作：

resize2fs 分区路径

挂载新分区

执行以下命令挂载分区：

mount 分区路径 挂载点
这里通过mount /dev/vdb1 /data

手动挂载新分区，并使用df -h命令查看，出现以下信息说明挂载成功，即可以查看到数据盘了。

---

MBR分区云硬盘扩容

下面两种情况都可以选择使用自动扩容工具（devresize.py）进行扩容
1、原有的硬盘（数据盘）只有一个MBR主分区并制作了文件系统
2、原有的硬盘（数据盘）没有分区，直接在此硬盘上制作了文件系统

自动扩容工具适用于Linux操作系统，用于将扩容时新扩的云硬盘存储空间添加到已存在的文件系统中，扩容能够成功必须满足下面3个条件：
1、文件系统是ext2/ext3/ext4,并且只有一个主分区没有其他主分区和扩展分区
2、当前文件系统不能有错误
3、扩容后的磁盘大小不超过2TB

umount 挂载点
wget -O /tmp/devresize.py http://mirrors.tencentyun.com/install/virts/devresize.py 下载一键扩容工具
python /tmp/devresize.py 硬盘路径 云硬盘，而不是分区名
mount 分区路径 挂载点

# 具体命令
umount -lf /data
wget -O /tmp/devresize.py http://mirrors.tencentyun.com/install/virts/devresize.py 
python /tmp/devresize.py  /dev/vdb
mount /dev/vdb1  /data
或
echo "/dev/vdb  /data  ext4  defaults,noatime,nodiratime   0 0" >>/etc/fstab
mount -a

新空间增加到已有分区中(MBR分区格式)

卸载正在使用的硬盘分区

执行以下命令卸载分区：

umount 挂载点

下载一键扩容工具

执行以下命令下载工具：

wget -O /tmp/devresize.py http://mirrors.tencentyun.com/install/virts/devresize.py
执行扩容工具

执行以下命令进行扩容：

python /tmp/devresize.py 硬盘路径
请注意，这里硬盘路径是需要扩容的云硬盘，而不是分区名。若您的文件系统在vdb1上，则应执行

python /tmp/devresize.py /dev/vdb

若输出“The filesystem on /dev/vdb1 is now XXXXX blocks long.“则表示扩容成功。

重新挂载扩容后的分区

执行以下命令挂载扩容后的分区：

mount 分区路径 挂载点
并通过以下命令查看扩容后的分区容量：

df -h
这里通过mount /dev/vdb1 /data命令手动挂载扩容后的分区(如果原先是没有分区的，执行mount /dev/vdb /data)，

用df -h命令查看，出现以下信息说明挂载成功，即可以查看到数据盘了:

再执行ll /data命令，可以查看到，扩容后原分区的数据没有丢失，新增加的存储空间已经扩容到文件系统中。

devresize.py

#!/usr/bin/env python
# coding: utf-8
# FileName: devresize.py
"""
It only handle the following two situations:
1. There is only one primary partiion in the disk with a format of ext2/3/4;
2. The disk is raw with a file system whose format is ext2/3/4.
"""

import struct
import array
import fcntl
import time
import sys
import os
import glob
import logging

BLKSSZGET = 0x1268
BLKGETSIZE = 0x1260
BLKRRPART = 0x125f
BLKGETSIZE64 = 0x80041272

logger = None


def read_ub(data):
    return struct.unpack('B', data[0])[0]


def read_us(data):
    return struct.unpack('<H', data[0:2])[0]


def read_ui(data):
    return struct.unpack('<I', data[0:4])[0]


def read_ul(data):
    return struct.unpack('<Q', data[0:8])[0]


def init_log():
    global logger
    log_file = 'devresize.log'
    fmt_file = '%(asctime)s - [%(levelname)-5.5s]- %(filename)s:%(lineno)s - %(message)s'
    fmt_stream = '[%(levelname)s] - %(message)s'
    logger = logging.getLogger('devresize')
    logger.setLevel(logging.DEBUG)

    file_handler = logging.FileHandler(log_file)
    file_handler.setLevel(logging.DEBUG)
    file_handler.setFormatter(logging.Formatter(fmt_file))
    logger.addHandler(file_handler)

    stream_handler = logging.StreamHandler()
    stream_handler.setLevel(logging.INFO)
    stream_handler.setFormatter(logging.Formatter(fmt_stream))
    logger.addHandler(stream_handler)


class PartitionEntry(object):
    PartitionTypes = {
        0x05: "Microsoft Extended",
        0x83: "Linux",
        0x85: "Linux Extended"
    }

    def __init__(self, data):
        self.data = data
        self.boot_sig = data[0]

        self.start_head, self.start_sector, self.start_cylinder = (
            PartitionEntry.get_hsc(data[1:1 + 3]))

        self.partition_type = read_ub(data[4])

        self.end_head, self.end_sector, self.end_cylinder = (
            PartitionEntry.get_hsc(data[5:5 + 3]))

        self.start_lba = read_ui(data[8:8 + 4])
        self.sector_num = read_ui(data[12:12 + 4])

        self.partition_type_name = PartitionEntry.PartitionTypes.get(self.partition_type, "other")

    @staticmethod
    def get_hsc(data):
        h, s, c = struct.unpack('BBB', data[0:3])
        c = (c | ((s & 0xC0) << 2))
        s = (s & 0x3F)
        return h, s, c

    @staticmethod
    def cal_hsc(sector, hh, ss):
        s = sector % ss + 1
        sector /= ss
        h = sector % hh
        sector /= hh
        c = sector & 0xFF
        s |= (sector >> 2) & 0xC0
        return h, s, c

    def vaild_type(self):
        return self.partition_type in self.PartitionTypes

    def isprimary(self):
        return self.partition_type == 0x83

    def __str__(self):
        if not self.vaild_type():
            print "%x" % self.partition_type
            return "This isn't a Linux Partition!"
        return """
        Start h,s,c: %u %u %u
        End h,s,c: %u %u %u
        Partition Type Name:%s
        Start LBA: %u
        Sector Number: %u
        """ % (self.start_head, self.start_sector, self.start_cylinder,
               self.end_head, self.end_sector, self.end_cylinder,
               self.partition_type_name, self.start_lba, self.sector_num)


class MBR(object):
    def __init__(self, data):
        self.data = data
        self.boot_code = data[:446]
        self.mbr_sig = data[510:512]

        if self.check_mbr_sig():
            self.partitions = ([PartitionEntry(data[446 + 16 * i:446 + 16 * (i + 1)])
                                for i in range(0, 4)])
        else:
            self.partitions = None

        if self.partitions is not None:
            self.vaild_part_num = len(filter(lambda x: x.vaild_type(), self.partitions))
        else:
            self.vaild_part_num = 0

        self.device_heads = 0
        self.device_sectors = 0

        self.cal_device_hs()

    def cal_device_hs(self):
        if self.partitions is not None and self.vaild_part_num == 1:
            self.device_heads = self.partitions[0].end_head + 1
            self.device_sectors = self.partitions[0].end_sector & 0x3F

    def check_mbr_sig(self):
        mbr_sig = read_us(self.mbr_sig)
        if mbr_sig == 0xAA55:
            return True
        else:
            return False


def get_device_size(fd):
    buf = array.array('c', [chr(0)] * 8)
    fcntl.ioctl(fd, BLKSSZGET, buf, True)
    logical_sector_size = read_ul(buf)

    buf = array.array('c', [chr(0)] * 8)
    try:
        fcntl.ioctl(fd, BLKGETSIZE, buf, True)
        device_size = read_ul(buf) * 512
    except IOError:
        fcntl.ioctl(fd, BLKGETSIZE64, buf, True)
        device_size = read_ul(buf)
    device_sector_number = device_size / logical_sector_size
    logger.debug(
        '''device_size:%d
        device_sector_number:%d
        logical_sector_size:%d''' % (device_size, device_sector_number, logical_sector_size))
    return device_size, device_sector_number, logical_sector_size


def backup_mbr(data):
    bak_name = '/tmp/MBR_%s_bak' % time.strftime("%Y-%m-%d_%X", time.localtime())
    bak_file = open(bak_name, 'w')
    bak_file.write(data)
    bak_file.close()
    logger.info("Backup MBR to %s" % bak_name)
    return bak_name


def is_first_start():
    bak_file_list = glob.glob('/tmp/MBR_%s*bak' % time.strftime("%Y-%m-%d", time.localtime()))
    return len(bak_file_list) == 0


def cal_new_part(part_data, mbr, start_lab, new_end):
    device_heads, device_sectors = mbr.device_heads, mbr.device_sectors

    new_partition_sector_num = new_end - start_lab + 1
    begin_h, begin_s, begin_c = PartitionEntry.cal_hsc(start_lab, device_heads, device_sectors)
    end_h, end_s, end_c = PartitionEntry.cal_hsc(new_end, device_heads, device_sectors)

    new_part_data = list(part_data[:])
    new_part_data[1:1 + 3] = list(struct.pack('BBB', begin_h, begin_s, begin_c))
    new_part_data[5:5 + 3] = list(struct.pack('BBB', end_h, end_s, end_c))
    new_part_data[0xc:] = list(
        struct.pack('BBBB', (new_partition_sector_num & 0xff), ((new_partition_sector_num >> 8) & 0xff),
                    ((new_partition_sector_num >> 16) & 0xff), ((new_partition_sector_num >> 24) & 0xff)))

    logger.debug("""
    Start h,s,c: %u %u %u
    End h,s,c: %u %u %u
    Partition Type Name:%s
    Start LBA: %u
    Sector Number: %u
    """ % (begin_h, begin_s, begin_c,
           end_h, end_s, end_c,
           mbr.partitions[0].partition_type_name,
           mbr.partitions[0].start_lba,
           new_partition_sector_num))
    return new_part_data


# filesystem type must be ext2/3/4
def check_format(dev):
    blkid_ret = os.popen('blkid %s' % dev)
    s = blkid_ret.read()
    if blkid_ret.close() is not None:
        return False
    return True in [i in s for i in ['ext2', 'ext3', 'ext4']]


def part_probe(fd):
    logger.debug('part_probe')
    fd.flush()
    fcntl.ioctl(fd, BLKRRPART)


def resize2fs(dev):
    ret = os.system('e2fsck -f %s' % dev)
    logger.debug('e2fsck ret is %d' % ret)
    if ret not in (0, 1):
        raise RuntimeError('e2fsck failed!!')
    if ret == 1:
        logger.info('File system errors corrected')

    ret = os.system('resize2fs %s' % dev)
    logger.debug('resize2fs ret is %d' % ret)
    if ret != 0:
        raise RuntimeError('resize2fs failed!!')


def check_mount(target_dev):  # target_dev is mounted!
    return os.system('mount | grep "%s " > /dev/null' % target_dev) == 0


def write_mbr(fd, mbr_data):
    fd.seek(0)
    fd.write(mbr_data)
    part_probe(fd)
    fd.close()
    time.sleep(1)


def check_arg(device):
    return not device[-1].isdigit()


def get_disk_path(partation_name):
    for i, ch in enumerate(os.path.basename(partation_name)[::-1]):
        if not ch.isdigit():
            return partation_name[::-1][i::][::-1]
    logger.error("invalid para %s" % partation_name)
    raise Exception("invalid para %s" % partation_name)


def main():
    if len(sys.argv) < 2:
        print "Usage: %s block_device" % sys.argv[0]
        sys.exit(1)

    init_log()
    logger.debug("user input:%s" % ' '.join(sys.argv))
    device = sys.argv[1]

    if not check_arg(device):
        logger.error("The argument should be a whole disk not a partation!like %s" % get_disk_path(device))
        sys.exit(1)

    if not os.access(device, os.W_OK):
        logger.error("Permission denied")
        sys.exit(1)

    fd = open(device, 'r+')

    data = fd.read(512)
    mbr = MBR(data)

    device_size, device_sector_number, logical_sector_size = get_device_size(fd)

    if mbr.vaild_part_num > 1:
        logger.error("Disk %s have multi partition." % device)
        sys.exit(1)

    target_partition = ''
    resize_part_flag = True

    if mbr.vaild_part_num == 1:  # only one partition, which is the primary partition
        if not mbr.partitions[0].isprimary():  # and the filesystem type is ext2/3/4.
            logger.error("Must be primary partition.")
        resize_part_flag = True
        if device[-1].isdigit():
            target_partition = device + 'p1'  # ex: /dev/nbd0 -> /dev/nbd0p1
        else:
            target_partition = device + '1'  # ex: /dev/vdb -> /dev/vdb1
        logger.debug(mbr.partitions[0])

    if mbr.vaild_part_num == 0:  # no partition but whole disk is ext2/3/4
        resize_part_flag = False
        target_partition = device

    logger.debug('target_partition:%s' % target_partition)

    if not check_format(target_partition):
        logger.error("Only can process ext2/3/4.")
        sys.exit(1)

    if check_mount(target_partition):
        logger.error("Target device %s must be unmounted." % target_partition)
        sys.exit(1)

    logger.info("It will resize (%s).
"
                "This operation may take from several minutes to several hours, continue? [Y/n]"
                % target_partition)
    user_input = raw_input()

    if user_input.lower() != 'y' and user_input != '':
        logger.warn("User input neither 'y' nor '[Enter]',exit.")
        sys.exit(1)

    if not is_first_start():
        logger.warn('We find some MBR backup file in /tmp,maybe the MBR is already changed,'
                    'do you want to just resize the filesystem? [Y/n]')
        user_input = raw_input()
        if user_input.lower() == 'y' or user_input == '':
            resize_part_flag = False

    if resize_part_flag:
        logger.debug("Begin to change the partation")
        if (mbr.partitions[0].start_lba + mbr.partitions[0].sector_num) == device_sector_number:
            logger.error("No free sectors available.")
            sys.exit(1)
        if mbr.partitions[0].sector_num > 0xFFFFFFFF * 512 / logical_sector_size:
            logger.error("Can't process the partition which have exceeded 2TB.")
            sys.exit(1)
        new_start_sector = mbr.partitions[0].start_lba
        new_end_sector = device_sector_number - 1
        if (new_end_sector - new_start_sector + 1) * logical_sector_size > 0xFFFFFFFF * 512:
            user_input = raw_input("The size of this disk is %.2fTB (%d bytes).
"
                                   "But DOS partition table format can not be used on drives for volumes larger than 2TB (2199023255040 bytes).
"
                                   "Do you want to resize (%s) to 2TB?[Y/n]"
                                   % (round(device_size / 1024.0 / 1024 / 1024 / 1024, 2), device_size,
                                      target_partition))
            if user_input.lower() != 'y' and user_input != '':
                logger.warn("User input neither 'y' nor '[Enter]',exit.")
                sys.exit(1)
            new_end_sector = 0xFFFFFFFF * 512 / logical_sector_size + new_start_sector - 1

        new_mbr_data = list(data)[:]
        new_mbr_data[446:446 + 16] = cal_new_part(data[446:446 + 16], mbr,
                                                  new_start_sector, new_end_sector)

        backup_mbr(data)
    try:
        if resize_part_flag:
            write_mbr(fd, ''.join(new_mbr_data))
        resize2fs(target_partition)
    except Exception, e:
        logger.error(e)
        logger.error('Some error occurred!!Maybe you should call the customer service staff.')


if __name__ == '__main__':
    main()

---

腾讯云主机做raid

https://cloud.tencent.com/document/product/362/2932

都是系统层面的raid

windows主机

windows2012R2：使用系统自带磁盘管理器里的 ，新建镜像卷功能，把两个弹性云硬盘连接到云主机，然后新建镜像卷做raid1

Linux主机

centos6：Linux内核提供了md模块在底层管理RAID设备，我们可以使用mdadm工具来调用md模块。安装mdadm（以CentOS为例）

f