-
1.理解块存储服务
操作系统获得存储空间的方式一般有两种:
通过某种协议(SAS,SCSI,SAN,iSCSI 等)挂接裸硬盘,然后分区、格式化、创建文件系统;或者直接使用裸硬盘存储数据(数据库)
通过 NFS、CIFS 等 协议,mount 远程的文件系统
第一种裸硬盘的方式叫做 Block Storage(块存储),每个裸硬盘通常也称作 Volume(卷)
第二种叫做文件系统存储。NAS 和 NFS 服务器,以及各种分布式文件系统提供的都是这种存储。
OpenStack 提供 Block Storage Service 的是 Cinder,其具体功能是:
①提供 REST API 使用户能够查询和管理 volume、volume snapshot 以及 volume type
②提供 scheduler 调度 volume 创建请求,合理优化存储资源的分配
③通过 driver 架构支持多种 back-end(后端)存储方式,包括 LVM,NFS,Ceph,GlusterFS
-
2.Cinder 架构以及块存储服务组件介绍
Cinder逻辑架构图:
cinder不是一个存储软件,而是属于管理存储软件。块存储服务通常包含下列组件:
cinder-api: 接受API请求,调用 cinder-volume 执行操作。
cinder-volume:与块存储服务和例如“cinder scheduler”的进程进行交互
cinder-scheduler守护进程:scheduler 通过调度算法选择最合适的存储节点创建 volume,和nova-scheduler类似
cinder-backup daemon:备份进程
消息队列:Cinder 各个子服务通过消息队列实现进程间通信和相互协作。因为有了消息队列,子服务之间实现了解耦,这种松散的结构也是分布式系统的重要特征。
Database:Cinder 有一些数据需要存放到数据库中,一般使用 MySQL。数据库是安装在控制节点上的,比如实验环境中,可以访问名称为“cinder”的数据库。
-
3.物理部署方案
Cinder 的服务会部署在两类节点上,控制节点和存储节点。
查看当前计算节点cinder的相关服务
[root@linux-node1 ~]# ps -e |grep cinder 1150 ? 01:19:09 cinder-api 2025 ? 00:02:15 cinder-api 3357 ? 00:17:16 cinder-schedule 24069 ? 00:27:01 cinder-volume 24089 ? 00:04:19 cinder-volume
cinder-api和cinder-schedule都部署在控制节点上,这无可厚非,思考:但是cinder-volume是否应该部署在存储节点上呢?
实际上,OpenStack是一个分布式系统,其每个组件的子服务都可以部署在任何节点上,只需要网络可通,这也表明了OpenStack的灵活性。无论哪个节点,只要运行了cinder-volume,它就是一个存储节点。同时,存储节点也可以部署其他组件的子服务。
-
4.安装组件并配置
(1)安装软件包
[root@linux-node1 ~]# yum install -y openstack-cinder
(2)修改配置:/etc/cinder/cinder.conf
[root@linux-node1 ~]# vim /etc/cinder/cinder.conf [database] <==配置数据库访问 connection = mysql+pymysql://cinder:cinder@192.168.56.11/cinder [DEFAULT] transport_url = rabbit://openstack:openstack@192.168.56.11 <==配置RabbitMQ消息队列访问权限 auth_strategy = keystone <==启动用keystoen认证 [keystone_authtoken] <==配置认证服务访问 auth_uri = http://192.168.56.11:5000 auth_url = http://192.168.56.11:35357 memcached_servers = 192.168.56.11:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = cinder password = cinder [oslo_concurrency] <==配置锁路径 lock_path = /var/lib/cinder/tmp 查看配置: [root@linux-node1 ~]# grep "^[a-z]" /etc/cinder/cinder.conf auth_strategy = keystone transport_url = rabbit://openstack:openstack@192.168.56.11 connection = mysql+pymysql://cinder:cinder@192.168.56.11/cinder auth_uri = http://192.168.56.11:5000 auth_url = http://192.168.56.11:35357 memcached_servers = 192.168.56.11:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = cinder password = cinder lock_path = /var/lib/cinder/tmp
-
5.初始化块存储服务的数据库,并验证
[root@linux-node1 ~]# su -s /bin/sh -c "cinder-manage db sync" cinder [root@linux-node1 ~]# mysql -h 192.168.56.11 -ucinder -pcinder -e "use cinder;show tables;"
-
6.配置计算服务以使用块设备存储
[root@linux-node1 ~]# vim /etc/nova/nova.conf
[cinder]
os_region_name=RegionOne
-
7.完成安装
[root@linux-node1 ~]# systemctl restart openstack-nova-api.service [root@linux-node1 ~]# systemctl enable openstack-cinder-api.service openstack-cinder-scheduler.service [root@linux-node1 ~]# systemctl start openstack-cinder-api.service openstack-cinder-scheduler.service
-
8.创建cinder和cinderv2服务实体
[root@linux-node1 ~]# openstack service create --name cinder > --description "OpenStack Block Storage" volume +-------------+----------------------------------+ | Field | Value | +-------------+----------------------------------+ | description | OpenStack Block Storage | | enabled | True | | id | c63c93beff014724b036a811e2b0d591 | | name | cinder | | type | volume | +-------------+----------------------------------+ [root@linux-node1 ~]# openstack service create --name cinderv2 > --description "OpenStack Block Storage" volumev2 +-------------+----------------------------------+ | Field | Value | +-------------+----------------------------------+ | description | OpenStack Block Storage | | enabled | True | | id | 6829915c1f9745409ca9bda364fe26c4 | | name | cinderv2 | | type | volumev2 | +-------------+----------------------------------+
-
9.创建块设备存储服务的API入口点
[root@linux-node1 ~]# openstack endpoint create --region RegionOne volume public http://192.168.56.11:8776/v1/%(tenant_id)s [root@linux-node1 ~]# openstack endpoint create --region RegionOne volume internal http://192.168.56.11:8776/v1/%(tenant_id)s [root@linux-node1 ~]# openstack endpoint create --region RegionOne volume admin http://192.168.56.11:8776/v1/%(tenant_id)s [root@linux-node1 ~]# openstack endpoint create --region RegionOne volumev2 public http://192.168.56.11:8776/v2/%(tenant_id)s [root@linux-node1 ~]# openstack endpoint create --region RegionOne volumev2 internal http://192.168.56.11:8776/v2/%(tenant_id)s [root@linux-node1 ~]# openstack endpoint create --region RegionOne volumev2 admin http://192.168.56.11:8776/v2/%(tenant_id)s [root@linux-node1 ~]# openstack service list +----------------------------------+----------+----------+ | ID | Name | Type | +----------------------------------+----------+----------+ | 18b41a6647e84ef68c5df6058c2f4eab | glance | image | | 436e446b475a46fa978349211d6c64eb | keystone | identity | | 613a3d7e61574fdbb7c330f6892a1b50 | neutron | network | | 6829915c1f9745409ca9bda364fe26c4 | cinderv2 | volumev2 | | 7347593df9034e369d27caf8f0240470 | nova | compute | | c63c93beff014724b036a811e2b0d591 | cinder | volume | +----------------------------------+----------+----------+ [root@linux-node1 ~]# openstack endpoint list +----------------------------------+-----------+--------------+--------------+---------+-----------+----------------------------------------------+ | ID | Region | Service Name | Service Type | Enabled | Interface | URL | +----------------------------------+-----------+--------------+--------------+---------+-----------+----------------------------------------------+ | 0ae3e6275b4c4c20a7e8619909726bd4 | RegionOne | cinder | volume | True | internal | http://192.168.56.11:8776/v1/%(tenant_id)s | | 1fba971a2dc6424eaa06ef61c910e739 | RegionOne | cinder | volume | True | admin | http://192.168.56.11:8776/v1/%(tenant_id)s | | 20222ecb6eeb4f378035e79c47810b08 | RegionOne | keystone | identity | True | public | http://192.168.56.11:5000/v3/ | | 45fd632b46684fdca9782a1e23b91f8c | RegionOne | glance | image | True | admin | http://192.168.56.11:9292 | | 64f9ee02b5d0489598f31e164d40e6df | RegionOne | nova | compute | True | public | http://192.168.56.11:8774/v2.1/%(tenant_id)s | | 6cc75ee06e5245059e106e89e1643a92 | RegionOne | keystone | identity | True | internal | http://192.168.56.11:35357/v3/ | | 77f141dede894dea877d505b60e60de7 | RegionOne | nova | compute | True | internal | http://192.168.56.11:8774/v2.1/%(tenant_id)s | | 7883d0f227a54ac5a0db3ad3a02606df | RegionOne | nova | compute | True | admin | http://192.168.56.11:8774/v2.1/%(tenant_id)s | | 7c7b33e8c2ac431aa7380ceeac80fb37 | RegionOne | keystone | identity | True | admin | http://192.168.56.11:35357/v3/ | | 84e4273b741148e2a2d9c71d2c62da1e | RegionOne | cinder | volume | True | public | http://192.168.56.11:8776/v1/%(tenant_id)s | | abd82401a31d453ca2e28fc17816fd6c | RegionOne | neutron | network | True | public | http://192.168.56.11:9696 | | af72b7e0d3824c1e82663d06c1bd0205 | RegionOne | glance | image | True | internal | http://192.168.56.11:9292 | | cb6a870ba8a543ee882afe4b07d3c087 | RegionOne | neutron | network | True | admin | http://192.168.56.11:9696 | | e27bccfa73984db889f9373f288b4c67 | RegionOne | cinderv2 | volumev2 | True | internal | http://192.168.56.11:8776/v2/%(tenant_id)s | | e3a7e437be8a4cf1968c82ceca932d57 | RegionOne | glance | image | True | public | http://192.168.56.11:9292 | | e54250bd44384b15b5bbf1bb6eb34337 | RegionOne | cinderv2 | volumev2 | True | admin | http://192.168.56.11:8776/v2/%(tenant_id)s | | eba3e70ff0a44ab28898169f4807145f | RegionOne | cinderv2 | volumev2 | True | public | http://192.168.56.11:8776/v2/%(tenant_id)s | | f5c7dad4452840d788ed59c905efb3e7 | RegionOne | neutron | network | True | internal | http://192.168.56.11:9696 | +----------------------------------+-----------+--------------+--------------+---------+-----------+----------------
-
10.cinder-scheduler的调度
创建 Volume 时,cinder-scheduler 会基于容量、Volume Type 等条件选择出最合适的存储节点,然后让其创建 Volume。
Filter scheduler 是 cinder-scheduler 默认的调度器。
scheduler_driver = cinder.scheduler.filter_scheduler.FilterScheduler
scheduler 调度过程如下:
通过过滤器(filter)选择满足条件的存储节点(运行 cinder-volume)
通过权重计算(weighting)选择最优(权重值最大)的存储节点。