目录
文章目录
Blazar
Blazar is a resource reservation service for OpenStack.
Blazar 是 OpenStack 的资源预留项目，通过 Blazar 用户可以让 OpenStack 在 “租约（leased）” 期内预留出特定的资源以供使用。
预留的资源类型：
- 虚拟预留资源：Nova Instances、Cinder Volumes、Neutron Networks
- 物理预留资源：Compute Host（full hosts with specific characteristics of RAM, CPU, etc）
应用场景:
- 为尖峰负载准备资源
- 将租约作为计量结算单元
- 优化能源消耗
- 申请专用资源
Blazar 的安装部署
通过 Devstack 部署，只需要 enable blazar devstack plugin 即可：
enable_plugin blazar https://git.openstack.org/openstack/blazar
Blazar 服务进程：
- devstack@blazar-a.service：API 服务
- devstack@blazar-m.service：Worker 服务
Blazar 的软件仓库：
- openstack/blazar：Blazar 项目
- openstack/python-blazarclient：Blazar CLI 项目
- openstack/blazar-nova：Blazar Nova Scheduler Filters 项目
- openstack/blazar-dashboard：Blazar Dashboard 项目
NOTE 1：需要手动添加 openstack/blazar-nova 中的 blazar_filter.py 模块手动拷贝到路径 /opt/stack/nova/nova/scheduler/filters/blazar_filter.py。
NOTE 2：nova.conf 的 enabled_filters 配置项必须启用 ServerGroupAntiAffinityFilter、AggregateInstanceExtraSpecsFilter、AggregateMultiTenancyIsolation 和 BlazarFilter 过滤器。
Blazar 的软件架构
- blazar-dashboard：Horizon Dashboard 的 Blazar 面板插件
- blazar-api： 提供北向 RESTful API
- blazar-manager：负责完成与租户签订的 Leases（租约）
- resource-plugin：可预留的资源类型插件，通过插件的方式开发者可以定义希望的资源类型。原生支持 physical.host 和 virtual.instance 两种 Plugins 实现。可以通过 blazar.conf 配置启用：
[manager]
plugins = physical.host.plugin,virtual.instance.plugin
- blazar-client：Linux 上的 CLI 工具
[root@localhost ~]# blazar help
...
host-create Create a host.
host-delete Delete a host.
host-list Print a list of hosts.
host-show Show host details.
host-update Update attributes of a host.
lease-create Create a lease.
lease-delete Delete a lease.
lease-list Print a list of leases.
lease-show Show details about the given lease.
lease-update Update a lease.
Blazar 的资源模型与状态机
- Reservation：预留，表示将某些特定的云资源分配给指定租户。
- Lease：租约，Blazar 与租户之间的约定资源预留期限，Blazar 保证在约定的时间范围内为租户要求的资源进行 Reservation，所以 Lease 往往具有一个开发时间（start time）和结束时间（end time）。
- Immediate reservation
- Reservation with retries.
- Best-effort reservation.
- Delayed resource acquiring or scheduled reservation.
- Event：事件，指 Blazar 执行 Lease 期间所发生的事件，比如： 开发执行租约（start lease）和结束租约（end lease）。
Reservation status
Lease status
Event status
Blazar 的主机资源预留功能（Host Reservation）
NOTE：Host Reservation 只有管理员有操作权限。
在 blazar-manager 启动时会调用 Nova API 创建一个 Host Aggregate freepool：
[root@localhost ~]# openstack aggregate show freepool
+-------------------+----------------------------+
| Field | Value |
+-------------------+----------------------------+
| availability_zone | None |
| created_at | 2019-03-27T07:43:28.000000 |
| deleted | False |
| deleted_at | None |
| hosts | [] |
| id | 1 |
| name | freepool |
| properties | |
| updated_at | None |
+-------------------+----------------------------+
“freepool” 的名称可以通过 blazar.conf 配置：
[physical:host]
aggregate_freepool_name = freepool
Step1. Add hosts into the freepool
[root@localhost ~]# openstack host list
+-----------------------+-------------+----------+
| Host Name | Service | Zone |
+-----------------------+-------------+----------+
| localhost.localdomain | scheduler | internal |
| localhost.localdomain | consoleauth | internal |
| localhost.localdomain | conductor | internal |
| localhost.localdomain | conductor | internal |
| localhost.localdomain | compute | nova |
+-----------------------+-------------+----------+
[root@localhost ~]# blazar host-create localhost.localdomain
Created a new host:
+---------------------+----------------------------------+
| Field | Value |
+---------------------+----------------------------------+
| availability_zone | nova |
| cpu_info | { |
| | "arch": "x86_64", |
| | "model": "Haswell-noTSX", |
| | "vendor": "Intel", |
| | "features": [ |
| | "pge", |
| | "avx", |
| | "xsaveopt", |
| | "clflush", |
| | "sep", |
| | "syscall", |
| | "tsc_adjust", |
| | "tsc-deadline", |
| | "invpcid", |
| | "tsc", |
| | "fsgsbase", |
| | "xsave", |
| | "vmx", |
| | "erms", |
| | "cmov", |
| | "smep", |
| | "fpu", |
| | "pat", |
| | "arat", |
| | "lm", |
| | "msr", |
| | "nx", |
| | "fxsr", |
| | "sse4.1", |
| | "pae", |
| | "sse4.2", |
| | "pclmuldq", |
| | "pcid", |
| | "fma", |
| | "vme", |
| | "mmx", |
| | "osxsave", |
| | "cx8", |
| | "mce", |
| | "de", |
| | "aes", |
| | "mca", |
| | "pse", |
| | "lahf_lm", |
| | "abm", |
| | "popcnt", |
| | "pdpe1gb", |
| | "apic", |
| | "sse", |
| | "f16c", |
| | "ds", |
| | "invtsc", |
| | "pni", |
| | "rdtscp", |
| | "avx2", |
| | "sse2", |
| | "ss", |
| | "hypervisor", |
| | "bmi1", |
| | "bmi2", |
| | "ssse3", |
| | "cx16", |
| | "pse36", |
| | "mtrr", |
| | "movbe", |
| | "rdrand", |
| | "x2apic" |
| | ], |
| | "topology": { |
| | "cores": 1, |
| | "cells": 1, |
| | "threads": 1, |
| | "sockets": 16 |
| | } |
| | } |
| created_at | 2019-03-27 07:52:53 |
| hypervisor_hostname | localhost.localdomain |
| hypervisor_type | QEMU |
| hypervisor_version | 2012000 |
| id | 1 |
| local_gb | 189 |
| memory_mb | 40959 |
| reservable | True |
| service_name | localhost.localdomain |
| status | |
| trust_id | 266b2c90dad24959a98bb0eea6defede |
| updated_at | |
| vcpus | 16 |
+---------------------+----------------------------------+
[root@localhost ~]# blazar host-list
+----+-----------------------+-------+-----------+----------+
| id | hypervisor_hostname | vcpus | memory_mb | local_gb |
+----+-----------------------+-------+-----------+----------+
| 1 | localhost.localdomain | 16 | 40959 | 189 |
+----+-----------------------+-------+-----------+----------+
[root@localhost ~]# openstack aggregate show freepool
+-------------------+----------------------------+
| Field | Value |
+-------------------+----------------------------+
| availability_zone | None |
| created_at | 2019-03-27T07:43:28.000000 |
| deleted | False |
| deleted_at | None |
| hosts | [u'localhost.localdomain'] |
| id | 1 |
| name | freepool |
| properties | |
| updated_at | None |
+-------------------+----------------------------+
NOTE：freepool 中的 hosts 是 Blazar 从 Nova 主机资源中通过 Host Aggregate 划分出来的用于进行资源预留的主机资源。
选择主机加入 freepool：
支持输入主机的扩展属性：只起到了自定义标签属性的作用（PS:Any key-value pair can be stored as one of extra capabilities of the host.）。
Step 2. Create a Host Reservation Lease
[root@localhost ~]# blazar lease-create --physical-reservation min=1,max=1,hypervisor_properties='[">=", "$vcpus", "2"]' --start-date "2019-03-28 08:10" --end-date "2019-03-28 08:25" host-lease-1
Matches: (u'min=1,max=1', u'hypervisor_properties', u'[">=", "$vcpus", "2"]')
Matches: (u'min=1', u'max', u'1')
Matches: (None, u'min', u'1')
Created a new lease:
+--------------+--------------------------------------------------------------+
| Field | Value |
+--------------+--------------------------------------------------------------+
| created_at | 2019-03-28 08:06:40 |
| degraded | False |
| end_date | 2019-03-28T08:25:00.000000 |
| events | { |
| | "status": "UNDONE", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "event_type": "start_lease", |
| | "created_at": "2019-03-28 08:06:45", |
| | "updated_at": null, |
| | "time": "2019-03-28T08:10:00.000000", |
| | "id": "ac7c0963-5de6-4bf9-a60e-f74416a057dc" |
| | } |
| | { |
| | "status": "UNDONE", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "event_type": "end_lease", |
| | "created_at": "2019-03-28 08:06:45", |
| | "updated_at": null, |
| | "time": "2019-03-28T08:25:00.000000", |
| | "id": "bdb2e272-a1a1-4b27-9464-a631cf999371" |
| | } |
| | { |
| | "status": "UNDONE", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "event_type": "before_end_lease", |
| | "created_at": "2019-03-28 08:06:45", |
| | "updated_at": null, |
| | "time": "2019-03-28T08:10:00.000000", |
| | "id": "feaeb0bb-b28d-449e-a93d-a10613c33139" |
| | } |
| id | a8b973f5-c61c-472b-a11e-68620a3f7c80 |
| name | host-lease-1 |
| project_id | f47d5bee7acd48d286224f2c411a17fb |
| reservations | { |
| | "status": "pending", |
| | "before_end": "default", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "resource_id": "08f7f776-11bd-44b7-bc31-89faa68f55a1", |
| | "max": 1, |
| | "created_at": "2019-03-28 08:06:40", |
| | "min": 1, |
| | "updated_at": "2019-03-28 08:06:45", |
| | "missing_resources": false, |
| | "hypervisor_properties": "[">=", "$vcpus", "2"]", |
| | "resource_properties": "", |
| | "id": "383371a7-652f-495d-b05e-df5ab5d67aa0", |
| | "resource_type": "physical:host", |
| | "resources_changed": false |
| | } |
| start_date | 2019-03-28T08:10:00.000000 |
| status | PENDING |
| trust_id | abb316d377ee4f139379b509b3f4da87 |
| updated_at | 2019-03-28 08:06:45 |
| user_id | e289dac11233451badb1409104a6505b |
+--------------+--------------------------------------------------------------+
[root@localhost ~]# blazar lease-list
+--------------------------------------+--------------+----------------------------+----------------------------+
| id | name | start_date | end_date |
+--------------------------------------+--------------+----------------------------+----------------------------+
| a8b973f5-c61c-472b-a11e-68620a3f7c80 | host-lease-1 | 2019-03-28T08:10:00.000000 | 2019-03-28T08:25:00.000000 |
+--------------------------------------+--------------+----------------------------+----------------------------+
--physical-reservation：表示创建 Host Reservation 类型的 Lease。min=1,max=1：表示要预留的主机数量（区间）hypervisor_properties='[">=", "$vcpus", "2"]'：描述 freepool 中的主机（Hypervisor）的特征信息
Resource Properties描述主机资源属性。
NOTE：初始状态下（当前时刻没有到达 Lease 时间），Lease host-lease-1 的 status 为 PENDING，对应的 Aggregate（Blazar 通过 Nova API 创建）e1c5add2-e3a8-433f-9b2d-ca19c8d329a8 中不包含任何主机。
[root@localhost ~]# blazar lease-show a8b973f5-c61c-472b-a11e-68620a3f7c80
+--------------+--------------------------------------------------------------+
| Field | Value |
+--------------+--------------------------------------------------------------+
| created_at | 2019-03-28 08:06:40 |
| degraded | False |
| end_date | 2019-03-28T08:25:00.000000 |
| events | { |
| | "status": "UNDONE", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "event_type": "start_lease", |
| | "created_at": "2019-03-28 08:06:45", |
| | "updated_at": null, |
| | "time": "2019-03-28T08:10:00.000000", |
| | "id": "ac7c0963-5de6-4bf9-a60e-f74416a057dc" |
| | } |
| | { |
| | "status": "UNDONE", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "event_type": "end_lease", |
| | "created_at": "2019-03-28 08:06:45", |
| | "updated_at": null, |
| | "time": "2019-03-28T08:25:00.000000", |
| | "id": "bdb2e272-a1a1-4b27-9464-a631cf999371" |
| | } |
| | { |
| | "status": "UNDONE", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "event_type": "before_end_lease", |
| | "created_at": "2019-03-28 08:06:45", |
| | "updated_at": null, |
| | "time": "2019-03-28T08:10:00.000000", |
| | "id": "feaeb0bb-b28d-449e-a93d-a10613c33139" |
| | } |
| id | a8b973f5-c61c-472b-a11e-68620a3f7c80 |
| name | host-lease-1 |
| project_id | f47d5bee7acd48d286224f2c411a17fb |
| reservations | { |
| | "status": "pending", |
| | "before_end": "default", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "resource_id": "08f7f776-11bd-44b7-bc31-89faa68f55a1", |
| | "max": 1, |
| | "created_at": "2019-03-28 08:06:40", |
| | "min": 1, |
| | "updated_at": "2019-03-28 08:06:45", |
| | "missing_resources": false, |
| | "hypervisor_properties": "[">=", "$vcpus", "2"]", |
| | "resource_properties": "", |
| | "id": "383371a7-652f-495d-b05e-df5ab5d67aa0", |
| | "resource_type": "physical:host", |
| | "resources_changed": false |
| | } |
| start_date | 2019-03-28T08:10:00.000000 |
| status | PENDING |
| trust_id | abb316d377ee4f139379b509b3f4da87 |
| updated_at | 2019-03-28 08:06:45 |
| user_id | e289dac11233451badb1409104a6505b |
+--------------+--------------------------------------------------------------+
[root@localhost ~]# openstack aggregate show 383371a7-652f-495d-b05e-df5ab5d67aa0
+-------------------+-------------------------------------------------+
| Field | Value |
+-------------------+-------------------------------------------------+
| availability_zone | blazar_383371a7-652f-495d-b05e-df5ab5d67aa0 |
| created_at | 2019-03-28T08:06:42.000000 |
| deleted | False |
| deleted_at | None |
| hosts | [] |
| id | 7 |
| name | 383371a7-652f-495d-b05e-df5ab5d67aa0 |
| properties | blazar:owner='f47d5bee7acd48d286224f2c411a17fb' |
| updated_at | None |
+-------------------+-------------------------------------------------+
Step 3. Use the leased resources
[root@localhost ~]# openstack server create --flavor m1.tiny --image cirros-0.3.4-x86_64-disk --network private --hint reservation=383371a7-652f-495d-b05e-df5ab5d67aa0 VM1
+-------------------------------------+-----------------------------------------------------------------+
| Field | Value |
+-------------------------------------+-----------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | Pm4XEBViavaZ |
| config_drive | |
| created | 2019-03-28T08:08:34Z |
| flavor | m1.tiny (1) |
| hostId | |
| id | afb7db27-ebe1-4313-b5d0-a42253f14a08 |
| image | cirros-0.3.4-x86_64-disk (0bdcd04f-2f77-42c3-ae61-5f10e2f1955c) |
| key_name | None |
| name | VM1 |
| progress | 0 |
| project_id | f47d5bee7acd48d286224f2c411a17fb |
| properties | |
| security_groups | name='default' |
| status | BUILD |
| updated | 2019-03-28T08:08:35Z |
| user_id | e289dac11233451badb1409104a6505b |
| volumes_attached | |
+-------------------------------------+-----------------------------------------------------------------+
[root@localhost ~]# openstack server list
+--------------------------------------+------+--------+----------+--------------------------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+----------+--------------------------+---------+
| afb7db27-ebe1-4313-b5d0-a42253f14a08 | VM1 | ERROR | | cirros-0.3.4-x86_64-disk | m1.tiny |
+--------------------------------------+------+--------+----------+--------------------------+---------+
--hint reservation=383371a7-652f-495d-b05e-df5ab5d67aa0：应用我（Tenant）的 Host Reservation Lease 预留资源来创建虚拟机。
虚拟机创建失败！nova-scheduler.service 日志报错，没有通过 BlazarFilter 过滤器。该结果在预期之内，因为我现在只有一台计算节点，而且被加入到 Host Reservation Lease，也就是说在 Lease 租期时间到来之前，我（Tenant）已经没有可以使用的主机资源了。
Filter BlazarFilter returned 0 hosts
NOTE：Blazar 履行 Lease 时，lease 的 status 为 ACTIVE，对应的 Aggregate e1c5add2-e3a8-433f-9b2d-ca19c8d329a8 中包含从 freepool 中 “划分” 过来的主机。e.g.
[root@localhost ~]# blazar lease-show a8b973f5-c61c-472b-a11e-68620a3f7c80
+--------------+--------------------------------------------------------------+
| Field | Value |
+--------------+--------------------------------------------------------------+
| created_at | 2019-03-28 08:06:40 |
| degraded | False |
| end_date | 2019-03-28T08:25:00.000000 |
| events | { |
| | "status": "DONE", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "event_type": "start_lease", |
| | "created_at": "2019-03-28 08:06:45", |
| | "updated_at": "2019-03-28 08:10:16", |
| | "time": "2019-03-28T08:10:00.000000", |
| | "id": "ac7c0963-5de6-4bf9-a60e-f74416a057dc" |
| | } |
| | { |
| | "status": "UNDONE", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "event_type": "end_lease", |
| | "created_at": "2019-03-28 08:06:45", |
| | "updated_at": null, |
| | "time": "2019-03-28T08:25:00.000000", |
| | "id": "bdb2e272-a1a1-4b27-9464-a631cf999371" |
| | } |
| | { |
| | "status": "DONE", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "event_type": "before_end_lease", |
| | "created_at": "2019-03-28 08:06:45", |
| | "updated_at": "2019-03-28 08:10:08", |
| | "time": "2019-03-28T08:10:00.000000", |
| | "id": "feaeb0bb-b28d-449e-a93d-a10613c33139" |
| | } |
| id | a8b973f5-c61c-472b-a11e-68620a3f7c80 |
| name | host-lease-1 |
| project_id | f47d5bee7acd48d286224f2c411a17fb |
| reservations | { |
| | "status": "active", |
| | "before_end": "default", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "resource_id": "08f7f776-11bd-44b7-bc31-89faa68f55a1", |
| | "max": 1, |
| | "created_at": "2019-03-28 08:06:40", |
| | "min": 1, |
| | "updated_at": "2019-03-28 08:10:16", |
| | "missing_resources": false, |
| | "hypervisor_properties": "[">=", "$vcpus", "2"]", |
| | "resource_properties": "", |
| | "id": "383371a7-652f-495d-b05e-df5ab5d67aa0", |
| | "resource_type": "physical:host", |
| | "resources_changed": false |
| | } |
| start_date | 2019-03-28T08:10:00.000000 |
| status | ACTIVE |
| trust_id | abb316d377ee4f139379b509b3f4da87 |
| updated_at | 2019-03-28 08:10:16 |
| user_id | e289dac11233451badb1409104a6505b |
+--------------+--------------------------------------------------------------+
[root@localhost ~]# openstack aggregate show 383371a7-652f-495d-b05e-df5ab5d67aa0
+-------------------+-------------------------------------------------+
| Field | Value |
+-------------------+-------------------------------------------------+
| availability_zone | blazar_383371a7-652f-495d-b05e-df5ab5d67aa0 |
| created_at | 2019-03-28T08:06:42.000000 |
| deleted | False |
| deleted_at | None |
| hosts | [u'localhost.localdomain'] |
| id | 7 |
| name | 383371a7-652f-495d-b05e-df5ab5d67aa0 |
| properties | blazar:owner='f47d5bee7acd48d286224f2c411a17fb' |
| updated_at | None |
+-------------------+-------------------------------------------------+
此时再尝试 Use the leased resources：
[root@localhost ~]# openstack server create --flavor m1.tiny --image cirros-0.3.4-x86_64-disk --network private --hint reservation=383371a7-652f-495d-b05e-df5ab5d67aa0 VM2
+-------------------------------------+-----------------------------------------------------------------+
| Field | Value |
+-------------------------------------+-----------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | 7Sgn5itS2xgb |
| config_drive | |
| created | 2019-03-28T08:15:38Z |
| flavor | m1.tiny (1) |
| hostId | |
| id | ec0a8419-c957-425d-bc32-deca2b193a14 |
| image | cirros-0.3.4-x86_64-disk (0bdcd04f-2f77-42c3-ae61-5f10e2f1955c) |
| key_name | None |
| name | VM2 |
| progress | 0 |
| project_id | f47d5bee7acd48d286224f2c411a17fb |
| properties | |
| security_groups | name='default' |
| status | BUILD |
| updated | 2019-03-28T08:15:39Z |
| user_id | e289dac11233451badb1409104a6505b |
| volumes_attached | |
+-------------------------------------+-----------------------------------------------------------------+
[root@localhost ~]# openstack server list
+--------------------------------------+------+--------+----------------------+--------------------------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+----------------------+--------------------------+---------+
| ec0a8419-c957-425d-bc32-deca2b193a14 | VM2 | ACTIVE | private=192.168.1.19 | cirros-0.3.4-x86_64-disk | m1.tiny |
| afb7db27-ebe1-4313-b5d0-a42253f14a08 | VM1 | ERROR | | cirros-0.3.4-x86_64-disk | m1.tiny |
+--------------------------------------+------+--------+----------------------+--------------------------+---------+
虚拟机创建成功！因为在 Lease 租期范围内，这台 Compute Host 就是专属于我（Tenant）的预留资源。
NOTE：Lease 执行完之后，lease 的 status 为 TERMINATED，对应的 Aggregate e1c5add2-e3a8-433f-9b2d-ca19c8d329a8 被删除，对应的虚拟机被删除。Lease 租约到期，blazar-manager 守护进程自动执行资源回收。e.g.
[root@localhost ~]# blazar lease-show a8b973f5-c61c-472b-a11e-68620a3f7c80
+--------------+--------------------------------------------------------------+
| Field | Value |
+--------------+--------------------------------------------------------------+
| created_at | 2019-03-28 08:06:40 |
| degraded | False |
| end_date | 2019-03-28T08:25:00.000000 |
| events | { |
| | "status": "DONE", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "event_type": "start_lease", |
| | "created_at": "2019-03-28 08:06:45", |
| | "updated_at": "2019-03-28 08:10:16", |
| | "time": "2019-03-28T08:10:00.000000", |
| | "id": "ac7c0963-5de6-4bf9-a60e-f74416a057dc" |
| | } |
| | { |
| | "status": "DONE", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "event_type": "end_lease", |
| | "created_at": "2019-03-28 08:06:45", |
| | "updated_at": "2019-03-28 08:25:21", |
| | "time": "2019-03-28T08:25:00.000000", |
| | "id": "bdb2e272-a1a1-4b27-9464-a631cf999371" |
| | } |
| | { |
| | "status": "DONE", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "event_type": "before_end_lease", |
| | "created_at": "2019-03-28 08:06:45", |
| | "updated_at": "2019-03-28 08:10:08", |
| | "time": "2019-03-28T08:10:00.000000", |
| | "id": "feaeb0bb-b28d-449e-a93d-a10613c33139" |
| | } |
| id | a8b973f5-c61c-472b-a11e-68620a3f7c80 |
| name | host-lease-1 |
| project_id | f47d5bee7acd48d286224f2c411a17fb |
| reservations | { |
| | "status": "deleted", |
| | "before_end": "default", |
| | "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80", |
| | "resource_id": "08f7f776-11bd-44b7-bc31-89faa68f55a1", |
| | "max": 1, |
| | "created_at": "2019-03-28 08:06:40", |
| | "min": 1, |
| | "updated_at": "2019-03-28 08:25:21", |
| | "missing_resources": false, |
| | "hypervisor_properties": "[">=", "$vcpus", "2"]", |
| | "resource_properties": "", |
| | "id": "383371a7-652f-495d-b05e-df5ab5d67aa0", |
| | "resource_type": "physical:host", |
| | "resources_changed": false |
| | } |
| start_date | 2019-03-28T08:10:00.000000 |
| status | TERMINATED |
| trust_id | abb316d377ee4f139379b509b3f4da87 |
| updated_at | 2019-03-28 08:25:21 |
| user_id | e289dac11233451badb1409104a6505b |
+--------------+--------------------------------------------------------------+
[root@localhost ~]# openstack aggregate list
+----+----------+-------------------+
| ID | Name | Availability Zone |
+----+----------+-------------------+
| 1 | freepool | None |
+----+----------+-------------------+
[root@localhost ~]# openstack server list
+--------------------------------------+------+--------+----------+--------------------------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+----------+--------------------------+---------+
| afb7db27-ebe1-4313-b5d0-a42253f14a08 | VM1 | ERROR | | cirros-0.3.4-x86_64-disk | m1.tiny |
+--------------------------------------+------+--------+----------+--------------------------+---------+
代码实现
我们主要关注实现 “预留” 的 BlazarFilter。
...
opts = [
cfg.StrOpt('aggregate_freepool_name',
default='freepool',
help='Name of the special aggregate where all hosts '
'are candidate for physical host reservation'),
cfg.StrOpt('project_id_key',
default='blazar:tenant',
help='Aggregate metadata value for key matching project_id'),
cfg.StrOpt('blazar_owner',
default='blazar:owner',
help='Aggregate metadata key for knowing owner project_id'),
cfg.StrOpt('blazar_az_prefix',
default='blazar_',
help='Prefix for Availability Zones created by Blazar')
]
...
class BlazarFilter(filters.BaseHostFilter):
"""Blazar Filter for nova-scheduler."""
...
def host_passes(self, host_state, spec_obj):
"""Check if a host in a pool can be used for a request
A host is in a pool if it is a member of an aggregate that has
a metadata item with a key value of "blazar:owner"
If the user does not pass "reservation=<id>" as a hint then only
hosts which are not in a pool (including freepool) pass.
If the user does pass "reservation=<id>" as a hint then the host only
passes if it is a member of the specified pool and that pool
has a metadata key of either :
- "blazar:owner=tenant_id (which grants automatically all the
users from the tenant from which the request came from)
- or, "tenant_id=blazar:tenant" (which grants extra tenants for
the reservation)
"""
# Find which Pools the user wants to use (if any)
# e.g. --hint reservation={reservation_id}
requested_pools = spec_obj.get_scheduler_hint('reservation')
if isinstance(requested_pools, six.text_type):
requested_pools = [requested_pools]
# the request is host reservation
if requested_pools:
# 检查该 Host 是否处于 --hint reservation={reservation_id} 指定的 freepool 中
# 如果处于则检测 Created Lease 和 Created Instance 的 Tenant 身份是否一致
# 如果一致，则 return True
return self.host_reservation_request(host_state, spec_obj,
requested_pools)
...
return True
def host_reservation_request(self, host_state, spec_obj, requested_pools):
# 获取 Host 所属的 freepools 对应的 Host Aggregates
pools = self.fetch_blazar_pools(host_state)
# 如果 pool（Host Aggregates） 的 name 等于 {reservation_id}
# 表示希望虚拟机在这个 pool 上启动
for pool in [p for p in pools if p.name in requested_pools]:
# Check tenant is allowed to use this Pool
# NOTE(sbauza): Currently, the key is only the project_id,
# but later will possibly be blazar:tenant:{project_id}
key = spec_obj.project_id
access = pool.metadata.get(key)
if access:
return True
# NOTE(sbauza): We also need to check the blazar:owner key
# until we modify the reservation pool for including the
# project_id key as for any other extra project
# pool（Host Aggregates）的元数据包含有 Created Lease 的 Tenant ID
# e.g. blazar:owner='f47d5bee7acd48d286224f2c411a17fb'
owner = cfg.CONF['blazar:physical:host'].blazar_owner
owner_project_id = pool.metadata.get(owner)
if owner_project_id == spec_obj.project_id:
return True
LOG.info(_("Unauthorized request to use Pool "
"%(pool_id)s by tenant %(tenant_id)s"),
{'pool_id': pool.name,
'tenant_id': spec_obj.project_id})
return False
return False
def fetch_blazar_pools(self, host_state):
# Get any reservation pools this host is part of
# Note this include possibly the freepool
# 获取 Host 处于的所有 Host Aggregate
aggregates = host_state.aggregates
pools = []
for agg in aggregates:
# 如果 Aggregate 的 AZ 以 blazar_ 或 blazar_ 开头，表示 Host 属于 Blazar freepool
if (str(agg.availability_zone).startswith(
cfg.CONF['blazar:physical:host'].blazar_az_prefix)
# NOTE(hiro-kobayashi): following 2 lines are for keeping
# backward compatibility
or str(agg.availability_zone).startswith('blazar:')):
pools.append(agg)
# 如果 Aggregate 的 name 为 freepool，表示 Host 属于 Blazar freepool
if agg.name == (
cfg.CONF['blazar:physical:host'].aggregate_freepool_name):
pools.append(agg)
return pools
Blazar 的虚拟机资源预留（Instance Reservation）
Instance Reservation 的实现依赖于 Nova Host Aggregate 和 Server Group 的功能，所以需要先对两者有一个了解。
Host Aggregate
Host Aggregates 是在 Availability Zones 的基础上更进一步地进行逻辑的分组和隔离。例如我们可以根据不同的 Compute Nodes 的物理硬件配置（e.g. NUMA、SR-IOV、SSD Disk）将具有相同共性的物理资源规划在同一个 Host Aggregate 之下，或者根据用户的具体需求将几个 Compute Nodes 规划在具有相同用途（e.g. 高性能主机聚合）的同一个 Host Aggregate 之下，通过这样的划分有利于提高 Nova 主机资源调度的灵活性。
管理员可以给 Aggregate 设置 metadata，一个 Compute Host 可以属于多个 Aggregates，基于 Aggregate 的 metadata 可以实现高级调度功能。下面通过一个案例来感受。
Step 1. 在指定的 AZ 下创建 Host Aggregate，如果指定的 AZ 不存在，则自动创建一个。
# nova aggregate-create <ag_name> <az_name>
[root@control01 ~]# nova aggregate-create vmware-agg nova
+----+------------+-------------------+-------+--------------------------+--------------------------------------+
| Id | Name | Availability Zone | Hosts | Metadata | UUID |
+----+------------+-------------------+-------+--------------------------+--------------------------------------+
| 1 | vmware-agg | nova | | 'availability_zone=nova' | 154bb7b6-217e-45ef-8759-34d418cd9dbe |
+----+------------+-------------------+-------+--------------------------+--------------------------------------+
Step 2. 设置指定的 Host Aggregate 的 metadata。
# nova aggregate-set-metadata <ag_name> <metadatas (key=value)>
[root@control01 ~]# nova aggregate-set-metadata vmware-agg DRIVE=esxi
Metadata has been successfully updated for aggregate 1.
+----+------------+-------------------+-------+----------------------------------------+--------------------------------------+
| Id | Name | Availability Zone | Hosts | Metadata | UUID |
+----+------------+-------------------+-------+----------------------------------------+--------------------------------------+
| 1 | vmware-agg | nova | | 'DRIVE=esxi', 'availability_zone=nova' | 154bb7b6-217e-45ef-8759-34d418cd9dbe |
+----+------------+-------------------+-------+----------------------------------------+--------------------------------------+
NOTE：Nova 还专门预留了一些特殊字段的 Aggregate metadata 用于 Nova Scheduler，e.g. cpu_allocation_ratio、ram_allocation_ratio、disk_allocation_ratio 表示超配比，在创建虚拟机，nova-scheduler.service 会根据这些特殊的 metadata 作为过滤条件。前提是启用了 AggregateCoreFilter，AggregateRamFilter 和 AggregateDiskFilter 过滤器。否则使用全局 ratios，全局的 ratios 参数配置在 nova.conf 文件里。
Step 3. 设置 Flavor 的 extra_specs 参数，保证这些参数与 Host aggregate 的 metadata 参数能够匹配上（一致）。这样就能够 达到通过选定 Flavor 来选定虚拟机启动的 Host Aggregate 了。
[root@control01 ~]# nova flavor-key 9cb153ea-da73-47bc-86ab-dd1ae41bfed7 set "DRIVE=esxi"
[root@control01 ~]# nova flavor-show 9cb153ea-da73-47bc-86ab-dd1ae41bfed7
+----------------------------+--------------------------------------+
| Property | Value |
+----------------------------+--------------------------------------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| disk | 5 |
| extra_specs | {"DRIVE": "esxi"} |
| id | 9cb153ea-da73-47bc-86ab-dd1ae41bfed7 |
| name | m1.xsmall |
| os-flavor-access:is_public | True |
| ram | 1024 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 1 |
+----------------------------+———————————————————+
Step 4. 将 AggregateInstanceExtraSpecsFilter（自定义主机聚合过滤器）添加到所有 Nova Scheduler 的配置文件中。
enabled_filters =...,AggregateInstanceExtraSpecsFilter
Step 5. 将 Compute Host 添加到 Host Aggregates 里。
[root@control01 ~]# nova aggregate-add-host vmware-agg aju-dev-vmware
Host aju-dev-vmware has been successfully added for aggregate 1
+----+------------+-------------------+------------------+----------------------------------------+--------------------------------------+
| Id | Name | Availability Zone | Hosts | Metadata | UUID |
+----+------------+-------------------+------------------+----------------------------------------+--------------------------------------+
| 1 | vmware-agg | nova | 'aju-dev-vmware' | 'DRIVE=esxi', 'availability_zone=nova' | 154bb7b6-217e-45ef-8759-34d418cd9dbe |
+----+------------+-------------------+------------------+----------------------------------------+———————————————————+
Server Group
Server Group: From Icehouse version, OpenStack allows user to explicitly specify if he wants or not a group of virtual machines (VMs) to share the same hosts. These policies are called “Affinity” and “Anti-Affinity”.
- Affinity: the policy that forces Nova to hosts the concerned VMs in a same hypervisor.
- Anti-Affinity: the policy that forces Nova to hosts the concerned VMs each in a different hypervisor.
Step 1. Creating ServerGroup
nova server-group-create <group_name> [< policy >]
<group_name>: Name of the ServerGroup<policy>: Policy of the ServerGroup (“affinity” or “anti-affinity”)
Step 2. Create a virtual machine in a ServerGroup
nova boot --image ubuntu --flavor m1.small --hint group=aa8a69f4-2567-47c6-a28c-2934a8c7959c vm1
--hint group=<group_id>：Group ID of the ServerGroup
Instance Reservation
Step 1. Add hosts into the freepool（这里不再重复）
Step 2. Create a lease
[root@localhost ~]# blazar lease-create --reservation resource_type=virtual:instance,vcpus=1,memory_mb=1024,disk_gb=20,amount=1,affinity=False --start-date "2019-03-28 09:28" --end-date "2020-03-28 09:45" instance-lease-1
Matches: (u'resource_type=virtual:instance,vcpus=1,memory_mb=1024,disk_gb=20,amount=1', u'affinity', u'False')
Matches: (u'resource_type=virtual:instance,vcpus=1,memory_mb=1024,disk_gb=20', u'amount', u'1')
Matches: (u'resource_type=virtual:instance,vcpus=1,memory_mb=1024', u'disk_gb', u'20')
Matches: (u'resource_type=virtual:instance,vcpus=1', u'memory_mb', u'1024')
Matches: (u'resource_type=virtual:instance', u'vcpus', u'1')
Matches: (None, u'resource_type', u'virtual:instance')
Created a new lease:
+--------------+-----------------------------------------------------------------+
| Field | Value |
+--------------+-----------------------------------------------------------------+
| created_at | 2019-03-28 09:19:20 |
| degraded | False |
| end_date | 2020-03-28T09:45:00.000000 |
| events | { |
| | "status": "UNDONE", |
| | "lease_id": "5e6bced2-4031-4688-bdc4-df7e05ec24e6", |
| | "event_type": "before_end_lease", |
| | "created_at": "2019-03-28 09:19:21", |
| | "updated_at": null, |
| | "time": "2020-03-28T08:45:00.000000", |
| | "id": "6be9533e-c504-4d00-83c8-dc76b1d511c4" |
| | } |
| | { |
| | "status": "UNDONE", |
| | "lease_id": "5e6bced2-4031-4688-bdc4-df7e05ec24e6", |
| | "event_type": "end_lease", |
| | "created_at": "2019-03-28 09:19:21", |
| | "updated_at": null, |
| | "time": "2020-03-28T09:45:00.000000", |
| | "id": "745b2d1b-c30f-4012-b781-4da3a51c8f79" |
| | } |
| | { |
| | "status": "UNDONE", |
| | "lease_id": "5e6bced2-4031-4688-bdc4-df7e05ec24e6", |
| | "event_type": "start_lease", |
| | "created_at": "2019-03-28 09:19:21", |
| | "updated_at": null, |
| | "time": "2019-03-28T09:28:00.000000", |
| | "id": "bd445bde-d8ce-4e0e-81dc-46c1e1ec29c7" |
| | } |
| id | 5e6bced2-4031-4688-bdc4-df7e05ec24e6 |
| name | instance-lease-1 |
| project_id | f47d5bee7acd48d286224f2c411a17fb |
| reservations | { |
| | "status": "pending", |
| | "memory_mb": 1024, |
| | "lease_id": "5e6bced2-4031-4688-bdc4-df7e05ec24e6", |
| | "resource_properties": "", |
| | "disk_gb": 20, |
| | "resource_id": "122b6fe9-fc75-41df-bbaa-25890ed1896e", |
| | "created_at": "2019-03-28 09:19:20", |
| | "updated_at": "2019-03-28 09:19:21", |
| | "missing_resources": false, |
| | "server_group_id": "484c4291-6a7d-4ba0-8e22-422374f64e50", |
| | "amount": 1, |
| | "affinity": false, |
| | "flavor_id": "67479531-7c37-4c12-97fa-ed56f9f45a55", |
| | "id": "67479531-7c37-4c12-97fa-ed56f9f45a55", |
| | "aggregate_id": 9, |
| | "vcpus": 1, |
| | "resource_type": "virtual:instance", |
| | "resources_changed": false |
| | } |
| start_date | 2019-03-28T09:28:00.000000 |
| status | PENDING |
| trust_id | fddf072c97ba46c88a76183f95a05dd1 |
| updated_at | 2019-03-28 09:19:21 |
| user_id | e289dac11233451badb1409104a6505b |
+--------------+-----------------------------------------------------------------+
resource_type=virtual:instance,vcpus=1,memory_mb=1024,disk_gb=20,amount=1,affinity=False：指定为申请 Instance Reservation 类型资源预留，Instance 的规格和数量。
NOTE：可以通过 affinity=False 来指定 Instances 之间是否启动 Server Group 亲和/反亲和。但现阶段支持 anti-affinity，否则触发错误 ERROR: Malformed parameter affinity (only affinity = False is supported)。所以如果你希望同时预留多个 instances，那么 freepool 就需要具有相应数量 Hosts。
创建 Instance Reservation Lease 之后，Blazar 会调用 Nova API 分别创建相应的 Host Aggregate、Server Group、Flavor 记录。e.g.
[root@localhost ~]# openstack aggregate show 67479531-7c37-4c12-97fa-ed56f9f45a55
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| availability_zone | None |
| created_at | 2019-03-28T09:19:21.000000 |
| deleted | False |
| deleted_at | None |
| hosts | [] |
| id | 9 |
| name | 67479531-7c37-4c12-97fa-ed56f9f45a55 |
| properties | affinity_id='484c4291-6a7d-4ba0-8e22-422374f64e50', blazar:owner='f47d5bee7acd48d286224f2c411a17fb', filter_tenant_id='f47d5bee7acd48d286224f2c411a17fb', reservation='67479531-7c37-4c12-97fa-ed56f9f45a55' |
| updated_at | None |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[root@localhost ~]# openstack server group show 484c4291-6a7d-4ba0-8e22-422374f64e50
+----------+--------------------------------------------------+
| Field | Value |
+----------+--------------------------------------------------+
| id | 484c4291-6a7d-4ba0-8e22-422374f64e50 |
| members | |
| name | reservation:67479531-7c37-4c12-97fa-ed56f9f45a55 |
| policies | anti-affinity |
+----------+--------------------------------------------------+
[root@localhost ~]# openstack flavor show 67479531-7c37-4c12-97fa-ed56f9f45a55
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| access_project_ids | |
| disk | 20 |
| id | 67479531-7c37-4c12-97fa-ed56f9f45a55 |
| name | reservation:67479531-7c37-4c12-97fa-ed56f9f45a55 |
| os-flavor-access:is_public | False |
| properties | affinity_id='484c4291-6a7d-4ba0-8e22-422374f64e50', aggregate_instance_extra_specs:reservation='67479531-7c37-4c12-97fa-ed56f9f45a55' |
| ram | 1024 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 1 |
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
同样的，如果没有到 Lease instance-lease-1 租约时间之前使用 instance-lease-1 创建虚拟机会创建失败：
[root@localhost ~]# openstack server create --flavor 67479531-7c37-4c12-97fa-ed56f9f45a55 --image cirros-0.3.4-x86_64-disk --network private VM1
+-------------------------------------+-----------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------------------+-----------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | suypHUv8F92v |
| config_drive | |
| created | 2019-03-28T09:26:11Z |
| flavor | reservation:67479531-7c37-4c12-97fa-ed56f9f45a55 (67479531-7c37-4c12-97fa-ed56f9f45a55) |
| hostId | |
| id | 4f324ba9-39b0-4890-b09c-6d1fcae25c61 |
| image | cirros-0.3.4-x86_64-disk (0bdcd04f-2f77-42c3-ae61-5f10e2f1955c) |
| key_name | None |
| name | VM1 |
| progress | 0 |
| project_id | f47d5bee7acd48d286224f2c411a17fb |
| properties | |
| security_groups | name='default' |
| status | BUILD |
| updated | 2019-03-28T09:26:12Z |
| user_id | e289dac11233451badb1409104a6505b |
| volumes_attached | |
+-------------------------------------+-----------------------------------------------------------------------------------------+
[root@localhost ~]# openstack server list
+--------------------------------------+------+--------+----------+--------------------------+--------------------------------------------------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+----------+--------------------------+--------------------------------------------------+
| 4f324ba9-39b0-4890-b09c-6d1fcae25c61 | VM1 | ERROR | | cirros-0.3.4-x86_64-disk | reservation:67479531-7c37-4c12-97fa-ed56f9f45a55 |
+--------------------------------------+------+--------+----------+--------------------------+--------------------------------------------------+
--flavor 67479531-7c37-4c12-97fa-ed56f9f45a55通过 Lease 自动生成的 Flavor 来创建虚拟机。
nova-scheduler.service 错误日志：
(localhost.localdomain, localhost.localdomain) ram: 40447MB disk: 177152MB io_ops: 0 instances: 0 fails instance_type extra_specs requirements. Extra_spec reservation is not in aggregate.
Filter AggregateInstanceExtraSpecsFilter returned 0 hosts
根本的原因还是因为 Lease Flavor 匹配的 Aggregate 67479531-7c37-4c12-97fa-ed56f9f45a55 中并没有 hosts。等到了 Lease instance-lease-1 的租约时间后，再执行一次创建：
[root@localhost ~]# openstack server create --flavor 67479531-7c37-4c12-97fa-ed56f9f45a55 --image cirros-0.3.4-x86_64-disk --network private VM2
+-------------------------------------+-----------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------------------+-----------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | NdBfrWUp47wx |
| config_drive | |
| created | 2019-03-28T09:36:34Z |
| flavor | reservation:67479531-7c37-4c12-97fa-ed56f9f45a55 (67479531-7c37-4c12-97fa-ed56f9f45a55) |
| hostId | |
| id | 267e6621-b085-4d58-81e6-138c190a45cb |
| image | cirros-0.3.4-x86_64-disk (0bdcd04f-2f77-42c3-ae61-5f10e2f1955c) |
| key_name | None |
| name | VM2 |
| progress | 0 |
| project_id | f47d5bee7acd48d286224f2c411a17fb |
| properties | |
| security_groups | name='default' |
| status | BUILD |
| updated | 2019-03-28T09:36:35Z |
| user_id | e289dac11233451badb1409104a6505b |
| volumes_attached | |
+-------------------------------------+-----------------------------------------------------------------------------------------+
[root@localhost ~]# openstack server list
+--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
| 267e6621-b085-4d58-81e6-138c190a45cb | VM2 | ACTIVE | private=192.168.1.55 | cirros-0.3.4-x86_64-disk | reservation:67479531-7c37-4c12-97fa-ed56f9f45a55 |
| 4f324ba9-39b0-4890-b09c-6d1fcae25c61 | VM1 | ERROR | | cirros-0.3.4-x86_64-disk | reservation:67479531-7c37-4c12-97fa-ed56f9f45a55 |
+--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
创建成功！因为此时的 Aggregate 67479531-7c37-4c12-97fa-ed56f9f45a55 已经有 hosts 了：
[root@localhost ~]# openstack aggregate show 67479531-7c37-4c12-97fa-ed56f9f45a55
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| availability_zone | None |
| created_at | 2019-03-28T09:19:21.000000 |
| deleted | False |
| deleted_at | None |
| hosts | [u'localhost.localdomain'] |
| id | 9 |
| name | 67479531-7c37-4c12-97fa-ed56f9f45a55 |
| properties | affinity_id='484c4291-6a7d-4ba0-8e22-422374f64e50', blazar:owner='f47d5bee7acd48d286224f2c411a17fb', filter_tenant_id='f47d5bee7acd48d286224f2c411a17fb', reservation='67479531-7c37-4c12-97fa-ed56f9f45a55' |
| updated_at | None |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
这是因为 blazar-manager 监控 Lease instance-lease-1 的租期之间到了，所以调用 Nova API 从 Aggregate freepool 中选择一个资源满足的 Host 加入到 Lease instance-lease-1 对应的 Aggregate 中。
同样的，当 Lease 租约到期之后，blazar-manager 就会自动进行资源回收，将于 Lease 相关的（成功或失败）Instances、Server Group、Host Aggregate、Flavor 全部删除。
代码实现：
# /opt/stack/nova/nova/scheduler/filters/blazar_filter.py
...
FLAVOR_EXTRA_SPEC = "aggregate_instance_extra_specs:reservation"
...
class BlazarFilter(filters.BaseHostFilter):
"""Blazar Filter for nova-scheduler."""
...
def host_passes(self, host_state, spec_obj):
...
# the request is instance reservation
if FLAVOR_EXTRA_SPEC in spec_obj.flavor.extra_specs.keys():
# Scheduling requests for instance reservation are processed by
# other Nova filters: AggregateInstanceExtraSpecsFilter,
# AggregateMultiTenancyIsolation, and
# ServerGroupAntiAffinityFilter. What BlazarFilter needs to
# do is just pass the host if the request has an instance
# reservation key.
return True
对于 Instance Reservation Lease 而言 BlazarFilter 只是单纯的判断一下 Instance Flavor 是否具有 Blazar 标签 aggregate_instance_extra_specs:reservation 而已，当然在 BlazarFilter 之前，还经过了以下 Filters：
- AggregateInstanceExtraSpecsFilter：第一确保 Flavor 具有
aggregate_instance_extra_specs:reservation='16f3c0be-f069-4f1a-ab76-cab94833ad7a'扩展属性；第二确保 Flavor 对应的 Host Aggregate 内含有 hosts。因为如果 Host Aggregate 没有 hosts 的话metadata = utils.aggregate_metadata_get_by_host(host_state)返回的 metadata 是一个空字典，无法匹配到 key:reservation。 - AggregateMultiTenancyIsolation：确保创建预留的租户和使用预留的租户一致。
- ServerGroupAntiAffinityFilter：确保在 freepool 中不同的 Hosts 上安排 Instances。
Blazar 的设计思路与实现原理
整体来看，Blazar 的设计思路是基于资源隔离来实现的资源预留。Blazar 通过应用 Nova Host Aggregate 和 Nova Scheduler Filters 机制将 OpenStack 计算资源（以 Compute Node 为单位）划分成 “Nova 原生计算资源” 和 “Blazar 预留计算资源”。
- Nova 原生计算资源：原生的资源管理系统，由 Nova Compute 维护。
- Blazar 预留计算资源：Blazar 自身实现了一套基于 “资源租赁机制” 的资源管理系统。
Scheduling mechanism in Nova
Blazar 按照下列算法来保证 “预留资源” 调度的可靠性：
- non-reserved instances: scheduled to hypervisors which are outside of both the freepool aggregate and reservation-related aggregates.
- instances related to host reservation: scheduled to hypervisors which are inside the reservation-related aggregate. The hypervisors are not included in the freepool aggregate.
- instances related to instance reservation: scheduled to hypervisors which are inside the reservation-related aggregate. The hypervisors are included in the freepool aggregate.
Nova filters used by Blazar choose hypervisors with the following rules:
- AggregateInstanceExtraSpecsFilter picks up hypervisors from the aggregate related to an instance reservation based on extra_specs of the flavor, if the request is related to instance reservation. If not, the filter picks up hypervisors from neither reservation-related aggregates nor the freepool.
- BlazarFilter picks up hypervisors from the aggregate related to a host reservation based on the ‘reservation’ scheduler hint, if the request is related to host reservation. If not, the filter picks up hypervisors from neither host reservation-related aggregates nor the freepool.
- AggregateMultiTenancyIsolationFilter blocks requests to be scheduled to the freepool by users who do not have active reservation.
- Combination of AggregateInstanceExtraSpecsFilter and AggregateMultiTenancyIsolationFilter enables requests using instance reservation to be scheduled in the corresponding aggregate.
- ServerGroupAntiAffinityFilter ensures instance reservation related instances are spread on different hypervisors.
Blazar Pike 的局限
NOTE：以上分析适用于 Pike, Queens,Rocky 版本
- Instance Reservation 要依赖 Host Reservation，并没有实现真正 意义上纯粹的虚拟机预留。
- Instance Reservation 不支持 Affinity Server Group。
- Instance Reservation 只支持 CPU、RAM、Disk 资源类型，不支持 NUMA、SR-IOV 等高级资源类型。
Blazar 的未来
现在 Blazar 的实现是基于 Nova Scheduler Filters 机制来实现的，正如 Nova Scheduler 会被 Placement 替代一样，Blazar 最新的实现方式也正过渡到依赖 Placement 的 Resource Class。Blazar Instance reservation supports both affinity rule and anti-affinity rule. 的工作正在进行中。
The affinity rule reservation allows other instances or reservation to use unused instance slots in reserved hypervisors. The Nova team is developing placement API. The API already has custom resource classes and is now implementing a scheduler function[3] that uses custom resources classes. It enables operator to more efficiently manage hypervisors in the freepool.
当 Blazar 迁移至 Placement 之后，其调度过程将会变成：
- A tenant user creates their reservation, in term of Blazar called “lease”, with a time frame, the instance size, and how many instances.
- Blazar checks whether the reservation is acceptable during the time frame or not. If acceptable, Blazar records the reservation request in its database and updates the usage of hypervisor in freepool. Then Blazar returns the reservation id. If not, Blazar responds the reservation is not acceptable.
- At the start time of the reservation, Blazar creates a custom resource class, a flavor, and a resource provider of the custom resource class.
- The user creates reserved instances with the newly created flavor.
详细请看：https://docs.openstack.org/blazar/latest/specs/pike/new-instance-reservation.html
Blazar Stein: Placement API support for instance reservation
这个功能虽然是 Rocky 版本被提出（Introduced），但仅限于单元测试调用，直到 Stein 才被引入到主干程序逻辑。所以如果希望使用该功能，还是要部署 Stein 版本的 Blazar 代码库。
使用 Placement API 的 Instance Reservation 可以实现：
- A user wants to reserve instance resources with arbitrary affinity policy.
- A user wants to reserve more instances than the number of hosts.
使用 Placement API 的 Instance Reservation 带来以下改变：
- A host aggregate is not created for each instance reservation anymore because reserved hosts can be distinguished by the reservation provider inventory.
- A server group is not created anymore because the proposed approach does not depend on the ServerGroup(Anti)AffinityFilter.
详细的设计思路可浏览：https://docs.openstack.org/blazar/latest/specs/rocky/placement-api.html
Creating a host
创建一条定义为 nested resource provider 的 Resource Provider，又称为 reservation provider。
API：POST /placement/resource_providers
Body：
{
"name": "blazar_compute-1",
"parent_provider_uuid": "{compute node resource provider uuid}"
}
CLI：
[root@localhost ~]# openstack resource provider list
+--------------------------------------+------------------------------+------------+
| uuid | name | generation |
+--------------------------------------+------------------------------+------------+
| 88c5326f-68a9-490b-8298-f69dbe03b453 | localhost.localdomain | 3 |
| 3aa43b09-a7b9-4f21-ab01-a15cb29d1a8c | blazar_localhost.localdomain | 0 |
+--------------------------------------+------------------------------+------------+
[root@localhost ~]# openstack resource provider inventory list 3aa43b09-a7b9-4f21-ab01-a15cb29d1a8c
Creating a lease
API：POST /placement/resource_classes
Body：
{
"name": "CUSTOM_RESERVATION_4D17D41A_830D_47B2_91C7_4F9FC0AE611E"
}
CLI：
[root@localhost ~]# blazar lease-create --reservation resource_type=virtual:instance,vcpus=1,memory_mb=1024,disk_gb=20,amount=1,affinity=True --start-date "2019-03-29 03:55" --end-date "2020-03-29 04:10" instance-lease-1
Matches: (u'resource_type=virtual:instance,vcpus=1,memory_mb=1024,disk_gb=20,amount=1', u'affinity', u'True')
Matches: (u'resource_type=virtual:instance,vcpus=1,memory_mb=1024,disk_gb=20', u'amount', u'1')
Matches: (u'resource_type=virtual:instance,vcpus=1,memory_mb=1024', u'disk_gb', u'20')
Matches: (u'resource_type=virtual:instance,vcpus=1', u'memory_mb', u'1024')
Matches: (u'resource_type=virtual:instance', u'vcpus', u'1')
Matches: (None, u'resource_type', u'virtual:instance')
Created a new lease:
+--------------+-----------------------------------------------------------------+
| Field | Value |
+--------------+-----------------------------------------------------------------+
| created_at | 2019-03-29 03:54:08 |
| degraded | False |
| end_date | 2020-03-29T04:10:00.000000 |
| events | { |
| | "status": "UNDONE", |
| | "lease_id": "6a178427-ae5c-4ac5-a1dc-b69e6207166a", |
| | "event_type": "before_end_lease", |
| | "created_at": "2019-03-29 03:54:15", |
| | "updated_at": null, |
| | "time": "2020-03-29T03:10:00.000000", |
| | "id": "994c8a58-25b6-4e0f-982e-d92ad08e59cc" |
| | } |
| | { |
| | "status": "UNDONE", |
| | "lease_id": "6a178427-ae5c-4ac5-a1dc-b69e6207166a", |
| | "event_type": "end_lease", |
| | "created_at": "2019-03-29 03:54:15", |
| | "updated_at": null, |
| | "time": "2020-03-29T04:10:00.000000", |
| | "id": "d33fd540-e0bd-44bc-903b-1e04f63b3fe4" |
| | } |
| | { |
| | "status": "UNDONE", |
| | "lease_id": "6a178427-ae5c-4ac5-a1dc-b69e6207166a", |
| | "event_type": "start_lease", |
| | "created_at": "2019-03-29 03:54:15", |
| | "updated_at": null, |
| | "time": "2019-03-29T03:55:00.000000", |
| | "id": "d4936d4e-d797-4654-ba30-610c82ab5683" |
| | } |
| id | 6a178427-ae5c-4ac5-a1dc-b69e6207166a |
| name | instance-lease-1 |
| project_id | 9e782daaa46c4a7bb5ffda88160993c8 |
| reservations | { |
| | "status": "pending", |
| | "memory_mb": 1024, |
| | "lease_id": "6a178427-ae5c-4ac5-a1dc-b69e6207166a", |
| | "resource_properties": "", |
| | "disk_gb": 20, |
| | "resource_id": "b9bbb153-e661-49cb-8305-3f4ae748a1fb", |
| | "created_at": "2019-03-29 03:54:08", |
| | "updated_at": "2019-03-29 03:54:15", |
| | "missing_resources": false, |
| | "server_group_id": "96a89cb4-5a7e-4546-9b0f-90dc3f599fe3", |
| | "amount": 1, |
| | "affinity": true, |
| | "flavor_id": "abeaea5c-b14f-48df-ae46-aa56b55f61fc", |
| | "id": "abeaea5c-b14f-48df-ae46-aa56b55f61fc", |
| | "aggregate_id": 2, |
| | "vcpus": 1, |
| | "resource_type": "virtual:instance", |
| | "resources_changed": false |
| | } |
| start_date | 2019-03-29T03:55:00.000000 |
| status | PENDING |
| trust_id | 00e5c6c14e794af49f25489d8136c872 |
| updated_at | 2019-03-29 03:54:15 |
| user_id | 8a4b574706964edf85d2cc46b3bbef26 |
+--------------+-----------------------------------------------------------------+
[root@localhost ~]# openstack resource class list
+---------------------------------------------------------+
| name |
+---------------------------------------------------------+
| VCPU |
| MEMORY_MB |
| DISK_GB |
| PCI_DEVICE |
| SRIOV_NET_VF |
| NUMA_SOCKET |
| NUMA_CORE |
| NUMA_THREAD |
| NUMA_MEMORY_MB |
| IPV4_ADDRESS |
| VGPU |
| VGPU_DISPLAY_HEAD |
| NET_BW_EGR_KILOBIT_PER_SEC |
| NET_BW_IGR_KILOBIT_PER_SEC |
| PCPU |
| CUSTOM_RESERVATION_ABEAEA5C_B14F_48DF_AE46_AA56B55F61FC |
+---------------------------------------------------------+
[root@localhost ~]# openstack flavor show abeaea5c-b14f-48df-ae46-aa56b55f61fc
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| access_project_ids | 9e782daaa46c4a7bb5ffda88160993c8 |
| disk | 20 |
| id | abeaea5c-b14f-48df-ae46-aa56b55f61fc |
| name | reservation:abeaea5c-b14f-48df-ae46-aa56b55f61fc |
| os-flavor-access:is_public | False |
| properties | affinity_id='96a89cb4-5a7e-4546-9b0f-90dc3f599fe3', aggregate_instance_extra_specs:reservation='abeaea5c-b14f-48df-ae46-aa56b55f61fc', resources:CUSTOM_RESERVATION_ABEAEA5C_B14F_48DF_AE46_AA56B55F61FC='1' |
| ram | 1024 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 1 |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[root@localhost ~]# openstack server group show 96a89cb4-5a7e-4546-9b0f-90dc3f599fe3
+----------+--------------------------------------------------+
| Field | Value |
+----------+--------------------------------------------------+
| id | 96a89cb4-5a7e-4546-9b0f-90dc3f599fe3 |
| members | |
| name | reservation:abeaea5c-b14f-48df-ae46-aa56b55f61fc |
| policies | affinity |
+----------+--------------------------------------------------+
Starting a lease
API：PUT /placement/resource_providers/{reservation_provider_uuid}/inventories
Body:
{
"inventories": {
"CUSTOM_RESERVATION_4D17D41A_830D_47B2_91C7_4F9FC0AE611E": {
"total": 3,
"allocation_ratio": 1.0,
"min_unit": 1,
"max_unit": 1,
"step_size": 1
},
"snip"
},
"resource_provider_generation": 5
}
CLI：
[root@localhost ~]# openstack resource provider inventory list 3aa43b09-a7b9-4f21-ab01-a15cb29d1a8c
+---------------------------------------------------------+------------------+----------+----------+-----------+----------+-------+
| resource_class | allocation_ratio | max_unit | reserved | step_size | min_unit | total |
+---------------------------------------------------------+------------------+----------+----------+-----------+----------+-------+
| CUSTOM_RESERVATION_ABEAEA5C_B14F_48DF_AE46_AA56B55F61FC | 1.0 | 1 | 0 | 1 | 1 | 1 |
+---------------------------------------------------------+------------------+----------+----------+-----------+----------+-------+
[root@localhost ~]# openstack aggregate show 2
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| availability_zone | None |
| created_at | 2019-03-29T03:54:13.000000 |
| deleted | False |
| deleted_at | None |
| hosts | [u'localhost.localdomain'] |
| id | 2 |
| name | abeaea5c-b14f-48df-ae46-aa56b55f61fc |
| properties | affinity_id='96a89cb4-5a7e-4546-9b0f-90dc3f599fe3', blazar:owner='9e782daaa46c4a7bb5ffda88160993c8', filter_tenant_id='9e782daaa46c4a7bb5ffda88160993c8', reservation='abeaea5c-b14f-48df-ae46-aa56b55f61fc' |
| updated_at | None |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Launching instances
[root@localhost ~]# openstack flavor show abeaea5c-b14f-48df-ae46-aa56b55f61fc
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| access_project_ids | 9e782daaa46c4a7bb5ffda88160993c8 |
| disk | 20 |
| id | abeaea5c-b14f-48df-ae46-aa56b55f61fc |
| name | reservation:abeaea5c-b14f-48df-ae46-aa56b55f61fc |
| os-flavor-access:is_public | False |
| properties | affinity_id='96a89cb4-5a7e-4546-9b0f-90dc3f599fe3', aggregate_instance_extra_specs:reservation='abeaea5c-b14f-48df-ae46-aa56b55f61fc', resources:CUSTOM_RESERVATION_ABEAEA5C_B14F_48DF_AE46_AA56B55F61FC='1' |
| ram | 1024 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 1 |
+----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[root@localhost ~]# openstack server create --flavor abeaea5c-b14f-48df-ae46-aa56b55f61fc --image f2029152-4af3-460d-acf0-4798808753c0 --network 54726ee2-38c2-448f-ad88-06e4058f573a VM1
+-------------------------------------+-----------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------------------+-----------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | zNKCQVoSe3W2 |
| config_drive | |
| created | 2019-03-29T04:55:26Z |
| flavor | reservation:abeaea5c-b14f-48df-ae46-aa56b55f61fc (abeaea5c-b14f-48df-ae46-aa56b55f61fc) |
| hostId | |
| id | 65203ec7-ba30-45af-b2db-8648f9e10f2a |
| image | cirros-0.3.4-x86_64-disk (f2029152-4af3-460d-acf0-4798808753c0) |
| key_name | None |
| name | VM1 |
| progress | 0 |
| project_id | 9e782daaa46c4a7bb5ffda88160993c8 |
| properties | |
| security_groups | name='default' |
| status | BUILD |
| updated | 2019-03-29T04:55:26Z |
| user_id | 8a4b574706964edf85d2cc46b3bbef26 |
| volumes_attached | |
+-------------------------------------+-----------------------------------------------------------------------------------------+
[root@localhost ~]# openstack server list
+--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
| 65203ec7-ba30-45af-b2db-8648f9e10f2a | VM1 | ACTIVE | private=192.168.1.27 | cirros-0.3.4-x86_64-disk | reservation:abeaea5c-b14f-48df-ae46-aa56b55f61fc |
+--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
Launching instances 的实现逻辑
- 存在 Nested Resource Provider:blazar_{hypervisor}
[root@localhost ~]# openstack resource provider list
+--------------------------------------+------------------------------+------------+--------------------------------------+--------------------------------------+
| uuid | name | generation | root_provider_uuid | parent_provider_uuid |
+--------------------------------------+------------------------------+------------+--------------------------------------+--------------------------------------+
| 7b155336-faaa-4e39-a51b-851ae7495ac5 | localhost.localdomain | 4 | 7b155336-faaa-4e39-a51b-851ae7495ac5 | None |
| 88f75d49-7512-4969-981f-fc893d8d552f | blazar_localhost.localdomain | 1 | 7b155336-faaa-4e39-a51b-851ae7495ac5 | 7b155336-faaa-4e39-a51b-851ae7495ac5 |
+--------------------------------------+------------------------------+------------+--------------------------------------+--------------------------------------+
- Nested Resource Provider 存在有 CUSTOM_RESERVATION_{reservation_flavor_uuid}，到达 lease start time，Blazar 会自动将 CUSTOM_RESERVATION_{reservation_flavor_uuid} 添加到 Nested Resource Provider。
[root@localhost ~]# openstack resource provider inventory list 88f75d49-7512-4969-981f-fc893d8d552f
+---------------------------------------------------------+------------------+----------+----------+-----------+----------+-------+
| resource_class | allocation_ratio | max_unit | reserved | step_size | min_unit | total |
+---------------------------------------------------------+------------------+----------+----------+-----------+----------+-------+
| CUSTOM_RESERVATION_9259448D_88BC_45D3_A106_E56F7016F2AB | 1.0 | 1 | 0 | 1 | 1 | 1 |
+---------------------------------------------------------+------------------+----------+----------+-----------+----------+-------+
- Nova Scheduler 调用 Placement API 获取 allocation candidate。
[root@localhost ~]# openstack allocation candidate list --resource CUSTOM_RESERVATION_9259448D_88BC_45D3_A106_E56F7016F2AB=1
+---+-----------------------------------------------------------+--------------------------------------+-------------------------------------------------------------+--------+
| # | allocation | resource provider | inventory used/capacity | traits |
+---+-----------------------------------------------------------+--------------------------------------+-------------------------------------------------------------+--------+
| 1 | CUSTOM_RESERVATION_9259448D_88BC_45D3_A106_E56F7016F2AB=1 | 88f75d49-7512-4969-981f-fc893d8d552f | CUSTOM_RESERVATION_9259448D_88BC_45D3_A106_E56F7016F2AB=0/1 | |
+---+-----------------------------------------------------------+--------------------------------------+-------------------------------------------------------------+--------+
[root@localhost ~]# openstack resource provider usage show 88f75d49-7512-4969-981f-fc893d8d552f
+---------------------------------------------------------+-------+
| resource_class | usage |
+---------------------------------------------------------+-------+
| CUSTOM_RESERVATION_9259448D_88BC_45D3_A106_E56F7016F2AB | 0 |
+---------------------------------------------------------+-------+
NOTE：对于普通的虚拟机而言，BlazarFilter 限制不能创建在 Host Aggregate freepool 包含的 Hosts 上。
- 实际启动一个虚拟机后，Nested Resource Provider 的资源（CUSTOM_RESERVATION_{reservation_flavor_uuid}）被扣除，Nested Resource Provider 的 Parent Provider 即实际的 Hypervisor Resource Provider 的资源（CPU、RAM、DISK）被扣除。
[root@localhost ~]# openstack server create --flavor 9259448d-88bc-45d3-a106-e56f7016f2ab --image c4b1cc17-33f6-4964-933a-810fb81f581b --network 0b97b93b-3c55-4ef2-b8d8-ac4065a66c01 VM1
+-------------------------------------+-----------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------------------+-----------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | XYpiXKSYELm6 |
| config_drive | |
| created | 2019-04-03T08:54:25Z |
| flavor | reservation:9259448d-88bc-45d3-a106-e56f7016f2ab (9259448d-88bc-45d3-a106-e56f7016f2ab) |
| hostId | |
| id | 9c1af3e8-4985-46c1-ba25-e2b5f0fc0f52 |
| image | cirros-0.3.4-x86_64-disk (c4b1cc17-33f6-4964-933a-810fb81f581b) |
| key_name | None |
| name | VM1 |
| progress | 0 |
| project_id | e12cff1acee9459b8836cb2bad4ce0e4 |
| properties | |
| security_groups | name='default' |
| status | BUILD |
| updated | 2019-04-03T08:54:26Z |
| user_id | 3501ca5ee41d430ca628be73d63d1d4c |
| volumes_attached | |
+-------------------------------------+-----------------------------------------------------------------------------------------+
[root@localhost ~]# openstack server list
+--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
| 9c1af3e8-4985-46c1-ba25-e2b5f0fc0f52 | VM1 | ACTIVE | private=192.168.1.33 | cirros-0.3.4-x86_64-disk | reservation:9259448d-88bc-45d3-a106-e56f7016f2ab |
+--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
[root@localhost ~]# openstack resource provider usage show 88f75d49-7512-4969-981f-fc893d8d552f
+---------------------------------------------------------+-------+
| resource_class | usage |
+---------------------------------------------------------+-------+
| CUSTOM_RESERVATION_9259448D_88BC_45D3_A106_E56F7016F2AB | 1 |
+---------------------------------------------------------+-------+
[root@localhost ~]# openstack resource provider usage show 7b155336-faaa-4e39-a51b-851ae7495ac5
+----------------+-------+
| resource_class | usage |
+----------------+-------+
| VCPU | 1 |
| MEMORY_MB | 1024 |
| DISK_GB | 20 |
+----------------+-------+
[root@localhost ~]# openstack resource provider list
+--------------------------------------+------------------------------+------------+--------------------------------------+--------------------------------------+
| uuid | name | generation | root_provider_uuid | parent_provider_uuid |
+--------------------------------------+------------------------------+------------+--------------------------------------+--------------------------------------+
| 7b155336-faaa-4e39-a51b-851ae7495ac5 | localhost.localdomain | 5 | 7b155336-faaa-4e39-a51b-851ae7495ac5 | None |
| 88f75d49-7512-4969-981f-fc893d8d552f | blazar_localhost.localdomain | 2 | 7b155336-faaa-4e39-a51b-851ae7495ac5 | 7b155336-faaa-4e39-a51b-851ae7495ac5 |
+--------------------------------------+------------------------------+------------+--------------------------------------+--------------------------------------+
Blazar Stein 依旧存在的局限
- Instance Reservation 要依赖 Host Reservation，并没有实现真正 意义上纯粹的虚拟机预留。
- Instance Reservation 只支持 CPU、RAM、Disk 资源类型，不支持 NUMA、SR-IOV 等高级资源类型。
对于依赖 Host Reservation 问题的思索
我个人认为 “Instance Reservation 依赖 Host Reservation” 的局限性实属无奈之举，这完全是为了实现 Lease（租约）功能所作出的妥协。道理很简单，准守协约的文明人（租约预留资源的虚拟机）和随时来随时走的流氓（原生虚拟机）显然是没法共存于同一个资源池的。如果 Blazar 不存在 Lease 的概念，自然也不会有这样的局限，但没有 Lease 的资源预留很显然也是没有意义的。
所以，回过头再看 Blazar 采用 “基于资源隔离实现的资源预留” 思路或许才是一个好招。