【问题描述】
创建boot_from_volume的虚机时，磁盘后端为NFS，创建失败。
【错误日志】
nova-compute模块
Could not open '/var/lib/nova/mnt/b4a8885bf81bd517182c73c35df4496d/volume-598ddf2f-94c6-495d-b57b-20ebf2c337a8': Permission denied
|
: libvirtError: internal error: process exited while connecting to monitor: 2019-08-26T06:33:56.800031Z qemu-kvm: -drive file=/var/lib/nova/mnt/b4a8885bf81bd517182c73c35df4496d/volume-598ddf2f-94c6-495d-b57b-20ebf2c337a8,format=raw,if=none,id=drive-virtio-disk0,serial=598ddf2f-94c6-495d-b57b-20ebf2c337a8,cache=none,aio=native: Could not open '/var/lib/nova/mnt/b4a8885bf81bd517182c73c35df4496d/volume-598ddf2f-94c6-495d-b57b-20ebf2c337a8': Permission denied 2019-08-26 14:33:57.414 32 ERROR nova.virt.libvirt.driver [req-1e4be617-e159-4143-b925-00c52ea8ee05 f100901926164d48a913314f2776dfb4 e5debbe49ff24cd68f9a35cc2a934beb - default default] [instance: d512503a-467b-4992-8c51-19de191e131b] Failed to start libvirt guest: libvirtError: internal error: process exited while connecting to monitor: 2019-08-26T06:33:56.800031Z qemu-kvm: -drive file=/var/lib/nova/mnt/b4a8885bf81bd517182c73c35df4496d/volume-598ddf2f-94c6-495d-b57b-20ebf2c337a8,format=raw,if=none,id=drive-virtio-disk0,serial=598ddf2f-94c6-495d-b57b-20ebf2c337a8,cache=none,aio=native: Could not open '/var/lib/nova/mnt/b4a8885bf81bd517182c73c35df4496d/volume-598ddf2f-94c6-495d-b57b-20ebf2c337a8': Permission denied 2019-08-26 14:33:57.613 32 WARNING nova.compute.manager [req-dff6b914-4398-4a3f-876d-22a2972a1158 9e49905829cf4f72aa143aa5417c0e91 e49b1ca2e06c4aae87cbc70b6782d86a - default default] [instance: d512503a-467b-4992-8c51-19de191e131b] Received unexpected event network-vif-plugged-4aead7f4-3d5d-41bd-bd96-524219c69b5a for instance with vm_state building and task_state spawning. 2019-08-26 14:33:58.276 32 INFO os_vif [req-1e4be617-e159-4143-b925-00c52ea8ee05 f100901926164d48a913314f2776dfb4 e5debbe49ff24cd68f9a35cc2a934beb - default default] Successfully unplugged vif VIFBridge(active=False,address=fa:16:3e:a8:05:ba,bridge_name='qbr4aead7f4-3d',has_traffic_filtering=True,id=4aead7f4-3d5d-41bd-bd96-524219c69b5a,network=Network(8ff093db-c8fb-4909-9ecb-95291fcfccd1),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap4aead7f4-3d') 2019-08-26 14:33:59.069 32 INFO nova.virt.libvirt.driver [req-1e4be617-e159-4143-b925-00c52ea8ee05 f100901926164d48a913314f2776dfb4 e5debbe49ff24cd68f9a35cc2a934beb - default default] [instance: d512503a-467b-4992-8c51-19de191e131b] Deleting instance files /var/lib/kolla/volumes/nova/d512503a-467b-4992-8c51-19de191e131b_del 2019-08-26 14:33:59.070 32 INFO nova.virt.libvirt.driver [req-1e4be617-e159-4143-b925-00c52ea8ee05 f100901926164d48a913314f2776dfb4 e5debbe49ff24cd68f9a35cc2a934beb - default default] [instance: d512503a-467b-4992-8c51-19de191e131b] Deletion of /var/lib/kolla/volumes/nova/d512503a-467b-4992-8c51-19de191e131b_del complete 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [req-1e4be617-e159-4143-b925-00c52ea8ee05 f100901926164d48a913314f2776dfb4 e5debbe49ff24cd68f9a35cc2a934beb - default default] [instance: d512503a-467b-4992-8c51-19de191e131b] Instance failed to spawn: libvirtError: internal error: process exited while connecting to monitor: 2019-08-26T06:33:56.800031Z qemu-kvm: -drive file=/var/lib/nova/mnt/b4a8885bf81bd517182c73c35df4496d/volume-598ddf2f-94c6-495d-b57b-20ebf2c337a8,format=raw,if=none,id=drive-virtio-disk0,serial=598ddf2f-94c6-495d-b57b-20ebf2c337a8,cache=none,aio=native: Could not open '/var/lib/nova/mnt/b4a8885bf81bd517182c73c35df4496d/volume-598ddf2f-94c6-495d-b57b-20ebf2c337a8': Permission denied 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] Traceback (most recent call last): 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2296, in _build_resources 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] yield resources 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2111, in _build_and_run_instance 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] block_device_info=block_device_info) 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2965, in spawn 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] destroy_disks_on_failure=True) 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5516, in _create_domain_and_network 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] destroy_disks_on_failure) 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] self.force_reraise() 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] six.reraise(self.type_, self.value, self.tb) 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5486, in _create_domain_and_network 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] post_xml_callback=post_xml_callback) 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5404, in _create_domain 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] guest.launch(pause=pause) 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 144, in launch 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] self._encoded_xml, errors='ignore') 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] self.force_reraise() 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] six.reraise(self.type_, self.value, self.tb) 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 139, in launch 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] return self._domain.createWithFlags(flags) 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] result = proxy_call(self._autowrap, f, *args, **kwargs) 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] rv = execute(f, *args, **kwargs) 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] six.reraise(c, e, tb) 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] rv = meth(*args, **kwargs) 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1099, in createWithFlags 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] libvirtError: internal error: process exited while connecting to monitor: 2019-08-26T06:33:56.800031Z qemu-kvm: -drive file=/var/lib/nova/mnt/b4a8885bf81bd517182c73c35df4496d/volume-598ddf2f-94c6-495d-b57b-20ebf2c337a8,format=raw,if=none,id=drive-virtio-disk0,serial=598ddf2f-94c6-495d-b57b-20ebf2c337a8,cache=none,aio=native: Could not open '/var/lib/nova/mnt/b4a8885bf81bd517182c73c35df4496d/volume-598ddf2f-94c6-495d-b57b-20ebf2c337a8': Permission denied 2019-08-26 14:33:59.256 32 ERROR nova.compute.manager [instance: d512503a-467b-4992-8c51-19de191e131b] 2019-08-26 14:33:59.269 32 INFO nova.compute.manager [req-1e4be617-e159-4143-b925-00c52ea8ee05 f100901926164d48a913314f2776dfb4 e5debbe49ff24cd68f9a35cc2a934beb - default default] [instance: d512503a-467b-4992-8c51-19de191e131b] Terminating instance |
【定位原因】
由于使用了K8S，nova-compute这个pod是使用root用户启动的，但是在使用libvirt调用某些接口的时候，默认使用了nova用户。
而/var/lib/nova/mnt/文件夹的属主为root，其他用户无写权限，nova用户写失败，导致创建虚机失败。
【修改方法】
修改/var/lib/nova/mnt/文件夹的属主为nova，或者给nova用户赋权即可解决。