Nova ¶

Local LVM2 storage ¶

A volume group with the name nova is created first with file environments/kolla/files/overlays/nova/nova-compute.conf.

[libvirt]
images_type = lvm
images_volume_group = nova
sparse_logical_volumes = False
disk_cachemodes = "file=directsync,block=directsync,network=directsync"

Local file storage ¶

For using flat file enable in read as follow environments/kolla/files/overlays/nova/nova-compute.conf.

[libvirt]
images_type = default

Note

if the setup demands different backend types for ephemeral storage, it is recommend to seperate this compute hosts in an own host aggreate and bind this aggereate to an own flavor.

Virtual GPUs ¶

https://docs.openstack.org/nova/latest/admin/virtual-gpu.html

NVIDIA¶

https://docs.nvidia.com/grid/latest/grid-vgpu-release-notes-generic-linux-kvm/index.html
identify the available GPUs

$ lspci | grep NVIDIA
02:00.0 3D controller: NVIDIA Corporation Device 1b38 (rev a1)
82:00.0 3D controller: NVIDIA Corporation Device 1b38 (rev a1)
85:00.0 3D controller: NVIDIA Corporation Device 1b38 (rev a1)
86:00.0 3D controller: NVIDIA Corporation Device 1b38 (rev a1)

$ lshw -numeric -C display
WARNING: you should run this program as super-user.
  *-display
       description: 3D controller
       product: NVIDIA Corporation [10DE:1B38]
       vendor: NVIDIA Corporation [10DE]
[...]

resolve the PCI device ID e.g. with https://devicehunt.com

../../../_images/devicehunt-nvidia-p40.png

check https://docs.nvidia.com/grid/gpus-supported-by-vgpu.html

In most environments not all compute nodes are equipped with a GPU. Store the compute node specific configurations in host specific overall files, e.g. in environments/kolla/files/overlays/nova/52-10.betacloud.xyz/nova.conf.

[pci]
alias = { "vendor_id": "10de", "product_id":"1b38", "device_type":"type-PCI", "name":"nvidiap40" }
passthrough_whitelist = { "vendor_id": "10de", "product_id": "1b38" }

set the pci_passthrough:alias" property on a flavor

for one device
$ openstack flavor set --property "pci_passthrough:alias"="nvidiap40:1" 1C-1G-1GB-10GB-GPU

for more devices, the last digit is for the amount of devices
$ openstack flavor set --property "pci_passthrough:alias"="nvidiap40:2" 1C-1G-1GB-10GB-GPU

Hidden KVM for Nvidia GPU ¶

set hidden feature in flavor

$ openstack flavor set --property hide_hypervisor_id=true 1C-1G-1GB-10GB-GPU

Resource isolation ¶

$ python nova_mem_cpu_calc.py HOST_MEMORY_GBYTE OSDS_PER_SERVER GUEST_AVG_MEMORY_GBYTE GUEST_AVG_CPU_UTIL
$ python nova_mem_cpu_calc.py 256 56 6 8 0.1
Inputs:
- Total host RAM in GB: 256
- Total host cores: 56
- Ceph OSDs per host: 6
- Average guest memory size in GB: 8
- Average guest CPU utilization: 10%

Results:
- number of guests allowed based on memory = 28
- number of guest vCPUs allowed = 500
- nova.conf reserved_host_memory = 32000 MB
- nova.conf cpu_allocation_ratio = 8.928571

Compare “guest vCPUs allowed” to “guests allowed based on memory” for actual guest count in environments/kolla/files/overlays/nova.conf.

[DEFAULT]
reserved_host_cpus = 4
reserved_host_memory_mb = 32768
cpu_allocation_ratio = 9

Luks Encryption ¶

For a volume type witch uses Luks Encryption by keymanager it is required to enable it in nova-compute config in the following section environments/kolla/files/overlays/nova/nova-compute.conf.

[[key_manager]
backend = barbican