diff --git a/doc/source/reference/networking-guide.rst b/doc/source/reference/networking-guide.rst index 52fd39971a..c9dc65ce04 100644 --- a/doc/source/reference/networking-guide.rst +++ b/doc/source/reference/networking-guide.rst @@ -271,3 +271,212 @@ prior to upgrading. On ubuntu network manager is required for tunnel networking. This requirement will be removed in the future. + + +Neutron SRIOV +============= + +Preparation and deployment +-------------------------- + +SRIOV requires specific NIC and BIOS configuration and is not supported on all +platforms. Consult NIC and platform specific documentation for instructions +on enablement. + +Modify the configuration file ``/etc/kolla/globals.yml``: + +:: + + enable_neutron_sriov: "yes" + +Modify the file ``/etc/kolla/config/neutron/ml2_conf.ini``. Add ``sriovnicswitch`` +to the mechanism drivers and add the provider networks for use by SRIOV. Both +flat and VLAN are configured with the same physical network name in this example: + +:: + + [ml2] + mechanism_drivers = openvswitch,l2population,sriovnicswitch + + [ml2_type_vlan] + network_vlan_ranges = sriovtenant1:1000:1009 + + [ml2_type_flat] + flat_networks = sriovtenant1 + + +Modify the file ``/etc/kolla/config/nova.conf``. The Nova Scheduler service +on the control node requires the ``PciPassthroughFilter`` to be added to the +list of filters and the Nova Compute service(s) on the compute node(s) need +PCI device whitelisting: + +:: + + [DEFAULT] + scheduler_default_filters = , PciPassthroughFilter + scheduler_available_filters = nova.scheduler.filters.all_filters + + [pci] + passthrough_whitelist = [{"devname": "ens785f0", "physical_network": "sriovtenant1"}] + + +Modify the file ``/etc/kolla/config/neutron/sriov_agent.ini``. Add physical +network to interface mapping. Specific VFs can also be excluded here. Leave +blank to enable all VFs for the interface: + +:: + + [sriov_nic] + physical_device_mappings = sriovtenant1:ens785f0 + exclude_devices = + +Run deployment. + +Verification +------------ + +Check that VFs were created on the compute node(s). VFs will appear in the +output of both ``lspci`` and ``ip link show``. For example: + +:: + + lspci | grep net + 05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) + +:: + + ip -d link show ens785f0 + 4: ens785f0: mtu 1500 qdisc mq master ovs-system state UP mode DEFAULT qlen 1000 + link/ether 90:e2:ba:ba:fb:20 brd ff:ff:ff:ff:ff:ff promiscuity 1 + openvswitch_slave addrgenmode eui64 + vf 0 MAC 52:54:00:36:57:e0, spoof checking on, link-state auto, trust off + vf 1 MAC 52:54:00:00:62:db, spoof checking on, link-state auto, trust off + vf 2 MAC fa:16:3e:92:cf:12, spoof checking on, link-state auto, trust off + vf 3 MAC fa:16:3e:00:a3:01, vlan 1000, spoof checking on, link-state auto, trust off + +Verify the SRIOV Agent container is running on the compute node(s): + +:: + + docker ps --filter name=neutron_sriov_agent + CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + b03a8f4c0b80 10.10.10.10:4000/registry/centos-source-neutron-sriov-agent:17.04.0 "kolla_start" 18 minutes ago Up 18 minutes neutron_sriov_agent + + +Verify the SRIOV Agent service is present and UP: + +:: + + openstack network agent list + +--------------------------------------+--------------------+-------------+-------------------+-------+-------+---------------------------+ + | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | + +--------------------------------------+--------------------+-------------+-------------------+-------+-------+---------------------------+ + | 7c06bda9-7b87-487e-a645-cc6c289d9082 | NIC Switch agent | av09-18-wcp | None | :-) | UP | neutron-sriov-nic-agent | + +--------------------------------------+--------------------+-------------+-------------------+-------+-------+---------------------------+ + +Create a new provider network. Set ``provider-physical-network`` to the +physical network name that was configured in ``/etc/kolla/config/nova.conf``. +Set ``provider-network-type`` to the desired type. If using VLAN, ensure +``provider-segment`` is set to the correct VLAN ID. Type VLAN is used in this example: + + +:: + + openstack network create --project=admin \ + --provider-network-type=vlan \ + --provider-physical-network=sriovtenant1 \ + --provider-segment=1000 \ + sriovnet1 + +Create a subnet with a DHCP range for the provider network: + +:: + + openstack subnet create --network=sriovnet1 \ + --subnet-range=11.0.0.0/24 \ + --allocation-pool start=11.0.0.5,end=11.0.0.100 \ + sriovnet1_sub1 + +Create a port on the provider network with vnic_type set to direct: + +:: + + openstack port create --network sriovnet1 --vnic-type=direct sriovnet1-port1 + +Start a new instance with the SRIOV port assigned: + +:: + + openstack server create --flavor flavor1 \ + --image fc-26 \ + --nic port-id=`openstack port list | grep sriovnet1-port1 | awk '{print $2}'` \ + vm1 + +Verify the instance boots with the SRIOV port. Verify VF assignment by running +dmesg on the compute node where the instance was placed. + +:: + + dmesg + [ 2896.849970] ixgbe 0000:05:00.0: setting MAC fa:16:3e:00:a3:01 on VF 3 + [ 2896.850028] ixgbe 0000:05:00.0: Setting VLAN 1000, QOS 0x0 on VF 3 + [ 2897.403367] vfio-pci 0000:05:10.4: enabling device (0000 -> 0002) + +For more information see `OpenStack SRIOV documentation `_. + + +Nova SRIOV +========== + +Preparation and deployment +-------------------------- + +Nova provides a separate mechanism to attach PCI devices to instances that +is independent from Neutron. Using the PCI alias configuration option in +nova.conf, any PCI device (PF or VF) that supports passthrough can be attached +to an instance. One major drawback to be aware of when using this method is +that the PCI alias option uses a device's product id and vendor id only, +so in environments that have NICs with multiple ports configured for SRIOV, +it is impossible to specify a specific NIC port to pull VFs from. + +Modify the file ``/etc/kolla/config/nova.conf``. The Nova Scheduler service +on the control node requires the ``PciPassthroughFilter`` to be added to the list +of filters and the Nova Compute service(s) on the compute node(s) need PCI +device whitelisting. The Nova API service on the control node and the Nova +Compute service on the compute node also require the ``alias`` option under the +``[pci]`` section. The alias can be configured as 'type-VF' to pass VFs or 'type-PF' +to pass the PF. Type-VF is shown in this example: + +:: + + [DEFAULT] + scheduler_default_filters = , PciPassthroughFilter + scheduler_available_filters = nova.scheduler.filters.all_filters + + [pci] + passthrough_whitelist = [{"vendor_id": "8086", "product_id": "10fb"}] + alias = [{"vendor_id":"8086", "product_id":"10ed", "device_type":"type-VF", "name":"vf1"}] + +Run deployment. + +Verification +------------ + +Create (or use an existing) flavor, and then configure it to request one PCI device +from the PCI alias: + +:: + + openstack flavor set sriov-flavor --property "pci_passthrough:alias"="vf1:1" + +Start a new instance using the flavor: + +:: + + openstack server create --flavor sriov-flavor --image fc-26 vm2 + + +Verify VF devices were created and the instance starts successfully as in +the Neutron SRIOV case. + +For more information see `OpenStack PCI passthrough documentation `_. \ No newline at end of file