How-to: Build VPP FD.IO with Mellanox DPDK PMD on top CentOS 7.7 with inbox drivers.

https://docs.mellanox.com/pages/releaseview.action?pageId=15053848

Prerequisites

1. Install CentOS 7.7

2. Set up an account to use the Linux Foundation resources. To set up the account, refer to The Linux Foundation website: https://identity.linuxfoundation.org.

3. Set up a Gerrit account and apply an ssh public key. Follow the instructions on the FD.IO wiki web page: https://wiki.fd.io/view/DEV/Setting_up_Gerrit#Log_into_Gerrit

4. Install the following prerequisites packages:

Cli command

yum install -y git cpp gcc rpm-build openssl-devel libmnl-devel numactl-devel epel-release net-tools rdma-core-devel nasm  

Installation, Compilation and Configuration

1. Download VPP main master branch.

Cli commands

git clone ssh://<USERNAME>@gerrit.fd.io:29418/vpp.git

git checkout origin/stable/<"Release_version_number">

• Use your Linux Foundation user account name (see Prerequisites above) instead of <USERNAME> as shows in above example.
• To find desired <"Release_version_number"> do: "git -r branch" form VPP root git directory.

2. For VPP 18.07 ONLY: Enable the mlx5 PMD driver compilation as follow:

Cli commands

cd /vpp

sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk 

3. Compile and install VPP with DLopen linkage

Cli commands

cd /vpp

make install-dep

make dpdk-install-dev DPDK_MLX5_PMD=y DPDK_MLX5_PMD_DLOPEN_DEPS=y

## Copy manually newly compiled shared module: ##

cp /opt/vpp/external/x86_64/lib/librte_pmd_mlx5_glue.so* /usr/lib64/

## For VPP 18.07 ONLY, check vpp-dpdk-devel package is installed: ##

rpm -qa | grep vpp-dpdk-devel

## Compile new binary: ##

make build-release vpp_uses_dpdk_mlx5_pmd=yes DPDK_MLX5_PMD_DLOPEN_DEPS=y

## Building RPM packages: ##

make pkg-rpm vpp_uses_dpdk_mlx5_pmd=yes DPDK_MLX5_PMD_DLOPEN_DEPS=y

## Install VPP RPMs: ##

rpm -ivh build-root/vpp-sel*.rpm

rpm -ivh build-root/vpp-lib*.rpm

rpm -ivh build-root/vpp-19*.rpm

rpm -ivh build-root/vpp-plug*.rpm 

4. Compile and install without DLopen linkage

Cli commands

cd /vpp

make install-dep

make dpdk-install-dev DPDK_MLX5_PMD=y DPDK_MLX5_PMD_DLOPEN_DEPS=n

## Compile new binary: ##

make build-release DPDK_MLX5_PMD=y DPDK_MLX5_PMD_DLOPEN_DEPS=n

## Building RPM packages: ##

make pkg-rpm vpp_uses_dpdk_mlx5_pmd=yes DPDK_MLX5_PMD_DLOPEN_DEPS=n

## Install VPP RPMs: ##

rpm -ivh build-root/vpp-sel*.rpm

rpm -ivh build-root/vpp-lib*.rpm

rpm -ivh build-root/vpp-19*.rpm

rpm -ivh build-root/vpp-plug*.rpm 

5. Edit the /etc/vpp/startup.conf start file to include the following:

Important: Uncomment no-mulit-seg parameter. It disables jumbo packets handling and improves performances significantly.

File

unix {

  nodaemon

  log /tmp/vpp.log

  full-coredump

}

dpdk {

        dev 0000:07:00.0

        dev 0000:07:00.1

        no-multi-seg

        dev default {

                num-rx-queues 2

        }

}

Insert PCI slot numbers of ConnectX-4/5 interfaces with correct IDs at dpdk { dev } section, as above example.

To verify the correct IDs use:

Cli command

lspci -nn | grep -i mel

07:00.0 Ethernet controller [0200]: Mellanox Technologies MT27620 Family [15b3:1013]

07:00.1 Ethernet controller [0200]: Mellanox Technologies MT27620 Family [15b3:1013]

6. Start VPP.

Cli command

vpp -c /etc/vpp/startup.conf

7. Enter the VPP Command Line Interface.

Cli command

vppctl

8. Configure L3 interfaces as shown in the example below:

VPP Cli

_______    _        _   _____  ___

__/ __/ _   (_)__    | | / / _ / _

_/ _// // / / / _    | |/ / ___/ ___/

/_/ /____(_)_/\___/   |___/_/  /_/

 

vpp#  set int ip address HundredGigabitEthernet7/0/0 1.1.1.1/24

vpp#  set interface state HundredGigabitEthernet7/0/0 up

vpp#  set int ip address HundredGigabitEthernet7/0/1 2.2.2.2/24

vpp#  set interface state HundredGigabitEthernet7/0/1 up

vpp# show int

              Name               Idx       State          Counter          Count

HundredGigabitEthernet7/0/0              1         up

HundredGigabitEthernet7/0/1              2         up

vpp# show int address

HundredGigabitEthernet7/0/0 (up):

  1.1.1.1/24

HundredGigabitEthernet7/0/1 (up):

  2.2.2.2/24

local0 (dn):