Multus is the open source project that enables Kubernetes pods to attach to multiple networks. It does this by acting as a ‘meta’ plug-in, which is a fancy name for a plug-in that calls other plug-ins. First, let’s take a quick look at the ‘pluggable’ nature of container networking.

In 2017, the Cloud Native Computing Foundation (CNCF) voted to accept the Container Networking Interface (CNI) as the 10th hosted project. The CNI project is a minimal specification to define a common interface between the network plug-ins and container execution. CNI has 3 main components:

  • CNI specification – defines the API between runtimes and network plug-ins
  • Plug-ins – provide network setup for different use cases (reference examples)
  • Library – a Go implementation of the CNI specification that runtimes can use

Currently, the default OpenShift CNI is OpenShift SDN (network-policy), which configures an overlay network using Open vSwitch (OVS 2.11). The following diagram shows the CNI options for OpenShift and the status of each (supported, validated, etc. …):

 

 

In OCP 3.x by default, a standard pod configuration allows for a connection to one network interface.

 

Multus allows pods to have multiple network interface connections that can address various use cases; for example:

  • Splitting the control/data plane traffic, mostly applicable to vnf/cnf applications
  • Legacy/storage intensive applications
  • Multitenant networks

 

 

OpenShift provides the following CNI plug-ins for creating additional networks in your cluster:

  • Bridge: allows Pods on the same host to communicate with each other and the host
  • Host-device: allows Pods access to a physical Ethernet network device on the host system
  • Macvlan: allows Pods on a host to communicate with other hosts and Pods on those hosts by using a physical network interface. Each Pod that is attached to a macvlan-based additional network is provided a unique MAC address
  • Ipvlan: allows Pods on a host to communicate with other hosts and Pods on those hosts, similar to a macvlan-based additional network. Unlike a macvlan-based additional network, each Pod shares the same MAC address as the parent physical network interface
  • SR-IOV: allows Pods to attach to a virtual function (VF) interface on SR-IOV capable hardware on the host system

Currently, setting up secondary network interfaces is a ‘day 2’ operation. The following is an example where I’ve added a bridge ‘additional network’ to the network operator (highlighted):

Add network-

oc edit networks.operator.openshift.io cluster

# Please edit the object below. Lines beginning with a '#' will be ignored,

# and an empty file will abort the edit. If an error occurs while saving this file will be

# reopened with the relevant failures.

#

apiVersion: operator.openshift.io/v1

kind: Network

metadata:

creationTimestamp: 2020-02-05T13:13:48Z

generation: 2

name: cluster

resourceVersion: "24716"

selfLink: /apis/operator.openshift.io/v1/networks/cluster

uid: 3f48c7b7-4e1c-443e-8b52-d161e6e48adb

spec:

additionalNetworks:

- name: multus1

rawCNIConfig: '{ "cniVersion": "0.3.1", "type": "bridge", "master": "eth1", "ipam":

{ "type": "static", "addresses": [ { "address": "191.168.1.1/24" } ] } }'

type: Raw

clusterNetwork:

- cidr: 10.128.0.0/14

hostPrefix: 23

defaultNetwork:

type: OpenShiftSDN

logLevel: ""

serviceNetwork:

- 172.30.0.0/16

 

 

Then create a Pod with the following annotation (note the name -multus1 defined above):;

 

apiVersion: v1

kind: Pod

metadata:

name: example

labels:

app: hello-openshift

namespace: default

annotations:

k8s.v1.cni.cncf.io/networks: multus1

spec:

containers:

- name: hello-openshift

image: openshift/hello-openshift

ports:

- containerPort: 8080

 

And when the Pod is spun up, you can see the default plug-in ‘eth0’ and the secondary network interface ‘net1’;

 

kind: Pod

apiVersion: v1

metadata:

name: multus-example

namespace: default

selfLink: /api/v1/namespaces/default/pods/multus-example

uid: 0d68eb92-d74c-4f08-8924-6dcdcde26e7a

resourceVersion: '113663'

creationTimestamp: '2020-02-05T19:38:15Z'

labels:

app: hello-openshift

annotations:

k8s.v1.cni.cncf.io/networks: multus1

k8s.v1.cni.cncf.io/networks-status: |-

[{

"name": "openshift-sdn",

"interface": "eth0",

"ips": [

"10.129.2.19"

],

"dns": {},

"default-route": [

"10.129.2.1"

]

},{

"name": "multus1",

"interface": "net1",

"ips": [

"191.168.1.1"

],

"mac": "fa:70:12:98:e4:5f",

"dns": {}

}]

 

For high performance networking workloads SR-IOV provides near-native networking performance on bare-metal.This feature is Tech Preview for OpenShift 4.3.

 

 

The SR-IOV operator can be installed from the console and provides the following features:

  • Discover the SR-IOV network device in the cluster.
  • Initialize the supported SR-IOV NIC models on nodes.
  • Provision the SR-IOV network device plug-in on nodes.
  • Provision the SR-IOV CNI plug-in executable on nodes.
  • Provision the Network Resources Injector in the cluster.
  • Manage configuration of SR-IOV network device plug-in.
  • Generate NetworkAttachmentDefinition custom resources (CR) for the SR-IOV CNI plug-in.

 

Tested CNI plug-ins can be found here.

 


Categories

Kubernetes, Multus, OpenShift 4, Networking

< Back to the blog