UPDATED 12/18/2020: Red Hat OpenShift support for Windows Containers is now generally available (GA) and the Windows Machine Config Operator should now appear in the Operator Hub for OpenShift customers. For more information, see the Release Notes in the documentation.

We are proud to announce support for Windows Containers in Red Hat OpenShift will be here in a few weeks. As we are coming to the end of the year, we figured you could spend a little of your leisure time with us to learn how this new feature of OpenShift will allow you to get full support for running Windows worker nodes. Before we delve into the details, let’s start with some basics.

Windows applications on OpenShift, 3 ways

Why Containerize Traditional Windows Apps? 

Accelerate your public and hybrid cloud strategy: Windows Server enjoys a significant presence in the server operating system market, and C# is one of the top six programming languages. There is an enormous opportunity to accelerate customer adoption of Windows Server applications to public cloud via containers. 

Reduce infrastructure and management costs for Windows applications. You do not need to spin up an entire virtual machine to run your Microsoft SQL database or your IIS web server. The small, lightweight nature of containers provides more density and allows them to be moved easily across bare metal systems as well as public, private, hybrid, and multi-cloud environments.

Gain applications portability, agility, and control: Containers provide an attractive means of application packaging and delivery because of their low overhead and greater portability. Instead of virtualizing the hardware and carrying forward multiple full stacks of software from the application to the operating system, Windows containers rest on top of a single Windows operating system instance. Applications can then be built in Visual Studio or Visual Studio Code, containerized, and run anywhere, such as on OpenShift on premises, OpenShift on Azure, or OpenShift on AWS.

Use Cases for Windows Workloads on OpenShift

Based on Gartner’s 5R approach to application modernization, here are the different use cases for running Windows workloads on OpenShift:

Step

OpenShift Feature

Use Case

Advantages

Trade Offs

Rehost

OpenShift Virtualization

Lift & Shift Windows VMs to OpenShift

Easy and low friction 

Fewer of benefits of containerization

Refactor

Windows Machine Config Operator 

Containerize and run traditional .Net framework apps on Windows Server Containers and deploy to Windows worker nodes on OpenShift 

Benefits of containerization and OpenShift

Evolving Windows container ecosystem, supported only for newer version of Windows including Windows Server 2019

Rearchitect

RHEL/RHCOS containers 

Migrate traditional .Net frameworks apps to .Net Core and deploy to RHEL containers in OpenShift. 

Full benefit of containerization and OpenShift, highly evolved community

Migration effort involved, time consuming

Rebuild

RHEL/RHCOS containers

Build Cloud Native apps using Linux containers and deploy to RHEL/RHELCoreOS on OpenShift.

Full benefit of containerization and OpenShift, highly evolved community

Net new development may not be an option for customers running in maintenance mode

 

Windows Machine Config Operator (WMCO)

The Windows Machine Config Operator (WMCO) is the entry point for OpenShift customers who want to run Windows workloads on their clusters. The intent of this feature is to allow a cluster administrator to add a Windows worker node as a day 2 operation with a prescribed configuration to an installer provisioned OpenShift 4.6 cluster and enable scheduling of Windows workloads. The Prerequisite is an OpenShift 4.6+ cluster configured with hybrid OVN Kubernetes networking.

WMCO Supported Platforms

Platform

Supported Now

Coming Soon

Red Hat OpenShift Container Platform (OCP) on Azure

Yes

 

OCP on AWS

Yes

 

OCP on vSphere

No

Yes

OCP on Bare metal

No

Yes

OCP on Red Hat Virtualization

No

Yes

OCP on Red Hat OpenStack Platform

No

Yes

Managed OpenShift Offerings (Azure Red Hat OpenShift, OpenShift Dedicated, etc.)

No

Yes

 

Supported Operating Systems for Windows Worker Nodes

The following Windows Server operating systems are supported in the initial release of the WMCO: Windows Server Long-Term Servicing Channel (LTSC): Windows Server 2019*

* Has to be on version 10.0.17763.1457 or older

Windows Machine Config Operator Explained

The Windows Machine Config Operator configures Windows Machines into nodes, enabling Windows container workloads to be run on Red Hat OpenShift. The operator is configured to watch for machines with a machine.openshift.io/os-id: Windows label. The way a user will initiate the process is by creating a MachineSet which uses a Windows image with the Docker* container runtime installed. The operator will do all the necessary steps to configure the underlying VM so that it can join the cluster as a worker node.

How the WMCO operator works with OpenShift to provide Window container support

*Currently, the Docker-formatted container runtime is used in Windows nodes. Kubernetes is deprecating Docker as a container runtime; you can reference the Kubernetes documentation for more information in Docker deprecation. Containerd will be the new supported container runtime for Windows nodes in a future release of Kubernetes.

Windows Machine Config Operator Workflow

 

Using the Windows Machine Config Operator

The operator can be installed from the operators catalog on OperatorHub. It can also be built and installed from source manually, see the development instructions.

Install Operator

Navigate to the in-cluster Operator Hub and search for the Windows Machine Config Operator and Click Install

WMCO Operator icon in the Operator Hub

 

Once the openshift-windows-machine-config-operator namespace has been created, a secret must be created containing the private key that will be used to access the Windows VMs:

# Create secret containing the private key in the openshift-windows-machine-config-operator namespace
oc create secret generic cloud-private-key --from-file=private-key.pem=/path/to/key -n openshift-window

 

Create a MachineSet

The way a user will initiate the process is by creating a MachineSet which uses a Windows image with the Docker container runtime installed.  It usually takes about 15 minutes for the Windows Machine to be configured as a worker node. Ensure the Windows Node is in a Ready state before deploying a workload

apiVersion: machine.openshift.io/v1beta1
kind: MachineSet
metadata:
 labels:
   machine.openshift.io/cluster-api-cluster: <infrastructureID>
 name: winworker
 namespace: openshift-machine-api
spec:
 replicas: 1
 selector:
   matchLabels:
     machine.openshift.io/cluster-api-cluster: <infrastructureID>
     machine.openshift.io/cluster-api-machineset: winworker
 template:
   metadata:
     labels:
       machine.openshift.io/cluster-api-cluster: <infrastructureID>
       machine.openshift.io/cluster-api-machine-role: worker
       machine.openshift.io/cluster-api-machine-type: worker
       machine.openshift.io/cluster-api-machineset: winworker
       machine.openshift.io/os-id: Windows
   spec:
     metadata:
       labels:
         node-role.kubernetes.io/worker: ""
     providerSpec:
       value:
         apiVersion: azureproviderconfig.openshift.io/v1beta1
         credentialsSecret:
           name: azure-cloud-credentials
           namespace: openshift-machine-api
         image:
           offer: WindowsServer
           publisher: MicrosoftWindowsServer
           resourceID: ""
           sku: 2019-Datacenter-with-Containers
           version: 10.0.17763.1457
         kind: AzureMachineProviderSpec
         location: <location>
         managedIdentity: <infrastructureID>-identity
         networkResourceGroup: <infrastructureID>-rg
         osDisk:
           diskSizeGB: 128
           managedDisk:
             storageAccountType: Premium_LRS
           osType: Windows
         publicIP: false
         resourceGroup: <infrastructureID>-rg
         subnet: <infrastructureID>-worker-subnet
         userDataSecret:
           name: windows-user-data
           namespace: openshift-machine-api
         vmSize: Standard_D2s_v3
         vnet: <infrastructureID>-vnet
         zone: "<zone>"

WMCO configures machines to worker nodes

The operator is configured to watch for Machines with a machine.openshift.io/os-id: Windows label. The operator will do all the necessary steps to configure the underlying VM so that it can join the cluster as a worker node.

Upgrades

As a cluster administrator, you can upgrade the Windows Machine Config Operator (WMCO) that has been previously installed using Operator Lifecycle Manager (OLM) on your OpenShift Container Platform cluster.

When a new version of WMCO is released that is compatible with the current cluster version, an operator upgrade will take place, which will result in the Kubernetes components in the Windows Machine being upgraded. If the approval strategy in the Subscription is set to Automatic, the upgrade process initiates as soon as a new Operator version is available in the selected channel. If the approval strategy is set to Manual, you must manually approve pending upgrades.

For a non-disruptive upgrade, WMCO terminates the Windows Machines configured by previous version of WMCO and re-creates them using the current version. This is done by deleting the Machine object that results in the drain and deletion of the Windows node. To facilitate an upgrade, WMCO adds a version annotation to all the configured nodes. During an upgrade, a mismatch in version annotation will result in deletion and re-creation of Windows Machine. In order to have minimal service disruption during an upgrade, WMCO makes sure that the cluster will have at least one Windows Machine per MachineSet in the running state.

Upgrade Procedure

The procedure for an upgrade is as follows:

  • A new WMCO version is released
  • If the current cluster version fulfills the minimum Kubernetes version requirement, OLM upgrades WMCO. If the cluster version is not high enough, the WMCO upgrade will occur once it is.
  • The new WMCO reconciles as usual, ensuring that all unconfigured Windows Machines are configured and join the cluster as a node. Each of them is given an annotation indicating the WMCO version that configured them.
  • Each Windows node is checked for the WMCO version annotation, if the annotated version of a Windows node does not match the WMCO version, and the number of unavailable Windows nodes is less than maxUnhealthy value, the associated Machine is deleted.
  • When a replacement Machine is created by the Machine API Operator, WMCO will reconcile again and configure the VM. This will repeat until all Windows nodes have been configured by the upgraded WMCO.

WMCO Upgrade Process

Upgrade Summary

Component

Upgraded by WMCO

Support Offered by

kubelet

Yes

Red Hat & Microsoft

kube-proxy

Yes

Red Hat & Microsoft

hybrid-overlay

Yes

Red Hat & Microsoft

CNI

Yes

Red Hat & Microsoft

Worker ignition files

Yes

Red Hat

Windows Operating System

No

Microsoft

Apply Windows CVEs, security patches

No

Microsoft



WMCO is not responsible for Windows operating system updates. The cluster administrator provides the Window image while creating the VMs, and hence, the cluster administrator is responsible for providing an updated image. The cluster administrator can provide an updated image by changing the image in the MachineSet spec.

Enable Logging for Windows Containers

The ability to view logs is a basic requirement for debugging purposes and is commonly used by most customers. It is also critical for security as logs are used to figure out if a malicious event occurred or if secrets were being leaked. For Windows containers in OpenShift, here are the available logs and methods to collect them:

  • C:\var\log\kube-proxy\kube-proxy.exe.INFO
  • C:\var\log\kube-proxy\kube-proxy.exe.ERROR
  • C:\var\log\kube-proxy\kube-proxy.exe.WARNING
  • C:\var\log\hybrid-overlay\hybrid-overlay.log
  • C:\var\log\kubelet\kubelet.log
  • %APPDATA%\Local\Docker\log.txt

You can view OpenShift Container Platform cluster logs for Windows pods in the CLI: 

$ oc logs -f windows-pod-name -n openshift-windows

You can collect the following logs with must-gather:

Logs for Windows pods

 

Please note that viewing logs in the OpenShift Container Platform web console is not yet supported for Windows pods.

"With the Windows Machine Config Operator we are providing a jointly supported solution that helps organizations more quickly migrate on-premises Windows workloads to Red Hat OpenShift — helping to simplify the pathway to container adoption,” said Subodh Bhargava, Senior Program Manager, Microsoft. “Those Windows containers can then be seamlessly managed side-by-side with Linux containers, providing organizations with increased visibility and control over their container environments."

When you get back from your end of year holidays, Windows Container Support for Red Hat OpenShift will be waiting for you, giving you full support for running Windows worker nodes. Please take the Windows Machine Config Operator for a spin from the on-cluster #OperatorHub and provide feedback by opening GitHub issues.