The Red Hat Performance Team, along with our partners at Intel, have been working together to leverage the latest technologies and features in the container orchestration space to minimize the overhead from OpenShift when containerizing CPU compute-intensive FSI workloads. The team used the well-known STAC-A2 benchmark from STAC® (the Securities Technology Analysis Center), to prove out the technology.

STAC-A2 is the technology benchmark standard focused on financial market risk analysis. Designed by quants and technologists from some of the world's largest banks, STAC-A2 reports the performance, scaling, quality, and resource efficiency of any technology stack that is able to handle the workload (Monte Carlo estimation of Heston-based Greeks for a path-dependent, multi-asset option with early exercise).

In the past, our teams have demonstrated the performance of STAC-A2 on OpenShift running with GPUs (result). In this round of STAC benchmarking, we wanted to measure the overhead for compute-intensive CPU-based workloads with standard OpenShift kubelet and operator processes consuming some CPU resources by comparing to a bare metal system with the same hardware configuration. Like the GPU-based tests, these tests were unaudited, so we can’t divulge anything about the results, but let’s just say we were pleased.

Note: This unaudited STAC-A2 configuration is virtually identical to the one published by Intel in INTC190402. The only difference is with the software stack, which now includes OpenShift and a newer version of RHEL and Intel PSXE.

 

Deploying STAC-A2 on OpenShift

 

In addition to our benchmark result, which you can find here in the STAC Vault, we have created a new STAC repository to ease with building of the container image (including all STAC-A2 artifacts and binaries), deployment, and benchmarking on OpenShift for partners wanting to run STAC-A2 on their own OpenShift clusters. These tools can be found in the STAC repository Red Hat Openshift for STAC-A2.

There are two key artifacts involved in this process: a template containing a BuildConfig and ImageStream, and a template for deployment of the container image that we just built.

  1. The first step is to modify the BuildConfig `stac-a2-intel-buildconfig.yaml` to link to a few privately accessible assets such as the STAC-A2 pack and implementation as well as the Intel Parallel Studio XE license required for compilation. (see README.md in the repository for more information)
  2. We also want to change the compilation flags in the BuildConfig to suit the host CPU capabilities:
    cmake -G "Unix Makefiles" -DICC_VERSION=18 -DHOST_ARCH=SKX && \
    make -j$(nproc)
  3. We will decide which OpenShift node (usually one of the worker nodes) we want our build and our workload pod to land on and then label that node:
    $ oc label node <node_name> workload=stac-a2
    node/<node_name> labeled
  4. We will create a namespace for our deployment with a node-selector. This way, all artifacts that are created in this namespace will land on the matching labelled nodes:
    $ oc adm new-project stac-a2 --node-selector='workload=stac-a2'
    Created project stac-a2
  5. Once we have edited the BuildConfig parameters we can create the template on our OpenShift cluster:
    $ oc create -f stac-a2-intel-buildconfig.yaml
    template.template.openshift.io/stac-a2-intel-build-image created
  6. Now that we have our namespace with our template deployed, inside it we can also kickoff the BuildConfig:
    $ oc new-app stac-a2-intel-build-image
    --> Deploying template "stac-a2/stac-a2-intel-build-image" to project stac-a2
        stac-a2-intel-build-image
        ---------
        Template to build and run the STAC-A2 Intel Parallel Studio XE benchmark on OpenShift:
        * With parameters:
           * IMAGESTREAM_NAME=stac-a2-intel-ubi8
           * REGISTRY=image-registry.openshift-image-registry.svc:5000
    --> Creating resources ...
       buildconfig.build.openshift.io "stac-a2-intel-build-image" created
       imagestream.image.openshift.io "stac-a2-intel-ubi8" created
    --> Success
       Use 'oc start-build stac-a2-intel-build-image' to start a build.
       Run 'oc status' to view your app.
  7. As we can see from the output of the previous command, we can now start the build:
    $ oc start-build stac-a2-intel-build-image
    build.build.openshift.io/stac-a2-intel-build-image-1 started
  8. We can verify the build has successfully completed:
    $ oc get pods
    NAME                                READY   STATUS    RESTARTS   AGE
    stac-a2-intel-build-image-1-build   1/1     Completed     0          79m
  9. Now we can deploy the benchmark pod:
    $ oc create -f stac-a2-intel-pod.yaml
    template.template.openshift.io/stac-a2-intel-run created
    $ oc new-app stac-a2-intel-run
    --> Deploying template "stac-a2/stac-a2-intel-run" to project stac-a2
       stac-a2-intel-run
       ---------
       This template will run the STAC-A2 Intel Parallel Studio XE benchmark
       * With parameters:
          * IMAGESTREAM_NAME=stac-a2-intel-ubi8
          * REGISTRY=image-registry.openshift-image-registry.svc:5000
          * NUM_CPU=8
          * NAMESPACE=stac-a2
    --> Creating resources ...
      pod "stac-a2-intel" created
    --> Success
      Run 'oc status' to view your app.
  10. After the pod execution completes, we can grab the results from the benchmark run.
  11. Now we have a baseline for the run on the OpenShift node, now we can apply our custom node tunings using the Node Tuning Operator and repeat step 7 & 8.

Summary

It’s our belief that Linux containers and container orchestration engines, most notably Kubernetes, are positioned to power the future of enterprise applications across industries. In the area of performance-sensitive workloads, like financial risk analysis, we continue to reiterate our commitment to enabling the best workload performance possible regardless of which devices (CPU, accelerator add-in card, and others) are used. Working with Intel, we have once again demonstrated a successful pairing of high performance and low overhead with our full container orchestration solution.

We have created the artifacts discussed in this blog to show the ease of use and convenience of deploying performance-sensitive applications to OpenShift. These generic patterns can be reused for other compute-intensive applications, and with some small modifications, we can also enable other devices on the platform to fulfil the performance goals of different performance sensitive workloads.

While the STAC-A2 benchmark is important, and the results can be found in the links below, we believe this effort can have an impact beyond financial services. Artificial intelligence, machine learning, high-performance computing, and big data, to name a few areas, are all examples of workloads where the set of Performance-Sensitive Application Platform features are critical. Drawing from this experience with the Kubernetes community and STAC, we’re hoping to be able to drive OpenShift forward as the application platform for workloads on public, private, and hybrid cloud, offering a common development and deployment plane for all applications, regardless of industry, type, or performance-sensitive nature.

View the reports and precise configuration disclosure:

Bare Metal: https://STACresearch.com/INTC190402

OpenShift: https://STACresearch.com/INTC200410

 


About the author

Sebastian Jug, a Senior Performance and Scalability Engineer, has been working on OpenShift Performance at Red Hat since 2016. He is a software engineer and Red Hat Certified Engineer with experience in enabling Performance Sensitive Applications with devices such as GPUs and NICs. His focus is in automating, qualifying and tuning the performance of distributed systems. He has been a speaker at a number of industry conferences such as Kubecon and STAC Global.

Read full bio