LAST UPDATED: September 29, 2020

Red Hat OpenShift Dedicated AWS Customer Cloud Subscription Requirements

• Overview
• Customer Requirements
• Account
• Access
• Support
• Security
• Customer Procedure
• Minimum Required Service Control Policy
• Red Hat Managed AWS Resources
  • IAM References
  • AWS Infrastructure

Overview

Red Hat OpenShift Dedicated (OSD) provides a Customer Cloud Subscription (CCS) model that allows Red Hat to deploy and manage OpenShift Dedicated into a customer's AWS account. Red Hat requires several prerequisites be met in order to provide this service. This document is intended to outline our prerequisites, procedural requirements, and also provide a guide to the AWS resources and Identity and Access Management (IAM) users, roles, policies, and identity providers that will be provisioned as part of this service. This service is supported by Red Hat Site Reliability Engineers (SRE).

Figure 1. Red Hat recommended AWS Organization and Account structure

Red Hat recommends the usage of an AWS Organization to manage multiple AWS accounts.

The AWS Organization managed by the customer hosts multiple AWS accounts. There is a root account in the organization that all accounts ultimately refer to in the account hierarchy.

Best practices for OSD CCS is for the cluster to be hosted in an AWS account within an AWS Organizational Unit. A SCP is created and applied to the Organizational Unit that manages what services the AWS sub-accounts are permitted to access. The SCP applies only to available permissions within a single AWS account for all AWS sub-accounts within the Organizational Unit. It is also possible to apply a SCP to a single AWS account. All other accounts in the customer’s AWS Organization are managed in whatever manner the customer requires. Red Hat SRE will not have any control over SCPs within the AWS Organization.

Customer Requirements

Account Requirements

• Customer ensures AWS limits are sufficient to support managed services provisioned within the customer-provided AWS account.
• Customer-provided AWS account should be in the customer's AWS Organization with the applicable Service Control Policy applied
  • Note: It is not a requirement that the customer-provided account be within an AWS Organization or for the Service Control Policy to be applied, however Red Hat must be able to perform all the actions listed in the Service Control Policy without restriction.
• Customer-provided AWS account should not be transferable to Red Hat.
• Customer may not impose AWS usage restrictions on Red Hat activities. Imposing restrictions will severely hinder Red Hat's ability to respond to incidents.
• Red Hat will deploy monitoring into AWS to alert Red Hat when a highly privileged AWS account, e.g., a root account, logs into the customer-provided AWS account.
• Customer may deploy native AWS services within the same customer-provided AWS account
  • Note: Customers are encouraged, but not mandated, to deploy resources in a Virtual Private Cloud (VPC) separate from the VPC hosting OpenShift Dedicated and other Red Hat supported services. 

Access Requirements

• Red Hat must have the AdministratorAccess policy applied to the "admin" role at all times in order to appropriately manage the OSD service.
• Note: This only provides Red Hat with permissions and capabilities to affect resources in the customer-provided AWS account.
• Red Hat must have AWS Console access to the customer-provided AWS account. This access is protected and managed by Red Hat.
• Customer must not utilize the AWS account to elevate their permissions within the OSD cluster.
• Actions available in the OpenShift Cluster Manager must not be directly performed in the customer-provided AWS account.

Support Requirements

• Red Hat recommends that the customer have at least Business Support from AWS.
• Red Hat has authority from customer to request AWS support on behalf of the customer.
• Red Hat has authority from customer to request AWS resource limit increases on the customer-provided account.
• Red Hat manages all OSD clusters in the same manner including the same restrictions, limitations, expectations, and defaults unless otherwise specified in this requirements section.

Security Requirements

• Customer-provided IAM credentials should be unique to the customer-provided AWS account and should not be stored anywhere in the customer-provided AWS account.
• Volume snapshots will remain within the customer-provided AWS account and customer-specified region.
• Red Hat must have ingress access to EC2 hosts and the API server via white-listed Red Hat machines.
• Red Hat must have egress allowed in order to forward system and audit logs to a Red Hat managed central logging stack.

Customer Procedure

Figure 2. AWS configuration procedure

1. If customer is utilizing AWS Organizations, you must use an AWS account within your organization or create a new one.
2. In order to ensure that Red Hat can perform the necessary actions, you must either create a Service Control Policy or ensure that none is applied to the AWS account.
3. Attach the Service Control Policy to the AWS account.
4. Within the AWS account, you must create an osdCcsAdmin IAM user:
• This user needs at least Programmatic access access type enabled.
• This user must have the AdministratorAccess policy attached to it.
5. Provide the IAM user credentials to Red Hat:
• You must provide the access key ID and secret access key in OpenShift Cluster Manager creation form.

Minimum Required Service Control Policy

Service Control Policy (SCP) management is the responsibility of the customer. These policies are maintained in the AWS Organization and control what services are available within the attached AWS accounts.

	Service	Actions	Effect
Required	Amazon EC2	All	Allow
	Amazon EC2 Auto Scaling	All	Allow
	Amazon S3	All	Allow
	Identity And Access Management	All	Allow
	Elastic Load Balancing	All	Allow
	Elastic Load Balancing V2	All	Allow
	Amazon CloudWatch	All	Allow
	Amazon CloudWatch Events	All	Allow
	Amazon CloudWatch Logs	All	Allow
	AWS Support	All	Allow
	AWS Key Management Service	All	Allow
	AWS Security Token Service	All	Allow
	AWS Resource Tagging	All	Allow
	AWS Route53 DNS	All	Allow
	AWS Service Quotas	ListServices GetRequestedServiceQuotaChange GetServiceQuota RequestServiceQuotaIncrease ListServiceQuotas	Allow
Optional	AWS Billing	ViewAccount ViewBilling ViewUsage	Allow
	AWS Cost and Usage Report	All	Allow
	AWS Cost Explorer Service	All	Allow

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "autoscaling:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "cloudwatch:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "events:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "logs:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "support:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kms:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "sts:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "tag:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "route53:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "servicequotas:ListServices",
                "servicequotas:GetRequestedServiceQuotaChange",
                "servicequotas:GetServiceQuota",
                "servicequotas:RequestServiceQuotaIncrease",
                "servicequotas:ListServiceQuotas"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Red Hat Managed AWS Resources

The below sections define AWS resources that Red Hat is responsible for creating.

IAM References

IAM Policies

These policies are subject to modification as the capabilities of OpenShift Dedicated change.

AdministratorAccess

The AdministratorAccess policy is used by the "admin" role. It provides Red Hat the access necessary to administer the OSD cluster in the customer-provided AWS account.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "*",
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

CustomerAdministratorAccess

The CustomerAdministatorAccess role provides the customer access to administer a subset of services within the AWS account. At this time, the following are allowed:

• VPC Peering
• VPN Setup
• Direct Connect (only available if granted via SCP policy)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:AttachVpnGateway",
                "ec2:DescribeVpnConnections",
                "ec2:AcceptVpcPeeringConnection",
                "ec2:DeleteVpcPeeringConnection",
                "ec2:DescribeVpcPeeringConnections",
                "ec2:CreateVpnConnectionRoute",
                "ec2:RejectVpcPeeringConnection",
                "ec2:DetachVpnGateway",
                "ec2:DeleteVpnConnectionRoute",
                "ec2:DeleteVpnGateway",
                "ec2:DescribeVpcs",
                "ec2:CreateVpnGateway",
                "ec2:ModifyVpcPeeringConnectionOptions",
                "ec2:DeleteVpnConnection",
                "ec2:CreateVpcPeeringConnection",
                "ec2:DescribeVpnGateways",
                "ec2:CreateVpnConnection",
                "ec2:DescribeRouteTables",
                "ec2:CreateTags",
                "ec2:CreateRoute",
          "directconnect:*"
            ],
            "Resource": "*"
        }
    ]
}

BillingReadOnlyAccess

The BillingReadOnlyAccess role provides read-only access to view billing and usage information for the account if it is enabled.

Billing and usage access is only granted if the root account in the AWS Organization has it enabled. This is an optional step the customer must perform to enable read-only billing and usage access and does not impact the creation of this profile and the role that uses it. If not enabled, the impact is that users will not see billing and usage information. See this tutorial on how to enable access to billing data.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "aws-portal:ViewAccount",
                "aws-portal:ViewBilling"
            ],
            "Resource": "*"
        }
    ]
}

IAM Users

osdManagedAdmin

This user is created immediately after taking control of the customer-provided AWS account. This is the user that will be performing the OSD cluster install.

IAM Roles

network-mgmt

This role provides customer-federated administrative access to the AWS account via a separate AWS account. It also has the same access as read-only role.

Detail

Type: AWS account
Policies:

• • AmazonEC2ReadOnlyAccess
  • CustomerAdministratorAccess

read-only

This role provides customer-federated read-only access to the AWS account via a separate AWS account.

Detail

Type: AWS account
Policies:

• • AWSAccountUsageReportAccess
  • AmazonEC2ReadOnlyAccess
  • AmazonS3ReadOnlyAccess
  • IAMReadOnlyAccess
  • BillingReadOnlyAccess

Provisioned AWS Infrastructure

This is an overview of the OpenShift Dedicated specific deployments. For a more detailed listing of all AWS components provisioned, you can refer to the OpenShift Container Platform documentation.

EC2 Instances

AWS EC2 instances are required for deploying control plane and data plane functions of OpenShift Dedicated in the AWS public cloud.

Detail - Single AZ

• • 3 m5.xlarge minimum (Masters Nodes) - instance type may vary depending on compute node count
  • 2 r5.xlarge minimum (Infrastructure Nodes)
  • 2 m5.xlarge minimum but highly variable (Compute Nodes)

Detail - Multi AZ

• • 3 m5.xlarge minimum (Masters Nodes) - instance type may vary depending on compute node count
  • 3 r5.xlarge minimum (Infrastructure Nodes)
  • 3 m5.xlarge minimum but highly variable (Compute Nodes)

EBS Storage

Amazon EBS block storage is used for both local node storage and persistent volume storage.

Detail

Volume requirements for each EC2 instance:

• • Master Volume
    • • size: 350GB
      • type: io1
      • iops: 1000
  • Infrastructure Volume
    • • size: 300GB
      • type: gp2
      • Iops: 100
  • Compute Volume
    • • size: 300GB
      • type: gp2
      • Iops: 100

Elastic Load Balancers

Up to two Network Elastic Load Balancers (ELBs) for API and up to two Classic ELBs for application router. See here for more details on ELBs.

S3 storage

The image registry and EBS volume snapshots are backed by AWS S3 storage. Pruning of resources is performed regularly to optimize S3 usage and cluster performance.

Detail

Buckets required: 2
Typical size: 2TB each

VPC

Customers should expect to see one VPC per cluster. Additionally, the VPC will need following configurations:

• Subnets: 2 subnets if cluster with single availability zone is preferred, 6 subnets if multi-az is preferred
• Router Tables: 1 per private subnet, and 1 additional table per cluster
• Internet Gateways: 1 per cluster
• NAT Gateways: 1 per public subnet

Security Groups

AWS security groups are associated with EC2 instances and Elastic Load Balancers and provide security at the protocol and port access level. Each security group, working much the same way as a firewall, contains a set of rules that filter traffic coming into and out of an EC2 instance. You must ensure the ports required for OpenShift installation listed here are open on your network and configured to allow access between hosts.