POLICIES

Requirements for Customer Cloud Subscriptions

LAST UPDATED: September 29, 2020

Red Hat OpenShift Dedicated AWS Customer Cloud Subscription Requirements

Overview

Red Hat OpenShift Dedicated (OSD) provides a Customer Cloud Subscription (CCS) model that allows Red Hat to deploy and manage OpenShift Dedicated into a customer's AWS account. Red Hat requires several prerequisites be met in order to provide this service. This document is intended to outline our prerequisites, procedural requirements, and also provide a guide to the AWS resources and Identity and Access Management (IAM) users, roles, policies, and identity providers that will be provisioned as part of this service. This service is supported by Red Hat Site Reliability Engineers (SRE).

image1
Figure 1. Red Hat recommended AWS Organization and Account structure

Red Hat recommends the usage of an AWS Organization to manage multiple AWS accounts.

The AWS Organization managed by the customer hosts multiple AWS accounts. There is a root account in the organization that all accounts ultimately refer to in the account hierarchy.

Best practices for OSD CCS is for the cluster to be hosted in an AWS account within an AWS Organizational Unit. A SCP is created and applied to the Organizational Unit that manages what services the AWS sub-accounts are permitted to access. The SCP applies only to available permissions within a single AWS account for all AWS sub-accounts within the Organizational Unit. It is also possible to apply a SCP to a single AWS account. All other accounts in the customer’s AWS Organization are managed in whatever manner the customer requires. Red Hat SRE will not have any control over SCPs within the AWS Organization.

Customer Requirements

Account Requirements


  • Customer ensures AWS limits are sufficient to support managed services provisioned within the customer-provided AWS account.
  • Customer-provided AWS account should be in the customer's AWS Organization with the applicable Service Control Policy applied
    • Note: It is not a requirement that the customer-provided account be within an AWS Organization or for the Service Control Policy to be applied, however Red Hat must be able to perform all the actions listed in the Service Control Policy without restriction.
  • Customer-provided AWS account should not be transferable to Red Hat.
  • Customer may not impose AWS usage restrictions on Red Hat activities. Imposing restrictions will severely hinder Red Hat's ability to respond to incidents.
  • Red Hat will deploy monitoring into AWS to alert Red Hat when a highly privileged AWS account, e.g., a root account, logs into the customer-provided AWS account.
  • Customer may deploy native AWS services within the same customer-provided AWS account
    • Note: Customers are encouraged, but not mandated, to deploy resources in a Virtual Private Cloud (VPC) separate from the VPC hosting OpenShift Dedicated and other Red Hat supported services. 

Access Requirements

  • Red Hat must have the AdministratorAccess policy applied to the "admin" role at all times in order to appropriately manage the OSD service.
    • Note: This only provides Red Hat with permissions and capabilities to affect resources in the customer-provided AWS account.
  • Red Hat must have AWS Console access to the customer-provided AWS account. This access is protected and managed by Red Hat.
  • Customer must not utilize the AWS account to elevate their permissions within the OSD cluster.
  • Actions available in the OpenShift Cluster Manager must not be directly performed in the customer-provided AWS account.

Support Requirements

  • Red Hat recommends that the customer have at least Business Support from AWS.
  • Red Hat has authority from customer to request AWS support on behalf of the customer.
  • Red Hat has authority from customer to request AWS resource limit increases on the customer-provided account.
  • Red Hat manages all OSD clusters in the same manner including the same restrictions, limitations, expectations, and defaults unless otherwise specified in this requirements section.

Security Requirements

  • Customer-provided IAM credentials should be unique to the customer-provided AWS account and should not be stored anywhere in the customer-provided AWS account.
  • Volume snapshots will remain within the customer-provided AWS account and customer-specified region.
  • Red Hat must have ingress access to EC2 hosts and the API server via white-listed Red Hat machines.
  • Red Hat must have egress allowed in order to forward system and audit logs to a Red Hat managed central logging stack.

Customer Procedure

image2
Figure 2. AWS configuration procedure

  1. If customer is utilizing AWS Organizations, you must use an AWS account within your organization or create a new one.
  2. In order to ensure that Red Hat can perform the necessary actions, you must either create a Service Control Policy or ensure that none is applied to the AWS account.
  3. Attach the Service Control Policy to the AWS account.
  4. Within the AWS account, you must create an osdCcsAdmin IAM user:
    • This user needs at least Programmatic access access type enabled.
    • This user must have the AdministratorAccess policy attached to it.
  5. Provide the IAM user credentials to Red Hat:

Minimum Required Service Control Policy

Service Control Policy (SCP) management is the responsibility of the customer. These policies are maintained in the AWS Organization and control what services are available within the attached AWS accounts.

  Service Actions Effect
Required Amazon EC2 All Allow
Amazon EC2 Auto Scaling All Allow
Amazon S3 All Allow
Identity And Access Management All Allow
Elastic Load Balancing All Allow
Elastic Load Balancing V2 All Allow
Amazon CloudWatch All Allow
Amazon CloudWatch Events All Allow
Amazon CloudWatch Logs All Allow
AWS Support All Allow
AWS Key Management Service All Allow
AWS Security Token Service All Allow
AWS Resource Tagging All Allow
AWS Route53 DNS All Allow
AWS Service Quotas ListServices
GetRequestedServiceQuotaChange
GetServiceQuota
RequestServiceQuotaIncrease
ListServiceQuotas
Allow
Optional AWS Billing ViewAccount
ViewBilling
ViewUsage
Allow
AWS Cost and Usage Report All Allow
AWS Cost Explorer Service All Allow

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "autoscaling:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "cloudwatch:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "events:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "logs:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "support:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kms:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "sts:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "tag:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "route53:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "servicequotas:ListServices",
                "servicequotas:GetRequestedServiceQuotaChange",
                "servicequotas:GetServiceQuota",
                "servicequotas:RequestServiceQuotaIncrease",
                "servicequotas:ListServiceQuotas"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Red Hat Managed AWS Resources

The below sections define AWS resources that Red Hat is responsible for creating.

IAM References

IAM Policies

These policies are subject to modification as the capabilities of OpenShift Dedicated change.

AdministratorAccess

The AdministratorAccess policy is used by the "admin" role. It provides Red Hat the access necessary to administer the OSD cluster in the customer-provided AWS account.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "*",
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

CustomerAdministratorAccess

The CustomerAdministatorAccess role provides the customer access to administer a subset of services within the AWS account. At this time, the following are allowed:

  • VPC Peering
  • VPN Setup
  • Direct Connect (only available if granted via SCP policy)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:AttachVpnGateway",
                "ec2:DescribeVpnConnections",
                "ec2:AcceptVpcPeeringConnection",
                "ec2:DeleteVpcPeeringConnection",
                "ec2:DescribeVpcPeeringConnections",
                "ec2:CreateVpnConnectionRoute",
                "ec2:RejectVpcPeeringConnection",
                "ec2:DetachVpnGateway",
                "ec2:DeleteVpnConnectionRoute",
                "ec2:DeleteVpnGateway",
                "ec2:DescribeVpcs",
                "ec2:CreateVpnGateway",
                "ec2:ModifyVpcPeeringConnectionOptions",
                "ec2:DeleteVpnConnection",
                "ec2:CreateVpcPeeringConnection",
                "ec2:DescribeVpnGateways",
                "ec2:CreateVpnConnection",
                "ec2:DescribeRouteTables",
                "ec2:CreateTags",
                "ec2:CreateRoute",
          "directconnect:*"
            ],
            "Resource": "*"
        }
    ]
}

BillingReadOnlyAccess

The BillingReadOnlyAccess role provides read-only access to view billing and usage information for the account if it is enabled.

Billing and usage access is only granted if the root account in the AWS Organization has it enabled. This is an optional step the customer must perform to enable read-only billing and usage access and does not impact the creation of this profile and the role that uses it. If not enabled, the impact is that users will not see billing and usage information. See this tutorial on how to enable access to billing data.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "aws-portal:ViewAccount",
                "aws-portal:ViewBilling"
            ],
            "Resource": "*"
        }
    ]
}

IAM Users

osdManagedAdmin

This user is created immediately after taking control of the customer-provided AWS account. This is the user that will be performing the OSD cluster install.

IAM Roles

network-mgmt

This role provides customer-federated administrative access to the AWS account via a separate AWS account. It also has the same access as read-only role.

Detail

Type: AWS account
Policies:

    • AmazonEC2ReadOnlyAccess
    • CustomerAdministratorAccess

read-only

This role provides customer-federated read-only access to the AWS account via a separate AWS account.

Detail

Type: AWS account
Policies:

    • AWSAccountUsageReportAccess
    • AmazonEC2ReadOnlyAccess
    • AmazonS3ReadOnlyAccess
    • IAMReadOnlyAccess
    • BillingReadOnlyAccess

Provisioned AWS Infrastructure

This is an overview of the OpenShift Dedicated specific deployments. For a more detailed listing of all AWS components provisioned, you can refer to the OpenShift Container Platform documentation.

EC2 Instances

AWS EC2 instances are required for deploying control plane and data plane functions of OpenShift Dedicated in the AWS public cloud.

Detail - Single AZ
    • 3 m5.xlarge minimum (Masters Nodes) - instance type may vary depending on compute node count
    • 2 m5.xlarge minimum (Infrastructure Nodes)
    • 2 m5.xlarge minimum but highly variable (Compute Nodes)
Detail - Multi AZ
    • 3 m5.xlarge minimum (Masters Nodes) - instance type may vary depending on compute node count
    • 3 m5.xlarge minimum (Infrastructure Nodes)
    • 3 m5.xlarge minimum but highly variable (Compute Nodes)

EBS Storage

Amazon EBS block storage is used for both local node storage and persistent volume storage.

Detail

Volume requirements for each EC2 instance:

    • Master Volume
        • size: 350GB
        • type: io1
        • iops: 1000
    • Infrastructure Volume
        • size: 300GB
        • type: gp2
        • Iops: 100
    • Compute Volume
        • size: 300GB
        • type: gp2
        • Iops: 100

Elastic Load Balancers

Up to two Network Elastic Load Balancers (ELBs) for API and up to two Classic ELBs for application router. See here for more details on ELBs.

S3 storage

The image registry and EBS volume snapshots are backed by AWS S3 storage. Pruning of resources is performed regularly to optimize S3 usage and cluster performance.

Detail

Buckets required: 2
Typical size: 2TB each

VPC

Customers should expect to see one VPC per cluster. Additionally, the VPC will need following configurations:

  • Subnets: 2 subnets if cluster with single availability zone is preferred, 6 subnets if multi-az is preferred
  • Router Tables: 1 per private subnet, and 1 additional table per cluster
  • Internet Gateways: 1 per cluster
  • NAT Gateways: 1 per public subnet

Security Groups

AWS security groups are associated with EC2 instances and Elastic Load Balancers and provide security at the protocol and port access level. Each security group, working much the same way as a firewall, contains a set of rules that filter traffic coming into and out of an EC2 instance. You must ensure the ports required for OpenShift installation listed here are open on your network and configured to allow access between hosts.

LAST UPDATED: December 2, 2020

Red Hat OpenShift Dedicated Google Cloud Customer Cloud Subscription Requirements

Overview

Red Hat OpenShift Dedicated (OSD) provides a Customer Cloud Subscription (CCS) model that allows Red Hat to deploy and manage OpenShift Dedicated into a customer's Google Cloud account. Red Hat requires several prerequisites be met in order to provide this service. This document is intended to outline our prerequisites, procedural requirements, and also provide a guide to the Google Cloud resources and Identity and Access Management (IAM) users, roles, policies, and identity providers that will be provisioned as part of this service. This service is supported by Red Hat Site Reliability Engineers (SRE).

Red Hat recommends the usage of a Google Cloud (GCP) project, managed by the customer, to organize all of your GCP resources. A project consists of a set of users and APIs, as well as billing, authentication, and monitoring settings for those APIs.

It is a best practice for the OpenShift Dedicated CCS cluster to be hosted in a GCP project within a GCP organization. The Organization resource is the root node of the GCP resource hierarchy and all resources that belong to an organization are grouped under the organization node. An IAM service account with certain roles granted is created and applied to the GCP project. When you make calls to the API, you typically provide service account keys for authentication. Each service account is owned by a specific project, but service accounts can be provided roles to access resources for other projects.

Customer Requirements

Account Requirements

  • Customer ensures Google Cloud limits are sufficient to support managed services provisioned within the customer-provided cloud account.
  • The customer-provided Google Cloud account should be in the customer’s Google Cloud Organization with the applicable Service Account applied.
  • Customer-provided Google Cloud account should not be transferable to Red Hat.
  • Customer may not impose Google Cloud usage restrictions on Red Hat activities. Imposing restrictions will severely hinder Red Hat's ability to respond to incidents.
  • Red Hat will deploy monitoring into Google Cloud to alert Red Hat when a highly privileged account, e.g., a root account, logs into the customer-provided cloud account.
  • Customer may deploy native Google Cloud services within the same customer-provided cloud account.
    • Note: Customers are encouraged, but not mandated, to deploy resources in a Virtual Private Cloud (VPC) separate from the VPC hosting OpenShift Dedicated and other Red Hat supported services.

Access Requirements

  • Red Hat must have the AdministratorAccess policy applied to the "admin" role at all times in order to appropriately manage the OSD service.
    • Note: This only provides Red Hat with permissions and capabilities to affect resources in the customer-provided Google Cloud account.
  • Red Hat must have Google Cloud Console access to the customer-provided cloud account. This access is protected and managed by Red Hat.
  • Customer must not utilize the Google Cloud account to elevate their permissions within the OSD cluster.
  • Actions available in the OpenShift Cluster Manager must not be directly performed in the customer-provided Google Cloud account.

Support Requirements

  • Red Hat recommends that the customer have at least Production Support from Google Cloud.
  • Red Hat has authority from customer to request Google Cloud support on behalf of the customer.
  • Red Hat has authority from customer to request Google Cloud resource limit increases on the customer-provided account.
  • Red Hat manages all OSD clusters in the same manner including the same restrictions, limitations, expectations, and defaults unless otherwise specified in this requirements section.

Security Requirements

  • Customer-provided IAM credentials should be unique to the customer-provided Google Cloud account and should not be stored anywhere in the customer-provided cloud account.
  • Volume snapshots will remain within the customer-provided Google Cloud account and customer-specified region.
  • Red Hat must have ingress access the API server via white-listed Red Hat machines.
  • Red Hat must have egress allowed in order to forward system and audit logs to a Red Hat managed central logging stack.

Customer Procedure

The CCS model allows Red Hat to deploy and manage OpenShift managed services into a customer’s Google Cloud (GCP) project. Red Hat requires several prerequisites in order to provide these services.

  • WARNING: To use OpenShift Dedicated in your Google Cloud project, the Google Cloud organizational policy constraint, constraints/iam.allowedPolicyMemberDomains, cannot be in place.
  1. Create a Google Cloud project to host the OpenShift Dedicated cluster.
    • Note: The project name must be 10 characters or less.
  2. Enable the following required APIs in the project that hosts your OpenShift Dedicated cluster:
    API service Console service name
    Cloud Deployment Manager V2 API deploymentmanager.googleapis.com
    Compute Engine API compute.googleapis.com
    Google Cloud APIs cloudapis.googleapis.com
    Cloud Resource Manager API cloudresourcemanager.googleapis.com
    Google DNS API dns.googleapis.com
    IAM Service Account Credentials API iamcredentials.googleapis.com
    Identity and Access Management (IAM) API iam.googleapis.com
    Service Management API servicemanagement.googleapis.com
    Service Usage API serviceusage.googleapis.com
    Google Cloud Storage JSON API storage-api.googleapis.com
    Cloud Storage storage-component.googleapis.com
  3. To ensure that Red Hat can perform necessary actions, you must create an osd-ccs-admin IAM service account user within the Google Cloud project.
  4. The following roles must be granted to the service account:
    Role Console role name
    DNS Admin roles/dns.admin
    Organizational Policy Viewer roles/orgpolicy.policyViewer
    Owner roles/owner
    Project IAM Admin roles/resourcemanager.projectIamAdmin
    Service Management Administrator roles/servicemanagement.admin
    Service Usage Admin roles/serviceusage.serviceUsageAdmin
    Storage Admin roles/storage.admin
  5. Create the service account key for the osd-ccs-admin IAM service account. Export the key to a file named osServiceAccount.json, this JSON file will be uploaded in OpenShift Cluster Manager (OCM) when you create your cluster.

Red Hat Managed Google Cloud Resources

Red Hat is responsible for creating the following Google Cloud resources.

IAM References

IAM service account and roles

The osd-managed-admin IAM service account is created immediately after taking control of the customer-provided GCP account. This is the user that will be performing the OpenShift Dedicated cluster install.

The following roles are attached to the service account:

  • Compute Admin
  • DNS Administrator
  • Security Admin
  • Storage Admin
  • Service Account Admin
  • Service Account Key Admin
  • Service Account User

IAM group and roles

The sd-sre-platform-gcp-access Google group is created to allow Red Hat Site Reliability Engineering (SRE) access to the console for emergency troubleshooting purposes.

The following roles are attached to the group:

  • Compute Admin
  • Editor
  • Organization Policy Viewer
  • Project IAM Admin
  • Quota Administrator
  • Role Administrator
  • Service Account Admin
  • Service Usage Admin
  • Tech Support Editor