ec2:CreateVpc
ec2:CreateSubnet
ec2:CreateRouteTable
ec2:CreateInternetGateway
ec2:CreateNatGateway
ec2:ModifyVpcAttribute
(for DNS settings)ec2:AssociateRouteTable
, ec2:CreateRoute
(for public and private route tables)ec2:AllocateAddress
(for Elastic IP assignment to the NAT Gateway)ec2:AuthorizeSecurityGroupIngress/Egress
(to configure cluster and node security groups to allow VPC CIDR traffic)AmazonEKSClusterPolicy
and AmazonEKSVPCResourceController
to a role with sts:AssumeRole
trust for eks.amazonaws.com
AmazonEKSWorkerNodePolicy
(for node operations)AmazonEKS_CNI_Policy
(for networking)AmazonEC2ContainerRegistryReadOnly
(for ECR access)iam:CreateOpenIDConnectProvider
(to associate the EKS cluster with IAM OIDC)iam:CreateRole
+ iam:AttachRolePolicy
(for service accounts in the recommender
, etl-operator
, and data-broker
namespaces)s3:CreateBucket
s3:PutBucketVersioning
s3:PutBucketEncryption
u10d-*-etl-blob-cache
u10d-*-etl-job-db
u10d-*-etl-job-status
u10d-*-job-files
rds:CreateDBInstance
rds:CreateDBSubnetGroup
rds:CreateDBSecurityGroup
+ ec2:AuthorizeSecurityGroupIngress
(to allow VPC CIDR access)eks:CreateAddon
with IAM role attachment permissions for the ebs.csi.aws.com
service accountec2:CreateKeyPair
+ ec2:ExportKeyPair
(for node group remote access)iam:PassRole
(to assign roles to EKS, RDS, and S3)kms:CreateKey
(if using CMK for S3 and RDS encryption)cloudformation:*
u10d-*-etl*
).
The EKS Pod Identity Agent requires eks-auth:AssumeRoleForPodIdentity
permission on node roles when used with IRSA.
10.0.0.0/16
- Any CIDR should work, but make sure it has enough space.10.0.0.0/24
${region}a
10.0.1.0/24
, 10.0.2.0/24
${region}a
and ${region}b
0.0.0.0/0
) via IGWeks.amazonaws.com
AmazonEKSClusterPolicy
AmazonEKSVPCResourceController
ec2.amazonaws.com
, eks.amazonaws.com
AmazonEKSWorkerNodePolicy
AmazonEKS_CNI_Policy
AmazonEC2ContainerRegistryReadOnly
recommender
, etl-operator
, data-broker
sts:AssumeRoleWithWebIdentity
with OIDC provider1.31
or greaterc5.4xlarge
(or larger, depending on cost factors)10.0.0.0/16
), self, and metadata IPaws.eks.Addon
:
v1.3.4-eksbuild.1
v0.7.2-eksbuild.1
v1.38.1-eksbuild.2
eks.amazonaws.com/role-arn
ebs-sc
ebs.csi.aws.com
type=gp3
, encrypted=true
WaitForFirstConsumer
db.t3.micro
10.0.0.0/16
(keep in mind your CIDR group from the VPC)postgres
u10d-{stack_name}-etl-blob-cache
u10d-{stack_name}-etl-job-db
u10d-{stack_name}-etl-job-status
u10d-{stack_name}-job-files
private_key
(PEM)BLOB_STORAGE_ADAPTER_ACCESS_KEY_ID
BLOB_STORAGE_ADAPTER_SECRET_ACCESS_KEY
BLOB_STORAGE_ADAPTER_REGION_NAME
DB_USERNAME
DB_PASSWORD
DB_HOST
DB_NAME
DB_DATABASE
(used in platform-api
only)JWT_SECRET_KEY
AUTH_STRATEGY
(sometimes encoded, sometimes not)SESSION_SECRET
SHARED_SECRET
KEYCLOAK_CLIENT_SECRET
KEYCLOAK_ADMIN_SECRET
KEYCLOAK_ADMIN
KEYCLOAK_ADMIN_PASSWORD
API_BEARER_TOKEN
BLOB_STORAGE_ADAPTER_TYPE
(always s3
for AWS)BLOB_STORAGE_ADAPTER_BUCKET
ETL_BLOB_CACHE_BUCKET_NAME
ETL_API_BLOB_STORAGE_ADAPTER_BUCKET
ETL_API_BLOB_STORAGE_ADAPTER_TYPE
ETL_API_DB_REMOTE_BUCKET_NAME
ETL_API_JOB_STATUS_DEST_BUCKET_NAME
JOB_STATUS_BUCKET_NAME
JOB_DB_BUCKET_NAME
ENV
ENVIRONMENT
JOB_ENV
JOB_ENVIRONMENT
JOB_OTEL_EXPORTER_OTLP_ENDPOINT
JOB_OTEL_METRICS_EXPORTER
JOB_OTEL_TRACES_EXPORTER
OTEL_EXPORTER_OTLP_ENDPOINT
OTEL_METRICS_EXPORTER
OTEL_TRACES_EXPORTER
UNSTRUCTURED_API_URL
JWKS_URL
JWT_ISSUER
JWT_AUDIENCE
SINGLE_PLANE_DEPLOYMENT
API_BASE_URL
API_CLIENT_BASE_URL
API_URL
APM_SERVICE_NAME
APM_SERVICE_NAME_CLIENT
AUTH_STRATEGY
FRONTEND_BASE_URL
KEYCLOAK_CALLBACK_URL
KEYCLOAK_CLIENT_ID
KEYCLOAK_DOMAIN
KEYCLOAK_REALM
KEYCLOAK_SSL_ENABLED
KEYCLOAK_TRUST_ISSUER
PUBLIC_BASE_URL
PUBLIC_RELEASE_CHANNEL
SENTRY_DSN
SENTRY_SAMPLE_RATE
WORKFLOW_NODE_EDITOR_FF_REQUEST_FORM
CUSTOM_WORKFLOW_FF_REQUEST_FORM
REDIS_DSN
IMAGE_PULL_SECRETS
PRIVATE_KEY_SECRETS_ADAPTER_TYPE
PRIVATE_KEY_SECRETS_ADAPTER_AWS_REGION
SECRETS_ADAPTER_TYPE
SECRETS_ADAPTER_AWS_REGION
File name | Type | Resource name | Namespace | Data keys |
---|---|---|---|---|
data-broker-env-cm.yaml | ConfigMap | data-broker-env | api | JOB_STATUS_BUCKET_NAME , JOB_DB_BUCKET_NAME , BLOB_STORAGE_ADAPTER_TYPE |
data-broker-env-secret.yaml | Secret | data-broker-env | api | BLOB_STORAGE_ADAPTER_ACCESS_KEY_ID , BLOB_STORAGE_ADAPTER_REGION_NAME , BLOB_STORAGE_ADAPTER_SECRET_ACCESS_KEY |
dataplane-api-env-cm.yaml | Secret | dataplane-api-env | api | DB_PASSWORD , DB_USERNAME , DB_HOST , DB_NAME |
etl-operator-env-cm.yaml | ConfigMap | etl-operator-env | etl-operator | BLOB_STORAGE_ADAPTER_BUCKET , JOB_STATUS_BUCKET_NAME , JOB_DB_BUCKET_NAME , BLOB_STORAGE_ADAPTER_TYPE , ENV , ENVIRONMENT , REDIS_DSN , ETL_API_BLOB_STORAGE_ADAPTER_BUCKET , ETL_API_BLOB_STORAGE_ADAPTER_TYPE , ETL_API_DB_REMOTE_BUCKET_NAME , ETL_API_JOB_STATUS_DEST_BUCKET_NAME (x2) , ETL_BLOB_CACHE_BUCKET_NAME , IMAGE_PULL_SECRETS , JOB_ENV , JOB_ENVIRONMENT , JOB_OTEL_EXPORTER_OTLP_ENDPOINT , JOB_OTEL_METRICS_EXPORTER , JOB_OTEL_TRACES_EXPORTER , OTEL_EXPORTER_OTLP_ENDPOINT , OTEL_METRICS_EXPORTER , OTEL_TRACES_EXPORTER , UNSTRUCTURED_API_URL |
etl-operator-env-secret.yaml | Secret | etl-operator-env | etl-operator | BLOB_STORAGE_ADAPTER_ACCESS_KEY_ID , BLOB_STORAGE_ADAPTER_REGION_NAME , BLOB_STORAGE_ADAPTER_SECRET_ACCESS_KEY |
frontend-env-cm.yaml | ConfigMap | frontend-env | www | API_BASE_URL , API_CLIENT_BASE_URL , API_URL , APM_SERVICE_NAME , APM_SERVICE_NAME_CLIENT , AUTH_STRATEGY , ENV , FRONTEND_BASE_URL , KEYCLOAK_CALLBACK_URL , KEYCLOAK_CLIENT_ID , KEYCLOAK_DOMAIN , KEYCLOAK_REALM , KEYCLOAK_SSL_ENABLED , KEYCLOAK_TRUST_ISSUER , PUBLIC_BASE_URL , PUBLIC_RELEASE_CHANNEL , SENTRY_DSN , SENTRY_SAMPLE_RATE , WORKFLOW_NODE_EDITOR_FF_REQUEST_FORM , CUSTOM_WORKFLOW_FF_REQUEST_FORM |
frontend-env-secret.yaml | Secret | frontend-env | www | API_BEARER_TOKEN , KEYCLOAK_ADMIN_SECRET , KEYCLOAK_CLIENT_SECRET , SESSION_SECRET , SHARED_SECRET |
keycloak-secret.yaml | Secret | phasetwo-keycloak-env | www | KEYCLOAK_ADMIN , KEYCLOAK_ADMIN_PASSWORD |
platform-api-env-cm.yaml | ConfigMap | platform-api-env | api | JWKS_URL , JWT_ISSUER , JWT_AUDIENCE , SINGLE_PLANE_DEPLOYMENT |
platform-api-env-secret.yaml | Secret | platform-api-env | api | DB_PASSWORD , DB_USERNAME , DB_HOST , DB_NAME , DB_DATABASE , JWT_SECRET_KEY , AUTH_STRATEGY |
recommender-env-cm.yaml | ConfigMap | recommender-env | recommender | BLOB_STORAGE_ADAPTER_TYPE , ETL_BLOB_CACHE_BUCKET_NAME |
recommender-env-secret.yaml | Secret | recommender-env | recommender | BLOB_STORAGE_ADAPTER_ACCESS_KEY_ID , BLOB_STORAGE_ADAPTER_REGION_NAME , BLOB_STORAGE_ADAPTER_SECRET_ACCESS_KEY |
secret-provider-api-env-cm.yaml | ConfigMap | secrets-provider-api-env | secrets | ENV , ENVIRONMENT , OTEL_EXPORTER_OTLP_ENDPOINT , OTEL_METRICS_EXPORTER , OTEL_TRACES_EXPORTER , PRIVATE_KEY_SECRETS_ADAPTER_AWS_REGION , PRIVATE_KEY_SECRETS_ADAPTER_TYPE , SECRETS_ADAPTER_AWS_REGION , SECRETS_ADAPTER_TYPE |
secret-provider-api-env-secret.yaml | Secret | secrets-provider-api-env | secrets | BLOB_STORAGE_ADAPTER_ACCESS_KEY_ID , BLOB_STORAGE_ADAPTER_REGION_NAME , BLOB_STORAGE_ADAPTER_SECRET_ACCESS_KEY |
usage-collector-env-secret.yaml | Secret | usage-collector-env | api | DB_PASSWORD , DB_USERNAME , DB_HOST , DB_NAME , BLOB_STORAGE_ADAPTER_TYPE |
etl-operator-env-cm.yaml
ConfigMap file, the contents would look like this:
etl-operator-env-secret.yaml
Secret file, the contents would look like this: