Terraform · MySQL · WordPress · Secrets Manager · HPA
Provisioned a production-grade EKS cluster with Terraform, deployed a two-tier WordPress and MySQL application via Helm, managed all database credentials through AWS Secrets Manager with the External Secrets Operator syncing them into the cluster automatically, and verified horizontal autoscaling under real load with Siege.
Key design decision: Terraform owns the bottom half of the stack — VPC, EKS cluster, IAM roles, OIDC provider, and the Secrets Manager secret. Helm owns the top half — MySQL, WordPress, services, and HPA. AWS Secrets Manager sits in the middle: Terraform creates it, the External Secrets Operator pulls credentials into a Kubernetes Secret automatically on a 1-hour refresh interval. Rotating a password in Secrets Manager updates the cluster without touching a running deployment.
IRSA pattern used twice: Both the External Secrets Operator and the EBS CSI driver use IAM Roles for Service Accounts — pod-level IAM permissions via OIDC federation, no static credentials anywhere in the cluster.
Four Terraform files provisioning the full infrastructure layer: custom VPC with two public subnets across two AZs, EKS cluster with managed node group, all four cluster addons including the EBS CSI driver, IAM roles for the ESO and EBS CSI driver via OIDC, and an AWS Secrets Manager secret holding all database credentials. No manual steps — a single terraform apply builds everything.
AWS Secrets Manager holds all four database credential fields. The External Secrets Operator is installed via Helm and configured with a SecretStore pointing at Secrets Manager through an IRSA-backed IAM role. An ExternalSecret object maps the credential keys into a Kubernetes Secret that MySQL and WordPress both consume via environment variables. Credentials rotate in Secrets Manager — the cluster follows automatically.
MySQL 8.0 deployed as a single pod with a 10Gi EBS-backed PersistentVolumeClaim — data survives pod restarts and rescheduling. All four credentials (root password, DB name, username, password) are injected from the Kubernetes Secret via environment variables. Exposed internally via a ClusterIP service with clusterIP: None so WordPress reaches it by service name. Nothing external can reach MySQL directly.
WordPress deployed via a custom Helm chart with a patched deployment template to pass the env block from values.yaml to the container. Database connection details read from the Kubernetes Secret at startup. Exposed to the internet via a LoadBalancer service that provisions an AWS ELB automatically. Resource limits set to 768Mi memory and 500m CPU to handle concurrent traffic without OOMKills.
HPA configured against the WordPress deployment with a 50% CPU utilization target and a ceiling of 5 replicas. Metrics Server installed to feed real CPU data to the HPA controller. Under Siege load, CPU spiked to 127% on the initial pod — the HPA scaled from 1 to 3 replicas within one minute. New pods went from Pending to Running in under two seconds. Scale-down completed within 5 minutes after load stopped.
EKS 1.31 requires the EBS CSI driver addon for dynamic EBS volume provisioning — the in-tree provisioner is removed. The driver runs with its own IRSA-backed IAM role bound to the ebs-csi-controller-sa service account, granting the EC2 and EBS permissions the controller needs. Managed as an EKS addon in Terraform so it provisions automatically with the cluster and tears down cleanly with terraform destroy.
CREATE_FAILED with Ec2SubnetInvalidConfiguration. Nodes couldn't join the cluster because the VPC subnets weren't configured to auto-assign public IPs — instances launched into those subnets had no public IP and couldn't reach the EKS control plane.
map_public_ip_on_launch = true to the VPC module block in main.tf. The terraform-aws-modules VPC module doesn't enable this by default — it has to be set explicitly. Ran terraform destroy and re-applied clean.
kubectl apply on the SecretStore manifest failed with no matches for kind SecretStore in version external-secrets.io/v1beta1. The installed version of ESO had promoted the CRDs from v1beta1 to v1.
kubectl api-resources | grep external-secrets to confirm the correct API version, then updated apiVersion in both secretstore.yaml and externalsecret.yaml from external-secrets.io/v1beta1 to external-secrets.io/v1.
Pending for 7+ minutes. No storageClassName set on the PVC — Kubernetes had no provisioner to request a volume from. Combined with the cluster's WaitForFirstConsumer binding mode, the PVC and pod were deadlocked.
storageClassName: gp2 to the PVC spec. Then discovered the EBS CSI driver wasn't installed — kubectl describe pvc showed Kubernetes waiting on ebs.csi.aws.com. Installed the aws-ebs-csi-driver addon. EKS 1.21+ requires the CSI driver — the in-tree provisioner is gone.
CrashLoopBackOff. Logs showed UnauthorizedOperation on ec2:DescribeAvailabilityZones. The controller was running under the node IAM role which had no EC2 API permissions. Addon stuck in CREATING — couldn't be updated in that state.
main.tf with the AmazonEBSCSIDriverPolicy managed policy, bound to the ebs-csi-controller-sa service account in kube-system. Applied with Terraform, then deleted and recreated the addon with --service-account-role-arn set from the start. Now managed as a Terraform addon block so it provisions correctly on every apply.
helm install failed twice in succession — first with nil pointer on serviceAccount.create, then on httpRoute.enabled. Both values were stripped when the entire values.yaml was replaced with the WordPress configuration.
serviceAccount: create: false and httpRoute: enabled: false back to values.yaml. Newer Helm scaffold versions include an httproute template that older documentation doesn't account for — every template in the chart needs its required values present even if the feature is disabled.
Access Denied for both root and wordpress_user with the correct passwords. WordPress showed "Error establishing a database connection." MySQL had initialized its data directory during a previous failed deployment with different credentials — once written to the PVC, those credentials are locked in.
OOMKilled under Siege load before CPU could spike high enough to trigger the HPA. The 512Mi memory limit was too tight for WordPress handling 100 concurrent users with active database queries. Pod died before autoscaling could respond.
values.yaml, then ran helm upgrade to apply without tearing down the deployment. Second Siege run completed successfully — CPU hit 127%, HPA scaled to 3 replicas, availability reached 94.4%.
Siege ran 100 concurrent users for 5 minutes against the WordPress ELB endpoint with WordPress fully connected to MySQL — each request triggered real database queries, not a cached static response.
| Successful Transactions | 2,095 |
| Availability | 94.41% |
| Failed Transactions | 124 |
| Peak CPU Usage | 127% — well above the 50% HPA threshold |
| Replicas at Peak | 3 (scaled from 1) |
| Time to First Scale Event | ~1 minute after Siege started |
| Pod Startup Time | Pending → Running in under 2 seconds |
| Scale-Down After Load | Returned to 1 replica within 5 minutes |
| Data Transferred | 13.65 MB |
| Peak Concurrency | 93.13 of 100 users active |