cat projects/platform
- started
- 2024-05-01
- stack
- Terraform, AWS CDK, Jenkins, GitHub Actions, Python
- scale
- Every environment defined in code — no console-assembled infrastructure left
- outcome
- 40% less manual effort per deployment
- result
- in production
The problem. Infrastructure that exists only as a sequence of clicks cannot be reviewed, diffed or rebuilt. Onboarding a new environment was an exercise in remembering.
The architecture. Terraform for the long-lived account-level infrastructure,
AWS CDK where the resources are tightly coupled to application code, and a shared
library of reusable modules so that a bucket with the right encryption and
lifecycle rules is the default thing you get rather than something you have to
remember to configure. Jenkins and GitHub Actions run plan on every pull request
and apply on merge. What is deliberately not in here is the data catalog: table
definitions are their own body of Terraform, reviewed against what the
transforms actually write rather than against an account layout, and they live
in cat projects/catalog. Manual effort per deployment fell by about 40%, but the
number that mattered more was the one I cannot put a percentage on: an
environment change became something two people had looked at.
The interesting decision. Running Terraform and CDK side by side rather than standardising on one. Two IaC tools is a genuine cost — two mental models, two state stories, two ways to be wrong. I kept both because they are good at different things: Terraform for infrastructure with a long life and a slow change rate, CDK where the infrastructure and the code that uses it change in the same pull request. Forcing everything into one would have meant either writing application-shaped resources in HCL or managing account foundations from a language runtime, and both of those are worse than owning the seam.
The pipelines were code and the infrastructure under them was not. Environments were assembled by hand, so they drifted, so "works in dev" stopped carrying information. The only real documentation of how an environment got built was whoever had built one last.
Moved account-level infrastructure into Terraform and application-coupled resources into AWS CDK, behind a shared library of modules so that a correctly configured resource is the default you get rather than something you have to remember. Plan runs on every pull request, apply on merge — roughly 40% less manual effort per deployment, and every change seen by a second person.