This post will aim to describe one approach to configuring Terraform for use across multiple distinct AWS accounts.
The goal is to configure the Terraform aws provider and s3 backend with a single set of AWS credentials and parameters, while targeting deployment environments via Terraform workspaces. The workspaces will correspond to different AWS accounts.
SPOILER: The magic is in the Terraform Assume Role aws provider option and AWS IAM role delegation.
Assumptions
The below is written with the assumption of familiarity with Terraform and Amazon Web Services, particularly Identity and Access Management (IAM).
First - Single Account
For the last couple years I've been using Terraform to declare and provision resources within AWS. Up to this point, the day-to-day needs at my current employer have been contained within a single AWS account, utilizing separate VPCs to correspond to deployment environments such as DEV, QA and PROD. In conjunction with Terraform workspaces, this has been a fairly straight-forward configuration when using the aws provider...satisfy one of the options in the authentication chain and you are good to go.
Multi-AZ. No, I mean Multi-ACCT?
Multi-Accounts on AWS is a common architectural pattern for organizations for a variety of reasons, and is the setup we are aiming for going forward.
For this reason we have set up the following AWS accounts:
- Administrative
- Development
- Quality Assurance (QA)
- Production
Administrative
The purpose of this account is to serve as a location for shared services and common resources, such as CodeCommit repos, ECR repos, KMS keys, ect. In addition, access to the other 3 accounts will be provided by an IAM User within this account. Applications will not be deployed here.
In the context of Terraform, this account will contain the following resources:
- An S3 bucket used to contain Terraform state from all 3 deployment accounts.
- A DynamoDB table used to contain Terraform state locks.
- An IAM User with programmatic access, used to allow Terraform to authenticate.
- IAM Policies attached to the IAM User. These will allow access to the s3 bucket and dynamo table, as well as allow assumption of IAM roles within each of the deployment accounts.
Development, QA, Production
These three accounts will contain deployed applications and associated resources.
In order to allow provisioning permissions in these accounts via TF, each account will need:
- An IAM Role with a trust relationship to the Administrative AWS account.
- An IAM Policy providing the permissions required to provision resources. This could end up being a fairly long list of policy actions, depending on what services and components are being provisioned.
Making it Happen
Starting top-down...
Terraform
Example configuration:
locals {
role_arns = {
"dev" = ""
"qa" = ""
"prod" = ""
}
env = terraform.workspace
role_arn = local.role_arns[local.env]
}
provider "aws" {
version = "~> 2.54"
region = "us-east-1"
assume_role {
role_arn = local.role_arn
session_name = "terraform"
}
}
terraform {
required_version = ">= 0.12"
backend "s3" {
key = "my-app/terraform.tfstate"
bucket = ""
dynamodb_table = ""
region = "us-east-1"
}
}
- For this config, we will have 3 workspaces -
dev,qaandprod. - The assume_role provider block tells Terraform to attempt to assume the given role with the current credentials. Using the current workspace, we target the role for the corresponding account.
- We add placeholders for the s3 bucket, Dynamo table and 3 deployment account roles that don't exist yet.
Deployment Accounts
In each of the deployment accounts, create the following:
IAM Role
Create an IAM Role which will be used to provision resources. Set the Trust Relationship to be equal to the AWS account ID of the Administrative Account. If using the console, this can be done by selecting "Another AWS Account" from under the "Select type of trusted entity" heading.
Example trust relationship policy document:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<account_id>:root"
},
"Action": "sts:AssumeRole",
"Condition": {}
}
]
}
Once the role has been created, copy the Role ARN and paste into the corresponding value of the role_arns local block of the Terraform config. For example, if the role was created in the Development account, apply the ARN to the value for local.role_arns["dev"].
IAM Policy
A policy which will be used to grant access to provision resources within the same account for desired AWS services. Once created, attach to previously created role.
Example policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": [
"apigateway:*",
"cloudwatch:*",
"cloudwatch:ListMetrics",
"codedeploy:*",
"datapipeline:*",
"dax:*",
"dynamodb:*",
"ec2:*",
"ecr:*",
"ecs:*",
"es:*",
"events:*",
"iam:GetPolicy",
"iam:GetPolicyVersion",
"iam:GetRole",
"iam:GetRolePolicy",
"iam:GetUser",
"iam:GetUserPolicy",
"iam:ListAttachedRolePolicies",
"iam:ListAttachedUserPolicies",
"iam:ListPolicies",
"iam:ListPolicyVersions",
"iam:ListRolePolicies",
"iam:ListRoles",
"iam:PassRole",
"kms:*",
"lambda:*",
"logs:*"
],
"Resource": "*"
}
]
}
This will be highly variable based on needs, and will likely contain less wildcards for stricter controls.
Administrative Account
S3 Bucket
Creating an S3 bucket with default settings will usually do the trick. Copy bucket name to terraform s3 backend config once created.
DynamoDB Table
Create a dynamodb table for holding state locks. For the Primary Key / Partition Key, use the string LockID. Copy table name to TF config.
IAM User
Create an IAM User with programmatic access type. Save the generated credentials, and follow one of the methods for authentication with Terraform. I'm a fan of creating a separate AWS profile using the AWS CLI, then exporting that profile before interacting with Terraform.
IAM Policies
Create and attach the following policies to the previously created User, substituting variables indicated by brackets <...>:
Bucket and Dynamo Table Access
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": [
"s3:GetAccessPoint",
"s3:PutAccountPublicAccessBlock",
"s3:ListAccessPoints",
"dynamodb:ListTables",
"s3:ListJobs",
"dynamodb:ListBackups",
"dynamodb:ListStreams",
"dynamodb:ListContributorInsights",
"s3:GetAccountPublicAccessBlock",
"s3:ListAllMyBuckets",
"dynamodb:ListGlobalTables",
"s3:CreateJob",
"s3:HeadBucket",
"dynamodb:DescribeLimits"
],
"Resource": "*"
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"dynamodb:*"
],
"Resource": [
"<dynamodb_state_table_arn>"
]
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::<bucket_name>",
"arn:aws:s3:::<bucket_name>/*"
]
}
]
}
This policy simply allows the User / Terraform to read/write state in the bucket and state locks to the Dynamo table.
IAM Role Assumption
{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": [
"<arn_for_development_role",
"<arn_for_qa_role",
"<arn_for_production_role"
]
}
}
This policy allows the User (and by extension, Terraform) to assume the roles in the deployment accounts. This policy could further be separated into their own policies, one for each deploy account, to enable multiple users to have different levels of access.
Workflow
With the previous steps completed, day-to-day interactions with Terraform should be fairly smooth...
$ terraform init
-> initialize state, provider.
$ terraform workspace new dev
-> create dev workspace.
$ terraform workspace select dev
-> target dev workspace for further commands.
$ terraform plan
-> output plan for dev account.
$ terraform apply
-> apply configuration to dev account.
Happy Terraforming!
References
- Multi-Account architecture from the official Terraform docs.
- IAM Role Delegation from the official AWS docs.