Secure TERRAFORM Deployment on AWS with GITLAB-CI and VAULT - Part 3 - Devoteam A Cloud (2023)

How do you secure Terraform deployments on AWS with Gitlab-CI and Vault? In previous articles we have seenIssues with CI/CD deployed in the cloudand how to solve these problemsUse Vault to dynamically generate secrets and validate Gitlab-CI pipelinesIn this third and final article, we will discuss secret recovery from the application side.

In the previous section, we saw how to solve various problems encountered in the process stage. But we use Gitlab CI is not limited to infrastructure implementation but also includes the application itself.

Summarizing what we have done so far at the workflow level, we have the following diagram:

As we can see, our application needs to retrieve the secrets stored in the Vault to connect to the database.

However, we want the interaction with the app and the vault to be as seamless as possible, and for that we need to reduce dependencies:

• About certification: Which method should we choose to make our application as transparent and secure as possible?
• About using secret keys: How to return dynamic secrets (short TTL) without affecting application code?

authentication application

When it comes to authenticating our apps using Vault, if we want it to be as seamless and secure as possible, it's important to base it on the environment where our app is deployed.

Our application is deployed on AWS, which is perfect because on the Vault side we haveAWS-style authentication methods.

There are two types of authentication:I amIEC2.

Since our application is deployed on an EC2 instance, we will use the EC2 type.

If we take a closer look at how this method works, we come across the following:

• 0 - So far Gitlab-CI has deployed our application to EC2 instances via Terraform and stored the database secrets in Vault.
• 1 – Once our EC2 instance is deployed, its metadata (such as instance ID, subnet, VPC ID where our EC2 instance is deployed, etc.) is obtained via the EC2 metadata service. You can findOfficial AWS Documentation.
• 2 - Our app is verified in Vault using AWS EC2 methodg PKCS7 signature.
• 3 – Vault authenticates and the EC2 instance hosting our application meets the authentication requirements (binding parameters) (eg is it in the correct VPC and subnet? is it the correct instance ID? etc.)
• 4 – If authentication is successful, Vault returns a token.

To implement this authentication method, you must:

• Vault must be able to verify the identity and metadata of an EC2 instanceto the target AWS account.

Specify authentication requirements (binding parameters) in the vault for which we want to allow the application to authenticate.

Identity and metadata verification

In order for Vault to view the information on our EC2 instance, it needs permission from the target account to describe the example as follows:ec2: Describes the case.

To do this, and following the same logic as in our previous demo, we'll create an IAM role that Vault will assume.

The IAM role must contain the following rules:

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "ec2:DescribeInstances" ], "Resource": "*" } ]}

Irelationship of trustAuthorize the source account (the account where the vault resides) to assume the IAM role. Regarding the configuration of the vault,We configure it through Terraform:

resource "vault_auth_backend" "aws" { description = "Auth backend to auth project in AWS env" type = "aws" path = "${var.project_name}-aws"}πόρος "vault_aws_auth_backend_sts_role" "role" { backend = vault_auth_backend. aws.path account_id = split(":", var.vault_aws_assume_role)[4] sts_role = var.vault_aws_assume_role}

As we can see, we also define the IAM role (STS) that Vault should assume.

At this point Vault can verify the information on our EC2 instance.

Set authentication constraints (binding parameters)

In terms of security, under what conditions do we allow our app to authenticate to the Vault to retrieve its secrets?

If we take a closer look at Vault's EC2-style AWS authentication approach, we can base it onvarious standards, For example:AMI ID, Account ID, Region, VPC ID, Subnet ID, and IAM Role ARN,ARN presence profiletheEC2 instance ID.

We can specify multiple criteria and multiple values ​​for each criterion. The value of each condition must be met for Vault to accept authentication.

In our case, our EC2 instances were created by Terraform. This is ideal because with Terraform we can retrieve all the snapshot properties and set them on the Treasury side asbinding parameters.which gives usNext Terraform clipCreate a Vault role for the AWS authentication backend:

资源 "vault_aws_auth_backend_role" "web" { backend = local.aws_backend role = var.project_name auth_type = "ec2" bound_ami_ids = [data.aws_ami.amazon-linux-2.id] bound_account_ids.ids. s = [data.aws_vpc.default.id] bound_subnet_ids = [aws_instance.web.subnet_id] bound_region = var.region token_ttl = var.project_token_ttl token_max_ttl = var.project_token_max_ttl",]project_token_max_ttl"]

In our case, we depend onAMI identifier,VPC number,subnet numberIAWS region.We could add an instance ID to strengthen the security of our authentication, but this pattern should be avoidedAuto Scaling Group.

At this point, our application can authenticate and retrieve its secret from the vault.

Secrets of using our application

On the Vault integration side of our application we will usecashier.

For those who want to know more details about Vault Agent integration, you can refer toThe articles below.

Let's take a closer look at our application workflow using Vault:

As we can see, the Vault agent manages two phases:

• AWSconfirmSpin using Vault and Vault Tokens
• secretrecoveryand their tobacco

underFormation of the Ministry of FinanceWe have:

auto_auth { 方法 { mount_path = "auth/${vault_auth_path}" type = "aws" config = { type = "ec2" role = "web" } } sink { type = "file" config = { path = "/home/ ec2-user/.vault-token" } }}predložak { izvor = "/var/www/secrets.tpl" odredište = "/var/www/secrets.json"}

Terraform can configure this file to override certain values, e.g.the role of the fundname orinstallation path.The days,As for the secret standards, we want to get the secret in JSON format, which gives us the following format:

{{{ με μυστικό "web-db/creds/web" }} "username":"{{ .Data.username }}", "password":"{{ .Data.password }}", "db_host": "${db_host}", "db_name":"${db_name}" {{ kraj }}}

Vault is responsible for all things Vault related (Vault tokens, secrets, refreshes, etc.)Let our app retrieve its secrets from the related files:

if (file_exists("/var/www/secrets.json")) { $secrets_json = file_get_contents("/var/www/secrets.json", "r"); $user = json_decode($json_secrets)->{'my friend'}; $pass = json_decode($json_secrets)->{'password'}; $host = json_decode($json_secrets)->{'db_host'}; $dbname = json_decode($secrets_json)->{'db_name ' };}else{ echo "This is not trying.";exception?}

This gives us the expected result:

Compared to what we do now, we have the following workflow:

• Terraform has a Vault service providerto simplify the interaction between the two tools.
• Some pipeline branches or Gitlab-CI tasks can be certified through VaultJWT authentication method.
• The Ministry of Finance allowsCloud provider credentials to createTo implement IaC through Terraform.for amazon, which can assume IAM roles across multiple AWS accounts, allowing our Terraform to implement IaC across multiple AWS accounts.
• The Ministry of Finance allowsCollect more types of secretsProvides information to projects in an environmentally neutral manner, including secrets generated by the IAC.
• Vault Agent simplifies vault integrationIn the application lifecycle that handles authentication and secrets
  • The Vault Token used by the Vault Agent is short-lived and changes frequently.
  • Application Secret (databaseIn our example) is short-lived and updated frequently via the Vault proxy.
• We allow our app to authenticate to Vault based on the environment it's in.For our AWS application, we rely on AWS EC2 authentication andbinding parametersFor example, Subnet ID, VPC ID, AWS Region, etc. where our app is located.

As we saw in this article, Vault allows us to protect Terraform end-to-end through Gitlab-CI, including IaC or our application itself.

Additionally, the Vault proxy allows us to reduce the application's dependency on Vault.

When used correctly, this integration is transparent to operators, developers, and applications. The secret becomes transparent to allshort life cycle.why would you know the secretWhat if we could have transparent secrets as a service?

Author :Mehdi Laruelle, Devoteam Revolve Alumni