Secrets and configuration management in IaC: best practices in HashiCorp Vault and SOPS for security and efficiency

IaC secrets are the credentials, tokens, and API keys that secure your cloud accounts. Your IaC tools need these secrets to interact with your infrastructure resources, but configuring credentials directly in your pipelines is a security risk. It increases the size of your attack surface and makes it harder to audit where secrets are used.

‍

In this article, we're going to explain how to safely manage secrets in IaC workflows using dedicated secrets management solutions. Platforms like HashiCorp Vault and Mozilla SOPS let you safely store secrets outside your IaC pipelines, but these tools must be correctly integrated to ensure maximum security, traceability, and ease of use. We'll discuss common pitfalls to look out for and share best practices for working with IaC secrets at scale.

‍

Understanding IaC secrets risks

‍

Improperly handled secrets can have a devastating impact on your IaC workflows and broader infrastructure. Secrets that are exposed to unauthorized access may allow threat actors to compromise your cloud accounts. These breaches have severe security and compliance implications, but can be difficult to detect.

‍

IaC workflows are vulnerable to secrets-related issues because it's easy for mistakes to occur. Developers may unwittingly hardcode secrets in IaC files, for example, potentially causing values to become exposed within repositories. This apparently simple mistake has had disastrous real-world consequences: Uber's 2016 data breach was caused by the loss of an AWS key. Attackers with stolen GitHub credentials found the key inside a private repository, proving that nominally "private" access isn't enough to avoid common secrets risks.

‍

Similar issues can occur when you're manually configuring secrets in your IaC services, GitOps sync agents, and configuration management tools. But you need secrets to run your pipelines, so what can you do to keep your credentials protected? The answer lies with purpose-built secrets managers and IaC orchestration platforms.

‍

How to implement secure IaC secrets management

‍

Secrets management solutions like HashiCorp Vault, Mozilla SOPS, Pulumi ESC, and AWS Secrets Manager are dedicated to the safe storage of credentials, tokens, keys and other secrets. They allow you to keep your secrets in a single centralized location, outside your IaC workflows, then retrieve values at runtime within your IaC pipeline.

‍

Using a secrets manager offers several benefits for IaC CI/CD pipeline security:

• Secrets don't need to be saved in CI/CD tools or hardcoded into IaC pipelines.
• Rotating, updating, or removing secrets in the secrets manager automatically affects every pipeline.
• You can reliably audit secret usage because all requests pass through your secrets manager.
• It’s easier to implement advanced secrets workflows, such as dynamically generating short-lived secrets as they're requested.

‍

These advantages mean secrets managers should be seen as a must-have for secure Infrastructure as Code at scale. Preventing direct access to secrets and providing clear visibility into secret usage significantly improves your DevOps security posture. The model stops secrets being scattered across different repositories, CI/CD services, and deployment systems.

‍

Implementing secrets management in IaC begins with picking a secrets platform that works with your IaC tools. We'll look at some of the options you could choose in the next section. Once you've found a solution that works for your projects, you can then update your IaC configs to read required secrets from the secrets manager. Your IaC tool will fetch each secret's actual value at runtime, instead of depending on unsafe hardcoded values or environment variables.

‍

The actual integration steps will vary depending on the IaC tool and secrets management solution you're using. The following example is a basic demo of how to configure Terraform to read a secret called aws_secret from a HashiCorp Vault instance, using Terraform's Vault provider:

‍

provider "vault" {

address = "<vault-server-address>"

}

‍

data "vault_generic_secret" "aws_secret" {

path = "secret/aws_secret"

}

‍

provider "aws" {

region = "us-east-1"

access_key = data.vault_generic_secret.aws_secret.data["access_key"]

secret_key = data.vault_generic_secret.aws_secret.data["secret_key"]

}

‍

The demo works by configuring a new Vault provider instance that connects to a specified Vault server address. Terraform reads the aws_secret generic secret from the Vault instance, then uses the secret's properties to configure the aws provider. This enables safe IaC connectivity to AWS, without requiring manual distribution of cloud credentials. If a credential does accidentally leak, you can use secrets rotation in Vault to safely replace the secret without updating any Terraform code.

‍

To handle authentication between Terraform and Vault, you should set the VAULT_TOKEN environment variable to provide Terraform with a Vault access token generated on the Vault server. Several other authentication methods are also supported for improved security.

‍

You can also use HCP Vault within infrastructure orchestration platforms like Spacelift and Env0. These platforms allow you to fully automate your infrastructure processes via GitOps-driven CI/CD. Integrating with Vault via OIDC lets you safely fetch Vault secrets in your CI/CD runs, without having to manually configure Vault access tokens first.

‍

Comparing secrets management solutions for IaC

‍

The DevOps ecosystem includes a variety of popular secrets management tools for different use cases. The following three options are some favorite candidates, each with slightly different features and focus areas, but there's many other solutions available. The major cloud providers offer their own secrets managers, for instance, which can be especially useful when you're all-in on a particular cloud—try checking out AWS Secrets Manager, Azure Key Vault, and GCP Secret Manager in addition to the platforms discussed below.

‍

HashiCorp Vault

‍

HashiCorp Vault is one of the best-known secrets management solutions. It includes a full suite of features for storing, accessing, rotating, and auditing your credentials with identity-based security. The platform ensures secrets are only provided to verified identities that can prove they are who they claim to be, helping to enhance security.

‍

‍

Vault is available as a cloud-hosted SaaS solution or self-hosted on your own hardware. As a leading secrets management platform, it's also well-supported throughout the IaC ecosystem. IaC tools like Terraform and Pulumi include Vault providers, while integrations are also available within orchestrators like Spacelift.

‍

Mozilla SOPS

‍

SOPS (Secrets Operations) is positioned as a simple and flexible secrets management tool. It's a fully open-source solution which was begun by Mozilla. The tool focuses on the safe encryption of YAML, JSON, ENV, INI, and binary data using AWS KMS, GCP KMS, Azure Key Vault, and PGP. SOPS then provides access to the encrypted data and records audit logs for each usage event.

‍

‍

SOPS has a different architecture to centralized secrets managers like Vault. Whereas Vault stores all your secrets in one location outside your repository, SOPS is designed to encrypt your secrets at rest. You can then safely commit those files back into your repositories, CI/CD configs, and storage locations, potentially simplifying your workflows. The SOPS editor allows you to conveniently edit your encrypted files from your terminal.

‍

SOPS secrets encryption hasn't gained as much traction as other projects, but it's popular with DevOps teams seeking efficiency in smaller environments. You can access SOPS secrets in Terraform by using a community-contributed provider. Alternatively, SOPS is natively supported in Terragrunt via the sops_decrypt_file() function.

‍

Pulumi ESC

‍

Pulumi ESC (Environments, Secrets, and Configuration) is a centralized secrets and configuration management platform. It goes beyond just working with secrets because you can also store and synchronize entire environment configurations. This lets you easily encapsulate different environments with their secrets to simultaneously improve security and ease of use.

‍

‍

You can group environments together within Pulumi ESC, then reference them within each other. This enables you to flexibly compose environments from other environments with automatic secret inheritance. ESC also tracks the changes you make to individual environments and secrets, enabling you to efficiently audit compliance.

‍

Pulumi ESC is a relatively young platform that only launched in 2024. It has good integration with Pulumi's IaC system, but is yet to attain widespread adoption in the broader ecosystem. It's possible to use ESC with Terraform, but this depends on accessing your ESC configs as Terraform environment variables. There's no native ESC Terraform provider available yet.

‍

Best practices for managing secrets in IaC workflows

‍

Safe secrets management plays a crucial role in securing IaC workflows, as we've discussed above. Here's five more tips for protecting your secrets and enforcing IaC security best practices.

‍

1. Use dedicated secrets management tools

‍

It’s worth saying again: storing your secrets in a centralized platform such as HashiCorp Vault, Mozilla SOPS, or Pulumi ESC gives you a single destination for every secrets management task. It removes the need to manually configure secrets each time they're used, keeping your infrastructure better protected. Secrets managers also allow you to easily rotate secrets, audit their usage, and ensure continual encryption.

‍

2. Automate token rotation for your IaC secrets

‍

Regularly rotating your cloud account tokens and other secrets helps to mitigate the impact of breaches. Once a token's been rotated, it's no use to any attackers who may have acquired it. Many secrets managers support automatic token rotation so you can reduce token expiry times and improve your security posture. HCP Vault Secrets supports scheduled rotations, for example, while AWS Secrets Manager lets you run Lambda functions to implement custom rotation processes.

‍

3. Implement IaC orchestration platforms for safe hands-Off infrastructure access

‍

Running your IaC pipelines using purpose-built orchestration platforms like Spacelift, Env0, and HCP Terraform enables you to completely eliminate developer interactions with secrets. These platforms fully automate your infrastructure provisioning and configuration processes using a GitOps-powered workflow. They natively integrate with your cloud account APIs to request short-lived credentials when they're required, removing the need to manually configure secrets for each new pipeline. Developers can run key tasks on-demand—such starting a new development environment—without possessing any secrets or direct cloud provider access.

‍

4. Keep audit logs for secret access, update, and usage events

‍

Events concerning secrets should be logged so you can effectively investigate any suspected incidents. Secrets management platforms including HCP Vault, AWS Secrets Manager, and Pulumi ESC include built-in audit logging capabilities that allow you to see when secrets are used and who's interacting with them. This provides crucial information to support compliance requirements, such as by providing evidence that secrets are being rotated on time.

‍

5. Prefer dynamic secrets with just-in-time token generation

‍

Using a secrets manager lets you keep your secrets secure when they're not being used. However, secrets could still be leaked from your IaC pipelines after they're retrieved from your manager. Regular rotations help defend against this risk, but for even greater protection you should use dynamic credential generation strategies wherever possible.

‍

Dynamic credentials are extremely short-lived tokens that your secrets manager creates on-demand, at the time a secret is requested. The secrets manager interfaces with your cloud provider's access APIs to create a new token that's unique to each request. This helps mitigate the risks of secrets being exposed by external tools. Dynamic secrets are available in platforms including HCP Vault, Pulumi ESC, and Spacelift.

‍

Summary

‍

IaC secrets management is the process of securing your cloud API keys, tokens, and other credentials within your IaC pipelines. It's implemented using dedicated platforms like HashiCorp Vault, Mozilla SOPS, and Pulumi ESC that either store your secrets outside your IaC workflows, or safely encrypt them in your repositories.

‍

Centralized secrets managers are popular because they ensure values stay protected until the time they're actually needed. They make it easy to implement secret generation, rotation, and auditing processes that support safe DevOps automation at scale.

‍

Combining a secrets manager with a dedicated IaC orchestration platform like Spacelift enables you to confidently operate your infrastructure with less risk of credentials becoming exposed. You can increase development velocity while maintaining crucial DevOps security standards, leading to improved software delivery outcomes.

‍