AWS Secrets Manager

Retrieve AWS Secrets Manager values in Mage configuration files without storing them in source code.

Mage can retrieve secret strings from AWS Secrets Manager when it renders YAML and SQL templates. Use this integration for database passwords, API keys, and other credentials that should not be committed to your project.

In Mage Pro, this guide is also available from Settings > Access & security > Secrets management.

​

Prerequisites

Before configuring Mage:

1. Create a secret in AWS Secrets Manager in the Region Mage will use.
2. Grant the IAM role or user running Mage permission to call secretsmanager:GetSecretValue for the required secrets.
3. If a secret uses a customer-managed KMS key, also grant kms:Decrypt for that key.

For example, scope an IAM policy to the secrets Mage needs:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "secretsmanager:GetSecretValue",
      "Resource": "arn:aws:secretsmanager:us-west-2:123456789012:secret:prod/*"
    }
  ]
}

​

Configure AWS authentication

Mage checks environment variables first, then built-in Mage secrets with the same credential names. When no explicit access keys are configured, the underlying AWS SDK uses its default provider chain. In AWS, prefer an IAM role attached to the Mage workload so that you do not need long-lived access keys.

When a workload role is not available, set the following environment variables on every Mage process that renders templates containing AWS secret references:

Variable	Required	Description
AWS_ACCESS_KEY_ID	For access-key authentication	AWS access key ID.
AWS_SECRET_ACCESS_KEY	For access-key authentication	AWS secret access key.
AWS_SESSION_TOKEN	For temporary credentials	Session token paired with temporary access keys.
AWS_REGION_NAME or AWS_DEFAULT_REGION	Recommended	Region containing the secret. Mage checks AWS_REGION_NAME first, then AWS_DEFAULT_REGION; if neither is configured, it defaults to us-west-2.

These values may be provided as protected deployment environment variables. Mage also checks its built-in secret store for variables with these exact names, but that bootstrapping option requires Mage’s secret database to be available before the AWS client is created.

Do not place literal AWS credentials in io_config.yaml, pipeline code, or version control.

​

Reference a secret

Use aws_secret_var anywhere Mage template variables are supported:

{{ aws_secret_var('prod/postgres/password') }}

For example:

version: 0.1.1
default:
  POSTGRES_HOST: database.internal
  POSTGRES_USER: analytics
  POSTGRES_PASSWORD: "{{ aws_secret_var('prod/postgres/password') }}"

Pass either the secret name or its full ARN as the argument. A full ARN is useful when names are ambiguous, but it is not required for a secret in the configured Region and account.

aws_secret_var returns the complete SecretString. If the AWS secret contains JSON, extract the field you need with json_value:

version: 0.1.1
default:
  POSTGRES_USER: "{{ json_value(aws_secret_var('prod/postgres'), 'username') }}"
  POSTGRES_PASSWORD: "{{ json_value(aws_secret_var('prod/postgres'), 'password') }}"

Binary secrets are not supported by this template helper because it reads SecretString rather than SecretBinary.

​

Cache behavior and rotation

Mage uses the AWS Secrets Manager caching client and its library-managed refresh interval for normal aws_secret_var lookups. A rotated value is not guaranteed to appear immediately. Bypass the cache for a lookup with:

{{ aws_secret_var('prod/postgres/password', cached=False) }}

An uncached lookup calls GetSecretValue directly and increases AWS API traffic. Existing database connections and clients can continue using an older credential until they are recreated.

​

Troubleshooting

​

Access denied

Confirm the active IAM identity can call secretsmanager:GetSecretValue on the requested secret. For a customer-managed KMS key, also confirm it can call kms:Decrypt. Check both the IAM policy and any resource-based policy on the secret.

​

Secret not found

Verify the secret name or ARN and the configured Region. Mage uses AWS_REGION_NAME, then AWS_DEFAULT_REGION, and otherwise us-west-2. If the secret is in another account, use a full ARN and ensure the secret policy permits the Mage identity.

​

Invalid secret identifier

Confirm that the value passed to aws_secret_var is a valid secret name or ARN. Mage reports invalid AWS parameters separately from missing secrets.

​

Invalid credentials or expired token

For access-key authentication, confirm the access key and secret key belong together. Temporary credentials also require a current AWS_SESSION_TOKEN. Prefer a workload IAM role to avoid manually rotating these credentials.

​

JSON secret renders as one string

aws_secret_var returns the full secret string. Use json_value(aws_secret_var('secret-name'), 'field') to select a value from a JSON object.

​

Security recommendations

• Prefer workload IAM roles over long-lived access keys.
• Grant secretsmanager:GetSecretValue only for the secrets Mage requires.
• Use separate roles and secret paths for development, staging, and production.
• Enable AWS Secrets Manager rotation where supported by the downstream service.
• Avoid logging rendered configuration or secret values.
• Audit access with AWS CloudTrail.