›

Using AWS Secrets Manager to manage APM authentication keys

When using the config options ELASTIC_APM_SECRET_TOKEN or ELASTIC_APM_API_KEY for authentication of the Elastic APM AWS Lambda extension, the corresponding keys are specified in plain text in the environment variables of your Lambda function. If you prefer to securely store the authentication keys, you can use the AWS Secrets Manager and let the extension retrieve the actual keys from the AWS Secrets Manager. Follow the instructions below to set up the AWS Secrets Manager with the extension.

Step 1: Create a secret in the AWS Secrets Manager.

edit

Create a secret in the AWS Secrets Manager for the APM Secret Token or the APM API key, depending on which one you prefer to use. Make sure to create the secret as a Plaintext typed secret and ensure it is created in the same AWS region as your target Lambda function that will use the secret.

We recommend using the AWS-managed encryption key aws/secretsmanager. However, you can optionally create and select a custom KMS key for encryption. Note that with a custom encryption key, you will need additional key permissions on your Lambda function (see Step 2).

In the Web Console of the AWS Secrets Manager, navigate to Store a new secret, select Other type of secret as the secret type, and choose the Plaintext tab for entering the secret value. Enter your APM secret token or APM API key value as a plain string (not as a JSON key value pair).

image showing the creation of a plaintext secret in Secrets Manager

Click Next, choose a secret name, and finalize the creation of the secret.

Execute the following command to create a secret in the AWS Secrets Manager for your APM secret token or the APM API key:

aws secretsmanager create-secret \
    --name MyAPMSecretToken \  
    --description "The Elastic APM secret token." \
    --secret-string "YOUR_APM_AUTHENTICATION_KEY"

	Choose a secret name
	APM secret token or APM API key

Add the following resources to your terraform file to create a secret in the AWS Secrets Manager for your APM secret token or the APM API key:

...
resource "aws_secretsmanager_secret" "apm_secret_token" {
   name = "MyAPMSecretToken" 
}

resource "aws_secretsmanager_secret_version" "apm_secret_token_version" {
  secret_id = aws_secretsmanager_secret.apm_secret_token.id
  secret_string = "YOUR_APM_AUTHENTICATION_KEY" 
}
...

	Choose a secret name
	APM secret token or APM API key

Remember your chosen secret name. You will use the secret name as the value for the config options ELASTIC_APM_SECRETS_MANAGER_SECRET_TOKEN_ID or ELASTIC_APM_SECRETS_MANAGER_API_KEY_ID when configuring your Elastic APM AWS Lambda extension.

Step 2: Add permissions to your AWS Lambda function

edit

For your Lambda function to be able to retrieve the authentication key from the AWS Secrets Manager, you need to provide the following permissions to your Lambda function.

In the Web Console of your AWS Lambda function navigate to Configuration → Permissions and click on the link in the Execution Role section. This will lead you to the Web Console for the corresponding IAM role.

Select Create inline policy in the Add permission dropdown button and select the JSON tab to define the following JSON-based policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetSecretValue"
            ],
            "Resource": [
                "THE_ARN_OF_YOUR_SECRET"
            ]
        },
        { 
            "Effect": "Allow",
            "Action": [
                "kms:Decrypt"
            ],
            "Resource": [
                "THE_ARN_OF_YOUR_CUSTOM_KMS_KEY"
            ]
        }
    ]
}

This permission is ONLY needed if you use a custom KMS encryption key for your secret

Review and save the policy.

Create a policy file (smElasticApmPolicy.json) with the following content:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetSecretValue"
            ],
            "Resource": [
                "THE_ARN_OF_YOUR_SECRET"
            ]
        },
        { 
            "Effect": "Allow",
            "Action": [
                "kms:Decrypt"
            ],
            "Resource": [
                "THE_ARN_OF_YOUR_CUSTOM_KMS_KEY"
            ]
        }
    ]
}

This permission is ONLY needed if you use a custom KMS encryption key for your secret

Retrieve the execution role name of your Lambda function with the following command:

aws lambda get-function-configuration --function-name THE_NAME_OF_YOUR_FUNCTION | grep "Role"

Attach the new policy to the execution role:

aws iam put-role-policy \
  --role-name NAME_OR_ARN_OF_THE_EXECUTION_ROLE \
  --policy-name SecretsManagerAPMKeyPolicy \
  --policy-document file://smElasticApmPolicy.json

In your SAM template.yml file add the following policy to give the Lambda function access to the stored secret:

...
AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Resources:
  yourLambdaFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      ...
      Policies:
      - Statement:
        - Sid: SecretsManagerGetSecretValue
          Effect: Allow
          Action:
            - secretsmanager:GetSecretValue
          Resource: 'THE_ARN_OF_YOUR_SECRET'
        - Sid: KMSDecrypt 
          Effect: Allow
          Action:
            - kms:Decrypt
          Resource: 'THE_ARN_OF_YOUR_CUSTOM_KMS_KEY'
...

This permission is ONLY needed if you use a custom KMS encryption key for your secret

In your serverless file add the following policy to give the Lambda function access to the stored secret:

...
provider:
  iam:
    role:
      statements:
        - Effect: Allow
          Action: 'secretsmanager:GetSecretValue'
          Resource: 'THE_ARN_OF_YOUR_SECRET'
        - Effect: Allow 
          Action: 'kms:Decrypt'
          Resource: 'THE_ARN_OF_YOUR_CUSTOM_KMS_KEY'
...

This permission is ONLY needed if you use a custom KMS encryption key for your secret

Add the following policy and policy attachement resources to your terraform file to give the Lambda function access to the stored secret:

...
resource "aws_iam_role" "lambda_role" {
  // Here is your Lambda execution role
  ...
}

resource "aws_iam_policy" "secrets_manager_elastic_apm_policy" {
  name        = "secrets_manager_elastic_apm_policy"
  description = "Allows the lambda function to access the APM authentication key stored in AWS Secrets Manager."

  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "secretsmanager:GetSecretValue"
      ],
      "Effect": "Allow",
      "Resource": "THE_ARN_OF_YOUR_SECRET"
    },
    { 
      "Action": [
        "kms:Decrypt"
      ],
      "Effect": "Allow",
      "Resource": "THE_ARN_OF_YOUR_CUSTOM_KMS_KEY"
    }
  ]
}
EOF
}

resource "aws_iam_policy_attachment" "secrets_manager_elastic_apm_policy_attach" {
  role      = aws_iam_role.lambda_role.name
  policy_arn = aws_iam_policy.secrets_manager_elastic_apm_policy.arn
}
...

This permission is ONLY needed if you use a custom KMS encryption key for your secret

Step 3: Configure the Elastic APM AWS Lambda extension

edit

Finally, you will need to configure the Elastic APM AWS Lambda extension to use the secret from the Secrets Manager instead of the value provided through ELASTIC_APM_SECRET_TOKEN or ELASTIC_APM_API_KEY.

Provide the name of the secret you created in Step 1 as the value for the ELASTIC_APM_SECRETS_MANAGER_SECRET_TOKEN_ID or ELASTIC_APM_SECRETS_MANAGER_API_KEY_ID config option, respectively, depending on whether you want to use the APM Secret Token or the APM API key.

The language-specific instructions describe how to set environment variables for configuring AWS Lambda for Elastic APM:

That’s it. With the first invocation (cold start) of your Lambda function you should see a log message from the Elastic APM AWS Lambda extension indicating that a secret from the secrets manager is used:

"Using the APM secret token retrieved from Secrets Manager."

« Configuration options Release notes »