Skip to main content

Set up AWS Systems Manager (SSM) Parameter Store

In this section, you'll learn how to use AWS Systems Manager (SSM) Parameter Store as a secrets backend on Astro.

If you use a different secrets backend tool or want to learn the general approach on how to integrate one, see Configure a Secrets Backend.

Prerequisites​

Step 1: Create Airflow secrets directories in Parameter Store​

Create directories for Airflow variables and connections in Parameter Store that you want to store as secrets.

Variables and connections should be stored in /airflow/variables and /airflow/connections, respectively. For example, if you're setting a secret variable with the key my_secret, it should be stored in the /airflow/connections/ directory. If you modify the directory paths, make sure you change the values for variables_prefix and connections_prefix in Step 2.

For instructions, see the AWS Systems Manager Console, the AWS CLI, or the Tools for Windows PowerShell documentation.

Step 2: Set up Parameter Store locally​

Add the following environment variables to your Astro project's .env file:

AIRFLOW__SECRETS__BACKEND=airflow.providers.amazon.aws.secrets.systems_manager.SystemsManagerParameterStoreBackend
AIRFLOW__SECRETS__BACKEND_KWARGS={"connections_prefix": "airflow/connections", "variables_prefix": "airflow/variables",  "role_arn": "<your-role-arn>", "region_name": "<your-region>"}

You can now run a DAG locally to check that your variables are accessible using Variable.get("<your-variable-key>").

Step 3: Deploy environment variables to Astro​

  1. Run the following commands to export your secrets backend configurations as environment variables to Astro.

    $ astro deployment variable create --deployment-id <your-deployment-id> AIRFLOW__SECRETS__BACKEND=airflow.providers.amazon.aws.secrets.systems_manager.SystemsManagerParameterStoreBackend

    $ astro deployment variable create --deployment-id <your-deployment-id> AIRFLOW__SECRETS__BACKEND_KWARGS='{"connections_prefix": "airflow/connections", "variables_prefix": "airflow/variables",  "role_arn": "<your-role-arn>", "region_name": "<your-region>"}' --secret

  2. (Optional) Remove the environment variables from your .env file or store your .env file in a safe location to protect your credentials in AIRFLOW__SECRETS__BACKEND_KWARGS.

Was this page helpful?