Skip to main content

Terraform examples

After you finish your first Terraform initialization, you can begin using Terraform scripts to automate, templatize, or programmatically manage your Astro environments.

Prerequisites

Example: Import existing resources

By adding both an import command and defining the configuration to your Terraform file, you can import existing Astro resources, such as Deployments or Workspaces, for Terraform to manage. After you successfully import them, Terraform fully manages your resources and you can update or delete them by updating the Terraform configuration file.

tip

Your Terraform file takes the highest priority in defining your Astro resource configuration. This means that when you import resources using Terraform, you need to specify all required fields if your Terraform file is different from any resource configurations that you configured through the Terraform CLI, Astro CLI, Astro API, or Astro UI. If the resources defined in your Terraform file and your actual resources are different, Terraform overrides your resource settings to match the configurations of the resources in your Terraform file.

Import existing Deployment as a resource example

In a Terraform file, add the following configuration.

// Import the new configuration to an existing Deployment. After you use `terraform apply` to apply this new configuration once, you can remove this section of code from your Terraform file.
import {
id = "<your-deployment-ID>" // ID of the existing deployment
to = astro_deployment.imported_deployment
}

// The new resource configuration.
resource "astro_deployment" "imported_deployment" {
name = "import me"
description = "an existing deployment"
type = "DEDICATED"
cluster_id = "<your-cluster-ID>"
contact_emails = ["preview@astronomer.test"]
default_task_pod_cpu = "0.25"
default_task_pod_memory = "0.5Gi"
executor = "KUBERNETES"
is_cicd_enforced = true
is_dag_deploy_enabled = true
is_development_mode = false
is_high_availability = true
resource_quota_cpu = "10"
resource_quota_memory = "20Gi"
scheduler_size = "SMALL"
workspace_id = "<your-workspace-ID>"
environment_variables = []
}

Example: Define variables with Terraform

You can define variables in a separate file and then call it from within other Terraform commands.

Define variables in Terraform code example

In a Terraform variables.tf file, you can define your variables:

variable "teams" {
type = map(object({
name = string
default_worker_queue_size = string
contact_emails = list(string)
}))
default = {
finance = {
name = "finance"
default_worker_queue_size = "A5"
contact_emails = ["foo@bar.com", "john@doe.com"]
}
ads = {
name = "ads"
default_worker_queue_size = "A10"
contact_emails = ["foo@bar.com", "john@doe.com"]
}
ml = {
name = "ml"
default_worker_queue_size = "A40"
contact_emails = ["foo@bar.com", "john@doe.com"]
}
}
}

Then, in your Terraform configuration file, main.tf, you can refer to those variables to create resources:

resource "astro_workspace" "team_workspaces" {
for_each = var.teams

name = "${each.value.name} Workspace"
description = "${each.value.name} Workspace"
cicd_enforced_default = true
}

Example: Create a Workspace per Team

You can use Terraform to script creating a Workspace for multiple Teams. The following example shows how to create a Workspace, a Cluster, and three Deployments for that Team. You can use the full code example in the Astro Terraform Provider's GitHub to create multiple teams following the same pattern.

Terraform Workspaces and Teams code example

Before using the code example, be sure to add your Astro Organization ID for the organization_id parameter.

/* Workspace Per Team

Team 1 Workspace
- Dev Deployment
- Stage Deployment
- Prod Deployment

*/

terraform {
required_providers {
astro = {
source = "astronomer/astro"
}
}
}

provider "astro" {
organization_id = "<your-astro-organization-id>"
}

resource "astro_workspace" "team_1_workspace" {
name = "Team 1 Workspace"
description = "Team 1 Workspace"
cicd_enforced_default = true
}

resource "astro_cluster" "team_1_cluster" {
type = "DEDICATED"
name = "Team 1 AWS Cluster"
region = "us-east-1"
cloud_provider = "AWS"
vpc_subnet_range = "172.20.0.0/20"
workspace_ids = []
timeouts = {
create = "3h"
update = "2h"
delete = "1h"
}
}

resource "astro_deployment" "team_1_dev_deployment" {
name = "Team 1 Dev Deployment"
description = "Team 1 Dev Deployment"
type = "STANDARD"
cloud_provider = "AWS"
region = "us-east-1"
contact_emails = []
default_task_pod_cpu = "0.25"
default_task_pod_memory = "0.5Gi"
executor = "CELERY"
is_cicd_enforced = true
is_dag_deploy_enabled = true
is_development_mode = true
is_high_availability = false
resource_quota_cpu = "10"
resource_quota_memory = "20Gi"
scheduler_size = "SMALL"
workspace_id = astro_workspace.team_1_workspace.id
environment_variables = []
worker_queues = [{
name = "default"
is_default = true
astro_machine = "A5"
max_worker_count = 10
min_worker_count = 0
worker_concurrency = 1
}]
scaling_spec = {
hibernation_spec = {
schedules = [{
is_enabled = true
hibernate_at_cron = "20 * * * *"
wake_at_cron = "10 * * * *"
}]
}
}
}

resource "astro_deployment" "team_1_stage_deployment" {
name = "Team 1 Stage Deployment"
description = "Team 1 Stage Deployment"
type = "STANDARD"
cloud_provider = "AWS"
region = "us-east-1"
contact_emails = []
default_task_pod_cpu = "0.25"
default_task_pod_memory = "0.5Gi"
executor = "CELERY"
is_cicd_enforced = true
is_dag_deploy_enabled = true
is_development_mode = true
is_high_availability = false
resource_quota_cpu = "10"
resource_quota_memory = "20Gi"
scheduler_size = "SMALL"
workspace_id = astro_workspace.team_1_workspace.id
environment_variables = []
worker_queues = [{
name = "default"
is_default = true
astro_machine = "A5"
max_worker_count = 10
min_worker_count = 0
worker_concurrency = 1
}]
scaling_spec = {
hibernation_spec = {
schedules = [{
is_enabled = true
hibernate_at_cron = "20 * * * *"
wake_at_cron = "10 * * * *"
}]
}
}
}

resource "astro_deployment" "team_1_prod_deployment" {
name = "Team 1 Prod Deployment"
description = "Team 1 Prod Deployment"
type = "DEDICATED"
cluster_id = astro_cluster.team_1_cluster.id
contact_emails = ["preview@astronomer.test"]
default_task_pod_cpu = "0.25"
default_task_pod_memory = "0.5Gi"
executor = "KUBERNETES"
is_cicd_enforced = true
is_dag_deploy_enabled = true
is_development_mode = false
is_high_availability = true
resource_quota_cpu = "10"
resource_quota_memory = "20Gi"
scheduler_size = "SMALL"
workspace_id = astro_workspace.team_1_workspace.id
environment_variables = [{
key = "key1"
value = "value1"
is_secret = false
}]
}

Was this page helpful?

Sign up for Developer Updates

Get a summary of new Astro features once a month.

You can unsubscribe at any time.
By proceeding you agree to our Privacy Policy, our Website Terms and to receive emails from Astronomer.