2023 Astro release notes: July to December

​

Bug fixes

• Fixed an issue where creating an alert for a dag through the Astro API would apply the alert to the incorrect Deployment.
• Fixed an issue where Deployment worker Pods could crash when running the Kubernetes executor.
• Fixed an issue where Deployment API key expiration dates were not applied correctly if you configured multiple API keys at once.
• Fixed an issue where logging features could be disrupted if you set AZURE_CLIENT_ID as an environment variable. Note that this fix applies only to Astro Runtime 10 and later.

​

Bug fixes

• Fixed an issue where the Astro UI would produce an error if you updated an environment variable on an Astro Hybrid Deployment running the Kubernetes Executor.

​

Additional improvements

• The Astro Environment Manager is now generally available. This feature allows you to create and manage Airflow connections in the Astro UI.

​

Bug fixes

• Fixed an issue where dag code that appeared in the Airflow UI did not roll back when you rolled back a Deployment, even though the running code was successfully rolled back.
• Fixed an issue where you could not view billing information from the Astro UI when you installed Astro through the Azure Marketplace.
• Fixed an issue where the Astro UI would produce a console error when a user accessed their Workspace list.

​

Support for Microsoft Entra Workload ID

​

Bug fixes

• Removed the ability to create Hybrid Azure clusters in eastasia because some workload identity features aren’t supported in this region.

​

Install Astro from the Azure Marketplace

• You can manage billing from the Azure Portal.
• Microsoft Entra ID is pre-configured for all Organizations.
• It’s easier to create Astro resources and get started directly from Azure.

​

Create Airflow connections in the Astro UI and link them to Deployments

​

Trigger a dag from an Astro alert

​

New Azure regions available on Astro Hosted

• eastus
• canadacentral
• uksouth
• brazilsouth
• centralindia
• francecentral
• japaneast

​

Roll back Deployments to previous versions of your code

​

Bug fixes

• Fixed an issue where, you could inadvertently open the support request window if you opened a dag that included “Support” in its name from the DAGs view. As a result of this change, the support request window URL has been updated from https://cloud.astronomer.io/support to https://cloud.astronomer.io/open-support-request.
• Fixed an issue where the Deployment configuration menu in the Astro UI didn’t always show your Deployment’s current configuration.
• Fixed an issue where you could not create a Deployment with some stable Runtime versions using the Astro Platform API.

​

Deployment API keys are now deprecated

• You can’t create new Deployment API keys.
• If a Deployment has no configured Deployment API keys, the API keys tab will not appear.
• You can continue using existing Deployment API keys until a future end-of-support date.

​

Additional improvements

• Workspace Operators can now create, update, and delete Deployment API tokens.

​

Bug fixes

• Fixed an issue where a Deployment’s Updated By field was not updated if you transferred the Deployment.

​

New Azure regions available on Astro Hosted

• eastus2
• westus2
• westeurope

​

Bug fixes

• Fixed an issue where Deployment API tokens weren’t deleted after their associated Deployment was deleted.
• Fixed an issue where you could not create a Deployment with some available Runtime versions using the Astro Platform API.

​

Additional improvements

• You can now view deploy history for both Hosted and Hybrid Astro Deployments in the Astro UI. For more information, see View deploy history.

​

Bug fixes

• Modified behavior for Astro Hosted so that KubernetesExecutor and KubernetesPodOperator pods running in-cluster have equivalent resource requests and limits. If you don’t configure them to have equivalent resource requests and limits, Astro modifies them to the become the limits. Previously, the dag deploy would fail if resources did not equal limits.

​

Deployment API tokens

​

New Edit Deployments in the Astro UI

​

Additional Improvements

• Added a DAG Success alert so you can now set up an alert for successful completion events. See how to set up Astro alerts.

​

Bug Fixes

• Fixed a problem in the Astro UI where a warning about Deployment Health was displayed when a Workspace had zero Deployments.

​

Introducing the Astro API

​

Manage Deployments programmatically using Deployment API tokens

• Pushing code to a Deployment.
• Updating a Deployment’s environment variables.
• Making requests to update your Deployment’s Airflow environment using the Airflow REST API.

​

Additional improvements

• When you create a new Deployment, the Astro UI now presents new options and suggestions for running your first dag.
• You can now retrieve a Workspace’s ID from the Astro UI. To find a Workspace’s ID, open the Workspace in the Astro UI and go to Workspace Settings > General.

​

Per-Deployment IAM workload identities on AWS

{ "elasticloadbalancing:DescribeLoadBalancers", "elasticloadbalancing:DeleteLoadBalancer" }

1. In the AWS Management Console, go to the Identity and Access Management (IAM) dashboard. Identify all of your trust policies that specify your cluster workload identity. They should look similar to the following trust policy:
   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Principal": {
                   "AWS": [
                       "arn:aws:iam::<dataplane-AWS-account-ID>:role/AirflowS3Logs-<cluster-ID>"
                   ]
               },
               "Action": "sts:AssumeRole"
           }
       ]
   }
   
2. For each trust policy, add the workload identities for any Deployments that you want to access the related resource. To locate your Deployment workload identity, open the Deployment in the Astro UI and copy the Workload Identity from the Details page. Your trust policy should now look like the following:
   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Principal": {
                   "AWS": [
                       "arn:aws:iam::123456789876:role/AirflowS3Logs-cl6zcnlc641hr0voibivf21jh",
                       "arn:aws:iam::<dataplane-AWS-account-ID>:role/astro-<namespace>"
                   ]
               },
               "Action": "sts:AssumeRole"
           }
       ]
   }
   
3. For each Deployment that you specified in your trust policies, open the Deployment in the Astro UI and click Details, then click Edit Details. In the Workload Identity section, select the new Deployment identity from the dropdown list, then click Update. To avoid disruption to tasks, don’t complete this step until you have added the Deployment workload identity to all of the trust policies it needs for access.
4. Upgrade to the latest Astro CLI release, which includes support for Per Deployment IAM Workload Identity.
5. After you’ve tested the policies with your Deployment workload identities, remove the cluster workload identity from your trust policies

​

Bug fixes

• Fixed an issue where Billing Admins could view task usage on the Usage page only for Workspaces that they belonged to. Now, Billing Admins can view usage for all Workspaces regardless of their Workspace role.

​

Additional improvements

• The Astro UI Usage page now shows task usage for deleted Deployments. If you’re an Astro Hybrid Billing Admin, this means that task usage metrics now better reflect your billable usage.
• When you create a Deployment through the Astro UI and choose an Astro Runtime version, you can now select only the most recent supported patch for each major version of Astro Runtime.
• You can now filter task logs by log level or source from the DAGs page in the Astro UI.

​

View deploy history in the Astro UI

​

View rendered data figures in the Astro Cloud IDE

​

Changes to Workspace user roles

• The Workspace Author role is a new role for users who primarily write and deploy dags. Users with this role can push code changes, but they can’t update Deployment or Airflow settings such as Airflow variables, Astro environment variables, or connections.
• The Workspace Admin role has been renamed to Workspace Owner. Users with this role are responsible for administrating membership to the Workspace.
• The Workspace Editor role has been renamed to Workspace Operator. In addition to pushing code changes, Workspace Editors can now also edit Airflow objects such as variables, connections, and XComs. Users with this role are responsible for managing the environments that dags run in.
• The Workspace Viewer role has been renamed to Workspace Member. Users with this role only need viewing permissions for a Deployment and don’t have permissions to make any code or configuration changes.

​

Additional improvements

• The lifespan of the personal user access token you can retrieve from cloud.astronomer.io/token has been reduced from 24 hours to 1 hour.
• The DAGs view of the Astro UI now shows your configured dependency edge labels in the graph view.
• The Astro UI now shows more detailed instructions for deploying code when you create a new Deployment.
• The Deployment Analytics page in the Astro UI has been renamed to Overview.

​

Bug fixes

• Fixed an issue where Deployments using the Kubernetes executor could not run dags with lower resource requests than the Default Pod Size. Minimum requests are now hard-coded and decoupled from default requests.

​

Additional improvements

• You can now configure task log forwarding to Datadog at the Deployment level.
• In the DAGs view of the Astro UI, you can now double click a task run node in the graph view to view the task run’s logs and mapped tasks.
• The A50 worker type has been renamed to A60 to make it consistent in scale with other worker types.
• The max possible CPU quota and Memory quota for a Deployment running in a Hosted dedicated cluster has increased to 1600 vCPU/ 3200 GiB respectively.

​

Additional improvements

• You can now see how many Astro alerts you’ve configured for a dag in the DAGs page of the Astro UI.

​

Bug fixes

• Fixed an issue where you couldn’t run Astro Cloud IDE pipelines that included a Markdown cell.
• Fixed an issue where an Organization’s SSO bypass link was formatted incorrectly in the Astro UI.

​

New Hosted worker type

​

Additional improvements

• When you create a new Astro Cloud IDE project, you can now specify whether you want the project to include an example pipeline.
• You can now access Organization-level settings in the Astro UI only through the Organization Settings link. Additionally, some Organization settings have been moved to the top level of navigation so that there is no longer a Settings menu.
• You can now use commas, apostrophes, and ampersands in Workspace and Organization names.
• The Workspace list view in the Astro UI has been redesigned so that Organization Owners can now edit and delete Workspaces directly from the list.

​

Bug fixes

• Fixed an issue where the Astro UI showed incorrect CPU and memory limits in bar charts on the Deployments list and Details page.

​

Hosted Deployments have dag-only deploys enabled by default

​

Teams now have Organization-level roles

​

Additional improvements

• The Astro UI now shows how many Workspaces, dags, clusters, and Astro Cloud IDE projects you have in the left sidebar.
• You can now create Deployments in standard clusters hosted in europe-west2 on GCP and eu-central-1 on AWS.
• The default metadata database instance type for new Deployments on GCP clusters in Astro Hybrid has been reduced to Small General Purpose with 2 vCPUs and 8GiB. See GCP Hybrid cluster settings.

​

Bug fixes

• Because some regions don’t support specific machine types that Astro Hosted uses, you can no longer create Hosted dedicated clusters in the following AWS regions:
  • af-south-1
  • ap-east-1
  • ap-northeast-2
  • ap-northeast-3
  • ca-central-1
  • eu-south-1
  • eu-west-3
  • me-south-1

​

Additional improvements

• The templates for Astro alert messages have been updated to include more information about the Deployment that the alert was triggered in, including a link to the dag that triggered the alert.

​

Bug fixes

• Fixed an issue where Azure AD single sign-on (SSO) connections were incorrectly labeled as SAML connections in the Astro UI.

​

Configure default Pod sizes

​

New regions available on Astro Hosted

• AWS
  • af-south-1 - Africa (Cape Town)
  • ap-east-1 - Asia Pacific (Hong Kong)
  • ap-northeast-1 - Asia Pacific (Tokyo)
  • ap-northeast-2 - Asia Pacific (Seoul)
  • ap-northeast-3 - Asia Pacific (Osaka)
  • ap-southeast-1 - Asia Pacific (Singapore)
  • ap-southeast-2 - Asia Pacific (Sydney)
  • ap-south-1 - Asia Pacific (Mumbai)
  • ca-central-1 - Canada (Central)
  • eu-central-1 - Europe (Frankfurt)
  • eu-south-1 - Europe (Milan)
  • eu-west-1 - Europe (Ireland)
  • eu-west-2 - Europe (London)
  • eu-west-3 - Europe (Paris)
  • me-south-1 - Middle East (Bahrain)
  • sa-east-1 - South America (São Paulo)
  • us-east-1 - US East (N. Virginia)
  • us-east-2 - US East (Ohio)
  • us-west-1 - US West (N. California)
  • us-west-2 - US West (Oregon)
• GCP
  • asia-east1 - Taiwan, Asia
  • asia-northeast1 - Tokyo, Asia
  • asia-northeast2 - Osaka, Asia
  • asia-northeast3 - Seoul, Asia
  • asia-south1 - Mumbai, Asia
  • asia-south2 - Delhi, Asia
  • asia-southeast1 - Singapore, Asia
  • asia-southeast2 - Jakarta, Asia
  • australia-southeast1 - Sydney, Australia
  • australia-southeast2 - Melbourne, Australia
  • europe-central2 - Warsaw, Europe
  • europe-north1 - Finland, Europe
  • europe-southwest1 - Madrid, Europe
  • europe-west1 - Belgium, Europe
  • europe-west2 - England, Europe
  • europe-west3 - Frankfurt, Europe
  • europe-west4 - Netherlands, Europe
  • europe-west6 - Zurich, Europe
  • europe-west8 - Milan, Europe
  • europe-west9 - Paris, Europe
  • northamerica-northeast1 - Montreal, North America
  • northamerica-northeast2 - Toronto, North America
  • southamerica-east1 - São Paulo, South America
  • southamerica-west1 - Santiago, South America
  • us-central1 - Iowa, North America
  • us-east1 - South Carolina, North America
  • us-east4 - Virginia, North America
  • us-east5 - Columbus, North America
  • us-south1 - Dallas, North America
  • us-west1 - Oregon, North America
  • us-west2 - Los Angeles, North America
  • us-west3 - Salt Lake City, North America
  • us-west4 - Nevada, North America

​

Bug fixes

• You can no longer create Hybrid GCP clusters in eu-north-1.

​

Send Astro alerts to email

​

Configure SCIM provisioning for Azure AD

​

Additional improvements

• On Astro Hosted deployments, the astronomer_monitoring_dag has been paused for all image-based Deployments and removed entirely from all Deployments with dag deploys enabled. It has been replaced with an implementation that allows workers on Deployments to fully scale to 0.

​

Configure SCIM provisioning for Okta

​

See pricing estimate when creating a Deployment

​

Additional improvements

• You can now configure Deployments with the A40 machine type, which has 8 vCPU and 16 GiB. See Astro hosted resource reference.
• You can now access your Organization settings from a Workspace by clicking the name of your Organization/ Workspace.
• On Astro Hybrid, the default Azure DB instance is now Standard D2ds_v4.
• The Deployment creation screen for Astro Hybrid has received several updates that were previously only available on Astro Hosted.

​

Bug fixes

• Fixed an issue where dags that used the KubernetesPodOperator and had tasks with in_cluster=False could not be parsed.