Global environment variables

This document is a reference for all environment variables on Astronomer with different default values than open source Apache Airflow. You can override default Runtime environment variables, but you can’t override system environment variables. For information on setting your own environment variables, see Environment variables.

System environment variables

On Astro, certain environment variables have special handling. There are three types of special handling:

Override variables have their value set by Astro, regardless of what you set in your Deployment.
Default change variables have values that are different from the values in open source Apache Airflow. But, if you specify a value, your value takes precedence.
Unsafe to change variables are environment variables used by Astro to function. Do not override variables listed as Unsafe to change, because it can break Astro functionality.

The following table provides information about each global environment variable set by Astronomer.

The Astro UI does not currently prevent you from setting the environment variables listed as Unsafe to change. Attempting to set them can result in unexpected behavior that can include access problems, missing task logs, and failed tasks.If you need to set one of these variables for a particular use case, contact Astronomer support.

Default changed

Environment Variable	Applicability	Value
`AIRFLOW__TRIGGERER__DEFAULT_CAPACITY`	Both Airflow 2 and 3	`1000`
`AIRFLOW__CORE__PARALLELISM`	Both Airflow 2 and 3	Computed automatically from worker queue settings
`AIRFLOW__DAG_PROCESSOR__FILE_PARSING_SORT_MODE`	Only applies if you use HA or have an extra large size Deployment. Both Airflow 2 and 3.	`random_seeded_by_host`
`AIRFLOW__DAG_PROCESSOR__REFRESH_INTERVAL`	Airflow 3	`30`
`AIRFLOW__SCHEDULER__DAG_DIR_LIST_INTERVAL`	Airflow 2	`30`
`AIRFLOW__KUBERNETES_EXECUTOR__WORKER_PODS_CREATION_BATCH_SIZE`	Both Airflow 2 and 3	`16`
`AIRFLOW__LOGGING__LOG_FILENAME_TEMPLATE`	Both Airflow 2 and 3	`dag_id={{ ti.dag_id }}/run_id={{ ti.run_id }}/task_id={{ ti.task_id }}/{% if ti.map_index >= 0 %}map_index={{ ti.map_index }}/{% endif %}attempt={{ try_number\|default(ti.try_number) }}/{{ ti.id }}.log` (for AstroExecutor)
`AIRFLOW__SCHEDULER__TASK_QUEUED_TIMEOUT`	Both Airflow 2 and 3	`600` (for `<12.5.0,11.15.0`), `300` otherwise
`OPENLINEAGE_DISABLED=True`	Airflow 3 Remote Deployments
`AIRFLOW__SCHEDULER__TASK_INSTANCE_HEARTBEAT_TIMEOUT`	Astro Executor only. Both Airflow 2 and 3.	`60` for hosted Deployments, `120` for Remote Execution Deployments. Other executors use the Airflow default of `300`.
`AIRFLOW__SCHEDULER__SCHEDULE_AFTER_TASK_EXECUTION`	Airflow 2	`FALSE`
`AIRFLOW__KUBERNETES_EXECUTOR__WORKER_POD_PENDING_FATAL_CONTAINER_STATE_REASONS`	Only applies if you are on Runtime Version 11.15.0 or higher. Both Airflow 2 and 3.	`CreateContainerConfigError,CreateContainerError,ImageInspectError,InvalidImageName`
`AIRFLOW__CORE__XCOM_BACKEND`	Airflow 3 Astro Hosted, except Remote Execution Deployments. See XCom and state store backends.	`airflow_object_store.backends.xcom.AstroObjectStoreXCom`
`AIRFLOW__WORKERS__STATE_STORE_BACKEND`	Astro Runtime 3.3 and later on Astro Hosted, except Remote Execution Deployments. See XCom and state store backends.	`airflow_object_store.backends.state_store.AstroObjectStoreStateBackend`

Override

Environment Variable	Applicability	Value
`AIRFLOW__ASTRONOMER__CASBIN_DEPLOYMENT`	Both Airflow 2 and 3	Your Deployment ID
`ASTRO_LOGGING_ROLE_ARN`	Both Airflow 2 and 3
`AIRFLOW_CONN_ASTRO_GCS_LOGGING`	Both Airflow 2 and 3	`URI`
`AIRFLOW_CONN_ASTRO_AZURE_LOGS`	Both Airflow 2 and 3
`AIRFLOW_CONN_ASTRO_S3_LOGGING`	Both Airflow 2 and 3
`AIRFLOW__WEBSERVER__STATIC_CDN`	Both Airflow 2 and 3	`https://cdn.astronomer.io/airflow/`
`AIRFLOW__API__BASE_URL`	Airflow 3 only	`https://{orgid}.astronomer.run/d{deployment_id_suffix}`
`AIRFLOW__OPENSEARCH__HOST`	Airflow 3 only	`“”`
`ASTRO_AGENT_CLIENT_DISALLOW_DEFAULT_SECRET_BACKEND`	Airflow 3 Astro Hosted	`FALSE`
`ASTRO_AGENT_CLIENT_DISALLOW_DEFAULT_XCOM_BACKEND`	Airflow 3 Astro Hosted	`FALSE`
`AIRFLOW__DAG_PROCESSOR__DAG_BUNDLE_CONFIG_LIST`	Airflow 3 hosted
`AIRFLOW__ASTRO__TASK_LOG_URL_PATTERN`	Airflow 3 Remote	`URI`
`AIRFLOW__METRICS__STATSD_ALLOW_LIST`	Both Airflow 2 and 3	Variable is removed
`AIRFLOW__METRICS__STATSD_STATSD_CUSTOM_CLIENT_PATH`	Both Airflow 2 and 3	Variable is removed
`AIRFLOW__LOGGING__LOG_FILENAME_TEMPLATE`	Both Airflow 2 and 3	Variable is removed
`AIRFLOW__DAG_PROCESSOR__DAG_BUNDLE_CONFIG_LIST`	Both Airflow 2 and 3	Variable is removed
`AIRFLOW__API__PORT`	Both Airflow 2 and 3	`9091`
`AIRFLOW__API_AUTH__JWT_ALGORITHM`	Both Airflow 2 and 3	`EdDSA`
`AIRFLOW__API_AUTH__JWT_KID`	Both Airflow 2 and 3
`AIRFLOW__API_AUTH__JWT_PRIVATE_KEY_PATH`	Both Airflow 2 and 3	`/var/run/secrets/airflow/api-token-secret/api-token-signing-key`
`AIRFLOW__API_AUTH__TRUSTED_JWKS_URL`	Both Airflow 2 and 3	`/var/run/secrets/airflow/api-token-secret/api-token-jwks-key`
`AIRFLOW__CELERY__BROKER_URL`	Both Airflow 2 and 3
`AIRFLOW__CORE__FERNET_KEY`	The secret key for saving connection passwords in the metadata database. Both Airflow 2 and 3.	`fernetKeySecret`
`AIRFLOW__CORE__LAZY_LOAD_PLUGINS`	Both Airflow 2 and 3	`FALSE`
`AIRFLOW__DATABASE__CHECK_MIGRATIONS`	Both Airflow 2 and 3	`FALSE`
`AIRFLOW__DATABASE__EXTERNAL_DB_MANAGERS`	Both Airflow 2 and 3	`fab` in Airflow 2, `fab+astro` in Airflow 3
`AIRFLOW__KUBERNETES_EXECUTOR__NAMESPACE`	Both Airflow 2 and 3
`AIRFLOW__KUBERNETES__POD_TEMPLATE_FILE`	Both Airflow 2 and 3	reference to mounted `/usr/local/airflow/pod_template_file.yaml`
`AIRFLOW__LOGGING__COLORED_CONSOLE_LOG`	Both Airflow 2 and 3	`FALSE`
`AIRFLOW__OPERATORS__DEFAULT_QUEUE`	Both Airflow 2 and 3	Worker queue defined as default, usually `"default"`
`AIRFLOW__SCHEDULER__STANDALONE_DAG_PROCESSOR`	Both Airflow 2 and 3	`True`/`False` depending on Scheduler Size
`AIRFLOW__WEBSERVER__SECRET_KEY`	Both Airflow 2 and 3
`AIRFLOW__CELERY__RESULT_BACKEND`	Both Airflow 2 and 3
`AIRFLOW__DATABASE__SQL_ALCHEMY_CONN`	Both Airflow 2 and 3
`AIRFLOW__EXECUTION_API__JWT_AUDIENCE`	Both Airflow 2 and 3
`ASTRO_ORGANIZATION_ID`	Both Airflow 2 and 3	Your Astro Organization ID
`ASTRO_WORKSPACE_ID`	Both Airflow 2 and 3	Your Astro Workspace ID
`ASTRO_DEPLOYMENT_ID`	Both Airflow 2 and 3	Your Astro Deployment ID
`ASTRO_DEPLOYMENT_NAMESPACE`	Both Airflow 2 and 3	Your Astro Deployment Namespace

Unsafe to change

Environment Variable	Description	Value
`OPENLINEAGE_NAMESPACE`	Used internally by Astro for lineage and alerting. Do not change or overwrite.
`AIRFLOW__OPENLINEAGE__NAMESPACE`	Used internally by Astro for lineage and alerting. Do not change or overwrite.
`AIRFLOW__CELERY_BROKER_TRANSPORT_OPTIONS__SOCKET_TIMEOUT`	The amount of time that the Celery executor waits for a response from the Celery backend before throwing an error. Both Airflow 2 and 3.	`30`
`AIRFLOW__CELERY_BROKER_TRANSPORT_OPTIONS__SOCKET_CONNECT_TIMEOUT`	The amount of time that the Celery executor will attempt to connect to the Celery backend before retrying. Both Airflow 2 and 3	`5`
`AIRFLOW__CELERY_BROKER_TRANSPORT_OPTIONS__SOCKET_KEEPALIVE`	Whether the Celery executor will check whether the connection to the Celery backend is still alive. Both Airflow 2 and 3	`TRUE`
`AIRFLOW__CELERY_BROKER_TRANSPORT_OPTIONS__RETRY_ON_TIMEOUT`	Whether the Celery executor will retry a connection to the Celery backend when the connection fails. Both Airflow 2 and 3	`TRUE`
`AIRFLOW__LOGGING__DAG_PROCESSOR_LOG_TARGET`	Routes scheduler logs to `stdout`. Both Airflow 2 and 3	`stdout`
`AIRFLOW__LOGGING__REMOTE_LOGGING`	Enables remote logging. Airflow 2 and only Airflow 3 Azure remote Deployments.	`TRUE`
`AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER`	Location of remote logging storage. Enables remote logging. Airflow 2 and only Airflow 3 Azure remote Deployments.	`baseLogFolder`
`AIRFLOW__LOGGING__REMOTE_LOG_CONN_ID`	ID of the connection that provides access to your remote logging location. Enables remote logging. Airflow 2 and only Airflow 3 Azure remote Deployments.
`AIRFLOW__LOGGING__LOGGING_CONFIG_CLASS`	Class name that specifies the logging configuration. Enables remote logging. Airflow 2 and only Airflow 3 Azure remote Deployments.	`astronomer.runtime.logging.logging_config`
`AIRFLOW_CONN_ASTRO_S3_LOGGING`	Connection URI for writing task logs to Astro’s managed S3 bucket. Enables remote logging. Airflow 2 and only Airflow 3 Azure remote Deployments.
`AIRFLOW__LOGGING__ENCRYPT_S3_LOGS`	Determines whether to use server-side encryption for S3 logs. Both Airflow 2 and 3.	`FALSE`
`AIRFLOW__WEBSERVER__BASE_URL`	The base URL of the Airflow UI. Both Airflow 2 and 3	`https://${fullIngressHostname`
`AIRFLOW__CORE__SQL_ALCHEMY_CONN`	The SqlAlchemy connection string for the metadata database. Both Airflow 2 and 3.	`dbConnSecret`.
`AIRFLOW__WEBSERVER__UPDATE_FAB_PERMS`	Determines whether to update FAB permissions on webserver startup. Airflow 2 Only.	`TRUE`
`AIRFLOW__WEBSERVER__ENABLE_PROXY_FIX`	Determines whether to enable werkzeug ProxyFix middleware for reverse proxy. Airflow 2 only.	`TRUE`
`AIRFLOW__API__ENABLE_PROXY_FIX`	Determines whether to enable werkzeug ProxyFix middleware for reverse proxy. Airflow 3 only.	`TRUE`
`AIRFLOW__CORE__EXECUTOR`	The executor class that Airflow uses. Astro supports the Celery and Kubernetes executor. Both Airflow 2 and 3.	executor
`AIRFLOW_HOME`	The home directory for an Astro project. Both Airflow 2 and 3.	`usr/local/airflow`
`AIRFLOW__KUBERNETES__NAMESPACE`	The Kubernetes namespace where Airflow workers are created. Both Airflow 2 and 3.	`namespace`
`AIRFLOW__CORE__HOSTNAME_CALLABLE`	Path to a callable, which resolves to the hostname. Both Airflow 2 and 3.	`airflow.utils.net.get_host_ip_address`
`AIRFLOW__SCHEDULER__STATSD_ON`	Determines whether StatsD is on. Both Airflow 2 and 3.	`TRUE`
`AIRFLOW__SCHEDULER__STATSD_HOST`	The hostname for StatsD. Both Airflow 2 and 3.	`statsd.Hostname`
`AIRFLOW__SCHEDULER__STATSD_PORT`	The port for StatsD. Both Airflow 2 and 3.	`<statsd-port>`
`AIRFLOW__METRICS__STATSD_ON`	Determines whether metrics are sent to StatsD. Both Airflow 2 and 3.	`TRUE`
`AIRFLOW__METRICS__STATSD_HOST`	The hostname for sending metrics to StatsD. Both Airflow 2 and 3.	`statsd.Hostname`
`AIRFLOW__METRICS__STATSD_PORT`	The port for sending metrics to StatsD. Both Airflow 2 and 3.	`<statsd-metrics-port>`
`AIRFLOW__METRICS__STATSD_PREFIX`	The prefix for sending the metrics to StatsD. Both Airflow 2 and 3.	`airflow`
`AIRFLOW__WEBSERVER__COOKIE_SECURE`	Sets a secure flag on server cookies. Airflow 2 only.	`TRUE`
`AIRFLOW__WEBSERVER__INSTANCE_NAME`	Shows the name of your Deployment in the Home view of the Airflow UI. Airflow 2.	`<Deployment-Name>`
`AIRFLOW__API__INSTANCE_NAME`	Shows the name of your Deployment in the Home view of the Airflow UI. Airflow 3.	`<Deployment-Name>`
`AIRFLOW__CELERY__WORKER_CONCURRENCY`	Determines how many tasks each Celery worker can run at any given time and is the basis of worker auto-scaling logic	`<Max-Tasks-Per-Worker>`
`AIRFLOW__WEBSERVER__EXPOSE_CONFIG`	Exposes the Configuration tab of the Airflow UI and hides sensitive values. Airflow 2	`NON-SENSITIVE-ONLY`
`AIRFLOW__API__EXPOSE_CONFIG`	Exposes the Configuration tab of the Airflow UI and hides sensitive values. Airflow 3.	`NON-SENSITIVE-ONLY`
`AIRFLOW__USAGE_DATA_COLLECTION__ENABLED`	Disables the Airflow usage data collection and reporting to Scarf. Both Airflow 2 and 3.	`FALSE`
`AWS_DEFAULT_REGION`	(AWS clusters only) The region where your cluster is located. Both Airflow 2 and 3.	The region where you configured your cluster.
`AWS_SECRET_ACCESS_KEY`	The key secret for accessing Astro’s managed S3 bucket¹. Both Airflow 2 and 3
`INSTANCE_TYPE`	Provides the instance size of the node the dag is scheduled on. Both Airflow 2 and 3	`(v1:metadata.labels['beta.kubernetes.io/instance-type'])`
`OPENLINEAGE_URL`	The URL for your Astro lineage backend. The destination for lineage metadata sent from external systems to the OpenLineage API. Both Airflow 2 and 3
`OPENLINEAGE_API_KEY`	Your OpenLineage API key. Both Airflow 2 and 3

XCom and state store backends

On Astro Hosted, Astro stores XCom values larger than 64 KB in an Astro-managed object storage bucket instead of the Airflow metadata database. On Astro Runtime 3.3 and later, Astro also stores task and asset state values larger than 64 KB in the same bucket. Values of 64 KB or less stay in the metadata database. To do this, Astro sets the following environment variables on Deployments that run Astro Runtime 3.0 or later:

AIRFLOW__CORE__XCOM_BACKEND
AIRFLOW__WORKERS__STATE_STORE_BACKEND, on Astro Runtime 3.3 and later only

Astro doesn’t set these variables on Remote Execution Deployments.

Use your own XCom or state store backend

Astro keeps your own backend when you set the variable as an Astro environment variable, either on the Deployment or as a Workspace-level environment variable linked to the Deployment. If you set the variable anywhere in your Astro project image, such as with an ENV instruction in your Dockerfile, Astro replaces it with its own backend.

A backend can’t read values that a different backend stored in object storage. When a Deployment changes backends, tasks in Dag runs that are already in progress can fail if they read an XCom or state value stored before the change. Make the change when no Dag runs that depend on these values are in progress.

To move a backend from your image to a Deployment environment variable:

Open your Astro project Dockerfile and find the ENV instructions that set AIRFLOW__CORE__XCOM_BACKEND or AIRFLOW__WORKERS__STATE_STORE_BACKEND.
Add each variable to your Deployment with the same value. See Use the Astro UI.

You need to move only the backend variables. Other settings that your backend reads, such as AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_PATH, can stay in your Dockerfile. You can also keep the backend ENV instructions for local development, because the Deployment environment variable takes precedence on Astro. To switch a Deployment back to the Astro backend, delete the variable from the Deployment and from any Workspace-level environment variables linked to it. The same risk to in-progress Dag runs applies.

Overview

Get started

AI features

Build and deploy

Run and operate

Observe and monitor

Administration

Best practices

Reference

Release notes

Airflow 3

Global environment variables

System environment variables

Default changed

Override

Unsafe to change

XCom and state store backends

Use your own XCom or state store backend

​System environment variables

​Default changed

​Override

​Unsafe to change

​XCom and state store backends

​Use your own XCom or state store backend

System environment variables

Default changed

Override

Unsafe to change

XCom and state store backends

Use your own XCom or state store backend