Upsert a Deployment with the APC API

The examples on this page show common ways to use the APC API — they’re not a complete API reference. For the full, interactive API documentation for your installation, including every available query, mutation, and type, go to https://houston.<your-base-domain>/v1, and click on the Docs tab. See Develop and test APC API queries for more on using the built-in GraphQL explorer.

You can use the upsertDeployment mutation to both create and update Deployments with all possible Deployment configurations. If you query upsertDeployment without a deploymentUuid, the APC API creates a new Deployment according to your specifications. If you specify an existing deploymentUuid, the APC API updates the Deployment with that ID. All queries to create a Deployment require specifying a workspaceUuid.

When you make upsert updates to your Airflow Deployments, you must explicitly specify all existing environment variables, otherwise, the upsert overwrites them.

The following query creates a new Deployment in a custom namespace test-new-dep and configures a Deployment environment variable AIRFLOW__CORE__COLORED_LOG_FORMAT.

mutation upsertDeployment(
  $workspaceUuid: Uuid,
  $deploymentUuid: Uuid,
  $label: String,
  $description: String,
  $releaseName: String,
  $namespace: String,
  $environmentVariables: [InputEnvironmentVariable],
  $image: String,
  $dockerconfigjson: JSON,
  $version: String,
  $airflowVersion: String,
  $runtimeVersion: String,
  $executor: ExecutorType,
  $workers: Workers,
  $webserver: Webserver,
  $scheduler: Scheduler,
  $triggerer: Triggerer,
  $dagProcessor: DagProcessor,
  $dagDeployment: DagDeployment,
  $properties: JSON,
  $cloudRole: String
) {
  upsertDeployment(
    workspaceUuid: $workspaceUuid,
    deploymentUuid: $deploymentUuid,
    label: $label,
    description: $description,
    releaseName: $releaseName,
    namespace: $namespace,
    environmentVariables: $environmentVariables,
    image: $image,
    dockerconfigjson: $dockerconfigjson,
    version: $version,
    airflowVersion: $airflowVersion,
    runtimeVersion: $runtimeVersion,
    executor: $executor,
    workers: $workers,
    webserver: $webserver,
    scheduler: $scheduler,
    triggerer: $triggerer,
    dagProcessor: $dagProcessor,
    dagDeployment: $dagDeployment,
    properties: $properties,
    cloudRole: $cloudRole
) {
    id
    config
    urls {
      type
      url
      __typename
    }
    properties
    description
    label
    releaseName
    namespace
    status
    type
    version
    workspace {
      id
      label
      __typename
    }
    airflowVersion
    runtimeVersion
    upsertedEnvironmentVariables {
      key
      value
      isSecret
      __typename
    }
    dagDeployment {
      type
      nfsLocation
      repositoryUrl
      branchName
      syncInterval
      syncTimeout
      ephemeralStorage
      dagDirectoryLocation
      rev
      sshKey
      knownHosts
      __typename
    }
    createdAt
    updatedAt
    __typename
  }
}
{
  "workspaceUuid": "cldemxl9502454yxe6vjlxy23",
	"environmentVariables": [
    {
      "key": "AIRFLOW__CORE__COLORED_LOG_FORMAT",
      "value": "test",
      "isSecret": false
    }
  ],
  "releaseName": "",
  "namespace": "test-new-dep",
  "executor": "CeleryExecutor",
  "workers": {},
  "webserver": {},
  "scheduler": {
    "replicas": 1
  },
  "dagProcessor": {},
  "label": "test-new-dep",
  "description": "",
  "runtimeVersion": "7.2.0",
  "properties": {
    "extra_au": 0
  },
  "dagDeployment": {
    "type": "image",
    "nfsLocation": "",
    "repositoryUrl": "",
    "branchName": "",
    "syncInterval": 1,
    "syncTimeout": 120,
    "ephemeralStorage": 2,
    "dagDirectoryLocation": "",
    "rev": "",
    "sshKey": "",
    "knownHosts": ""
  }
}

More upsertDeployment examples

The following examples show upsertDeployment used for a few other common, narrower use cases.

Deploy a pre-built image from CI/CD

This approach is useful when you need to integrate with systems that can’t use the Astro CLI directly.

mutation {
  upsertDeployment(
    workspaceUuid: "<workspace-uuid>"
    clusterId: "<cluster-id>"
    releaseName: "my-deployment"
    image: "quay.io/myorg/airflow:v1.2.3"
    runtimeVersion: "12.1.0"
    deployRevisionDescription: "CI/CD Pipeline Deploy"
  ) {
    id
    status
  }
}

The mutation accepts the following fields:

workspaceUuid: The ID of the Workspace that contains the Deployment. You can provide workspaceLabel instead. One of the two is required.
clusterId: The ID of the cluster that hosts the Deployment.
releaseName: The release name of your Deployment, following the pattern spaceyword-spaceyword-4digits. For example, infrared-photon-7780.
image: The full image path including registry, repository, and tag. The image must be accessible from your Astro Private Cloud data plane.
runtimeVersion: The Astro Runtime version that the image is based on. For example, 12.1.0.
deployRevisionDescription: An optional description for the deploy revision, useful for tracking deploys in the APC UI.

For more information about deploying custom images with the APC API, see Configure a custom image registry.

Configure NFS Dag deployment

Use upsertDeployment to configure a Deployment’s Dag deployment mechanism as an NFS volume mount:

mutation {
  upsertDeployment(
    workspaceUuid: "<workspace-uuid>"
    label: "<deployment-name>"
    dagDeployment: {
      type: volume
      nfsLocation: "192.168.0.1:/dags"
    }
  ) {
    id
    releaseName
  }
}

For the full NFS setup, see Deploy Dags with NFS.

Configure object storage mount Dag deployment

Use upsertDeployment to configure a Deployment’s Dag deployment mechanism as an object storage mount from cloud object storage (APC 2.2 and later):

mutation {
  upsertDeployment(
    workspaceUuid: "<workspace-uuid>"
    label: "<deployment-name>"
    dagDeployment: {
      type: object_storage_mount
      storageUri: "gs://<bucket>/<prefix>"
      iamIdentity: "<gsa-name>@<project>.iam.gserviceaccount.com"
    }
  ) {
    id
    releaseName
  }
}

storageUri is the cloud storage URI (gs://…, s3://…, or an Azure blob endpoint or container), and iamIdentity is the cloud IAM identity (GCP service account email, AWS IAM role ARN, or Azure managed identity client ID). You can also set an optional mountPath to override the default /usr/local/airflow/dags. For the full setup, including cloud prerequisites, see Deploy Dags with an object storage mount.

Skip Airflow database provisioning

To use pre-existing or managed databases, set skipAirflowDatabaseProvisioning to true in the upsertDeployment mutation:

mutation {
  upsertDeployment(
    workspaceUuid: "<workspace-uuid>"
    label: "<my-deployment-label>"
    skipAirflowDatabaseProvisioning: true
  ) {
    id
  }
}

When using external databases, provide the connection string in your Deployment configuration. For complete setup steps with connection string examples, see Bring your own Airflow database.