Deploy Dags with NFS

​

How NFS deploys work

1. APC validates the NFS location format (SERVER_IP:PATH).
2. APC creates a Kubernetes PersistentVolume (PV) pointing to your NFS server.
3. APC creates a PersistentVolumeClaim (PVC) bound to the PV.
4. The NFS volume is mounted read-only to the scheduler and workers at /usr/local/airflow/dags.
5. Dags are synced by writing files directly to your NFS server.

NFS Server (/dags)
       │
       ▼
┌──────────────┐
│ Kubernetes   │
│ PV + PVC     │
└──────┬───────┘
       │
       ├──► Scheduler (/usr/local/airflow/dags)
       │
       └──► Workers (/usr/local/airflow/dags)

​

Implementation considerations

• Namespace pools limitation: NFS deploys won’t work if you use namespace pools and set global.clusterRoles to false. The NFS deploy feature requires creating PersistentVolumes, which are cluster-scoped resources.
• Dags only: NFS volumes deploy only Dags. To add Python dependencies or system packages, update your requirements.txt and packages.txt files and deploy using the CLI or CI/CD.
• Airflow version: NFS volumes require Airflow 2.0 or later.
• Read-only mount: The NFS volume is mounted read-only to Airflow components. Write operations must happen directly on the NFS server.

​

Prerequisites

• APC 2.0 or later installed
• An NFS server accessible from your Kubernetes cluster
• Network connectivity between cluster nodes and the NFS server
• Read access configured for UID/GID 50000 on the NFS share

​

Enable NFS volume storage

1. In the APC UI, go to your Clusters page and select your cluster.
2. In the cluster details page, click Edit. In the Cluster Deployments Configuration YAML editor, click Find and search for the feature you want to override. Then add the following override in the appropriate YAML field:
   deployMechanisms:
     nfsMountDagDeployment:
       enabled: true
     configureDagDeployment:
       enabled: true
   
   For details on using the UI for configuration, see Override base configuration.
3. Save and apply your changes in the UI.

​

Provision an NFS volume

/srv/airflow-dags 10.0.0.0/8(rw,sync,no_subtree_check,all_squash,anonuid=50000,anongid=50000)

​

Configure NFS for a Deployment

astro deployment create my-deployment \
  --nfs-location "192.168.0.1:/dags"

astro deployment update <deployment-id> \
  --nfs-location "192.168.0.1:/dags"

mutation {
  upsertDeployment(
    workspaceUuid: "<workspace-uuid>"
    label: "my-deployment"
    dagDeployment: {
      type: volume
      nfsLocation: "192.168.0.1:/dags"
    }
  ) {
    id
    releaseName
  }
}

​

Deploy Dags to NFS volume

​

Direct copy

# Copy DAGs to NFS mount point
cp -r dags/* /mnt/nfs/dags/

​

Use kubectl

kubectl cp dags/ deployment-namespace/nfs-sync-pod:/dags/

​

CI/CD integration

name: Deploy DAGs to NFS
on:
  push:
    branches: [main]
    paths: ['dags/**']

jobs:
  deploy:
    runs-on: self-hosted  # Runner with NFS access
    steps:
      - uses: actions/checkout@v4
      - name: Sync DAGs
        run: |
          rsync -av --delete dags/ /mnt/nfs/airflow-dags/

​

Sync from cloud storage

# AWS S3 to EFS
aws s3 sync s3://my-bucket/dags/ /mnt/efs/dags/

# GCS to Filestore  
gsutil -m rsync -r gs://my-bucket/dags/ /mnt/filestore/dags/

# Azure Blob to Azure Files
azcopy sync "https://account.blob.core.windows.net/dags" "/mnt/azure/dags"

​

Verify NFS configuration

# List PersistentVolumes
kubectl get pv | grep dags

# List PersistentVolumeClaims in the deployment namespace
kubectl get pvc -n <deployment-namespace> | grep dags

kubectl exec -n <deployment-namespace> <scheduler-pod> -- \
  ls -la /usr/local/airflow/dags

​

Troubleshooting

​

Dags aren’t appearing

1. Check NFS connectivity:
   kubectl exec -n <namespace> <pod> -- \
     showmount -e <nfs-server-ip>
   
2. Verify mount permissions:
   kubectl exec -n <namespace> <scheduler-pod> -- \
     ls -la /usr/local/airflow/dags
   
3. Check PV/PVC status:
   kubectl describe pv <deployment>-dags-<hash>
   kubectl describe pvc -n <namespace> <deployment>-dags-<hash>
   

​

Permission denied errors

# On NFS server
chown -R 50000:50000 /srv/airflow-dags
chmod -R 755 /srv/airflow-dags

​

Stale file handle

# Restart affected pods
kubectl rollout restart deployment -n <namespace> <scheduler>
kubectl rollout restart statefulset -n <namespace> <workers>

​

Network connectivity issues

kubectl run nfs-test --rm -it --image=busybox -- \
  nc -zv <nfs-server-ip> 2049

​

Security considerations

​

In your Kubernetes cluster

• Network policies: Restrict which Pods can reach the NFS server using Kubernetes network policies in the Deployment namespace.

​

On your NFS server

• Export rules (/etc/exports): Restrict exports to specific Kubernetes node IPs or CIDR ranges, not broad networks. Avoid no_root_squash — it grants the client’s root user full root access on the NFS server, which is a significant security risk. Use root_squash or all_squash with anonuid=50000/anongid=50000 so Airflow can still read Dag files. The self-managed example above shows a working configuration.
• Export path scoping: Export only the directory subtree used for Dags. Don’t export a parent directory or the root filesystem.
• Firewall rules and NFS version: Prefer NFSv4, which uses only port 2049. NFSv3 requires opening additional ports for mountd, rpcbind, and statd.
• Kerberos: APC’s NFS deploy feature doesn’t support Kerberized NFS mounts. Houston generates the PersistentVolume with only a server and path — it doesn’t set mountOptions, so there’s no way to request sec=krb5 through this feature. If your NFS server requires Kerberos, this deploy mechanism isn’t currently compatible with it.
• Audit logging: Enable NFS server audit logging if required for compliance.

​

Alternative: Git-sync deploys

• Version control for Dags.
• No external storage infrastructure required.
• Webhook-based or polling synchronization.
• Branch-based deployment strategies.