Deploy DAGs with NFS

​

How NFS deploys work

1. APC validates the NFS location format (SERVER_IP:PATH).
2. APC creates a Kubernetes PersistentVolume (PV) pointing to your NFS server.
3. APC creates a PersistentVolumeClaim (PVC) bound to the PV.
4. The NFS volume is mounted read-only to the scheduler and workers at /usr/local/airflow/dags.
5. Dags are synced by writing files directly to your NFS server.

NFS Server (/dags)
       │
       ▼
┌──────────────┐
│ Kubernetes   │
│ PV + PVC     │
└──────┬───────┘
       │
       ├──► Scheduler (/usr/local/airflow/dags)
       │
       └──► Workers (/usr/local/airflow/dags)

​

Implementation considerations

• Namespace pools limitation: NFS deploys will not work if you use namespace pools and set global.clusterRoles to false. The NFS deploy feature requires creating PersistentVolumes, which are cluster-scoped resources.
• Dags only: NFS volumes deploy only Dags. To add Python dependencies or system packages, update your requirements.txt and packages.txt files and deploy using the CLI or CI/CD.
• Airflow version: NFS volumes require Airflow 2.0 or later.
• Read-only mount: The NFS volume is mounted read-only to Airflow components. Write operations must happen directly on the NFS server.

​

Prerequisites

• APC 1.0 or later installed
• An NFS server accessible from your Kubernetes cluster
• Network connectivity between cluster nodes and the NFS server
• Read access configured for UID/GID 50000 on the NFS share

​

Enable NFS volume storage

houston:
  config:
    deployments:
      configureDagDeployment: true
      nfsMountDagDeployment: true

helm upgrade astronomer astronomer/astronomer \
  -f values.yaml \
  --namespace astronomer

​

Provision an NFS volume

/srv/airflow-dags 10.0.0.0/8(rw,sync,no_subtree_check,all_squash,anonuid=50000,anongid=50000)

​

Configure NFS for a Deployment

astro deployment create my-deployment \
  --nfs-location "192.168.0.1:/dags"

astro deployment update <deployment-id> \
  --nfs-location "192.168.0.1:/dags"

mutation {
  upsertDeployment(
    workspaceUuid: "<workspace-uuid>"
    label: "my-deployment"
    dagDeployment: {
      type: volume
      nfsLocation: "192.168.0.1:/dags"
    }
  ) {
    id
    releaseName
  }
}

​

Deploy Dags to NFS volume

​

Direct copy

# Copy DAGs to NFS mount point
cp -r dags/* /mnt/nfs/dags/

​

Using kubectl

kubectl cp dags/ deployment-namespace/nfs-sync-pod:/dags/

​

CI/CD integration

name: Deploy DAGs to NFS
on:
  push:
    branches: [main]
    paths: ['dags/**']

jobs:
  deploy:
    runs-on: self-hosted  # Runner with NFS access
    steps:
      - uses: actions/checkout@v4
      - name: Sync DAGs
        run: |
          rsync -av --delete dags/ /mnt/nfs/airflow-dags/

​

Sync from cloud storage

# AWS S3 to EFS
aws s3 sync s3://my-bucket/dags/ /mnt/efs/dags/

# GCS to Filestore  
gsutil -m rsync -r gs://my-bucket/dags/ /mnt/filestore/dags/

# Azure Blob to Azure Files
azcopy sync "https://account.blob.core.windows.net/dags" "/mnt/azure/dags"

​

Verify NFS configuration

# List PersistentVolumes
kubectl get pv | grep dags

# List PersistentVolumeClaims in the deployment namespace
kubectl get pvc -n <deployment-namespace> | grep dags

kubectl exec -n <deployment-namespace> <scheduler-pod> -- \
  ls -la /usr/local/airflow/dags

​

Troubleshooting

​

Dags aren’t appearing

1. Check NFS connectivity:
   kubectl exec -n <namespace> <pod> -- \
     showmount -e <nfs-server-ip>
   
2. Verify mount permissions:
   kubectl exec -n <namespace> <scheduler-pod> -- \
     ls -la /usr/local/airflow/dags
   
3. Check PV/PVC status:
   kubectl describe pv <deployment>-dags-<hash>
   kubectl describe pvc -n <namespace> <deployment>-dags-<hash>
   

​

Permission denied errors

# On NFS server
chown -R 50000:50000 /srv/airflow-dags
chmod -R 755 /srv/airflow-dags

​

Stale file handle

# Restart affected pods
kubectl rollout restart deployment -n <namespace> <scheduler>
kubectl rollout restart statefulset -n <namespace> <workers>

​

Network connectivity issues

kubectl run nfs-test --rm -it --image=busybox -- \
  nc -zv <nfs-server-ip> 2049

​

Security considerations

• Network isolation: Use network policies to restrict which pods can access the NFS server.
• Access control: Configure NFS exports to allow only Kubernetes node IPs.
• Read-only mounts: APC mounts NFS volumes read-only to prevent accidental modifications from Airflow.
• Audit logging: Enable NFS server audit logging for compliance requirements.

​

Alternative: Git-sync deploys

• Version control for Dags.
• No external storage infrastructure required.
• Webhook-based or polling synchronization.
• Branch-based deployment strategies.