Mitigate the eMudhra CA compromise on older Runtime versions

In October 2026, the private keys for eight intermediate certificate authorities (CAs) operated by eMudhra were compromised. Because these intermediates chain to widely trusted roots, an attacker could issue a certificate that any system that hasn’t been updated accepts as valid. See the eMudhra incident summary for details. The following Astro Runtime versions include the complete fix:

Runtime line	Fixed in (release notes)
Astro Runtime 13 (Airflow 2)	13.12.2
Astro Runtime 3.3	3.3-11
Astro Runtime 3.2	3.2-13
Astro Runtime 3.1	3.1-24
Astro Runtime 3.0	3.0-21

The previous releases, 13.12.1, 3.3-10, 3.2-12, 3.1-23, and 3.0-20, include only a partial fix. If you run one of them, upgrade to the release in the preceding table instead of following this guide. Astro Runtime 12 and earlier are out of maintenance and don’t include the fix. If you can’t upgrade to a fixed version yet, use this guide to apply the same mitigation to your own image.

Upgrading is the recommended fix. Use this guide only as a temporary measure until you can upgrade. See Upgrade Astro Runtime.

Add each step to your project’s Dockerfile, after the FROM line. The FROM line depends on your Runtime version:

Astro Runtime 13 and earlier: FROM astrocrpublic.azurecr.io/astronomer/astro-runtime:<version>, such as 12.12.0.
Astro Runtime 3.x: FROM astrocrpublic.azurecr.io/runtime:<version>, such as 3.2-11.

Add the same suffix, such as -ubi9 or -python-3.11, that you use today. See Operating system variant images.

Affected root certificates

Distrust both of the following roots in your image:

emSign ECC Root CA - G3
Comodo AAA Services root

Distrusting these roots can break TLS connections to any endpoint whose certificate chain depends only on them, including some non-eMudhra services. Test your connections in a development Deployment before you deploy to production.

Step 1: Update the operating system trust store

Choose the tab that matches your image.

Debian 12
Debian 11
RHEL UBI

Use this tab for the default images of Astro Runtime 13, 12, and 3.x. Upgrading all packages also picks up other security fixes that have been released since your Runtime version.

USER root
RUN apt-get update && apt-get upgrade --yes && \
    rm -rf /var/lib/apt/lists/* && \
    test ! -e /etc/ssl/certs/emSign_ECC_Root_CA_-_G3.pem && \
    test ! -e /etc/ssl/certs/Comodo_AAA_Services_root.pem
USER astro

Use this tab for the default images of Astro Runtime 11. Debian 11 is end of life and its package repositories no longer receive the fix, so this example disables the two roots directly.

USER root
RUN sed -i \
        -e 's|^mozilla/emSign_ECC_Root_CA_-_G3.crt$|!&|' \
        -e 's|^mozilla/Comodo_AAA_Services_root.crt$|!&|' \
        /etc/ca-certificates.conf && \
    update-ca-certificates --fresh && \
    test ! -e /etc/ssl/certs/emSign_ECC_Root_CA_-_G3.pem && \
    test ! -e /etc/ssl/certs/Comodo_AAA_Services_root.pem
USER astro

USER root
RUN trust extract --overwrite --format=pem-bundle \
        --filter='pkcs11:object=emSign%20ECC%20Root%20CA%20-%20G3;type=cert' \
        /etc/pki/ca-trust/source/blocklist/emSign_ECC_Root_CA_-_G3.pem && \
    trust extract --overwrite --format=pem-bundle \
        --filter='pkcs11:object=Comodo%20AAA%20Services%20root;type=cert' \
        /etc/pki/ca-trust/source/blocklist/Comodo_AAA_Services_root.pem && \
    update-ca-trust && \
    ! grep --quiet --fixed-strings \
        --regexp="emSign ECC Root CA - G3" --regexp="Comodo AAA Services root" \
        /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem \
        /etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem \
        /etc/pki/ca-trust/extracted/pem/objsign-ca-bundle.pem
USER astro

All three examples fail the build if either root is still trusted.

Step 2: Update Python packages that bundle CA lists

Python libraries such as requests use the certifi CA bundle instead of the operating system trust store. gRPC clients, such as the OpenTelemetry gRPC exporter and Google Cloud client libraries, use the CA list that grpcio bundles, not certifi or the operating system trust store. The pip, virtualenv, and uv packaging tools also bundle their own CA lists. In Astro Runtime 13 and earlier, the PythonVirtualenvOperator uses virtualenv to create its environments. Add the fixed versions to your requirements.txt file:

certifi>=2026.10.19
grpcio>=1.84.1
pip>=26.2.2
virtualenv>=21.14.12
uv>=0.13.5

Astro Runtime 13 and earlier install packages with pip and don’t include uv. Adding uv is safe.

pip 26.2.2 and grpcio 1.84.1 require Python 3.10 or later, and no release of either for Python 3.9 includes the fix. If you use an Astro Runtime 11 image with Python 3.9, switch to an image with Python 3.10 or later, such as 11.20.0-python-3.10.

Step 3: Update the pip that new virtual environments get

When you create a virtual environment with python -m venv, Python installs the copy of pip that it bundles, not the pip installed in your image. In affected Runtime versions, that bundled copy is older than pip 26.2.2. The following example replaces it with pip 26.2.2:

USER root
RUN ENSUREPIP_DIR=$(python -c 'import ensurepip, os; print(os.path.dirname(ensurepip.__file__))') && \
    rm -f ${ENSUREPIP_DIR}/_bundled/pip-*.whl && \
    python -m pip download --quiet --no-deps --only-binary=:all: \
        --dest ${ENSUREPIP_DIR}/_bundled pip==26.2.2 && \
    sed -i 's/^_PIP_VERSION = .*/_PIP_VERSION = "26.2.2"/' ${ENSUREPIP_DIR}/__init__.py && \
    python -m venv /tmp/venv-check && \
    /tmp/venv-check/bin/python -m pip --version | grep --quiet "^pip 26.2.2 " && \
    rm -rf /tmp/venv-check
USER astro

The last three lines create a test environment and fail the build if it doesn’t get pip 26.2.2. If your image tag includes -ubi9, skip this step. In those images, the bundled pip uses the operating system trust store, which you updated in Step 1.

Step 4: Check for custom CA bundles

If you set REQUESTS_CA_BUNDLE, SSL_CERT_FILE, CURL_CA_BUNDLE, or GRPC_DEFAULT_SSL_ROOTS_FILE_PATH to a custom CA bundle, remove the two roots from that bundle as well. Tools that you add to your image with their own bundled CA list, such as the Google Cloud CLI, need their own updates.

Step 5: Test and deploy

Run astro dev restart to rebuild the image locally.
Run a Dag that connects to your key external services to confirm that TLS connections still work.
Deploy with astro deploy.

Next steps

Plan an upgrade to a Runtime version that includes the complete fix, such as Astro Runtime 3.3-11. Once you upgrade, you can revert your Dockerfile to the standard image and remove the extra steps.

​Affected root certificates

​Step 1: Update the operating system trust store

​Step 2: Update Python packages that bundle CA lists

​Step 3: Update the pip that new virtual environments get

​Step 4: Check for custom CA bundles

​Step 5: Test and deploy

​Next steps

Affected root certificates

Step 1: Update the operating system trust store

Step 2: Update Python packages that bundle CA lists

Step 3: Update the pip that new virtual environments get

Step 4: Check for custom CA bundles

Step 5: Test and deploy

Next steps