Set up your IDE for data engineering

After you use the Astro CLI to spin up an Airflow project locally, you can develop your Dags by editing the files in the project folder in any way you like. Some developers prefer to use an integrated development environment (IDE), such as VS Code or PyCharm, to edit their files. These code editors come with a lot of additional functionality and extensions. A common way to improve debugging for both you and your AI agent is to run your editor’s tooling inside a container built from your Airflow project, so it resolves code against the same Airflow version and Airflow provider versions your Dags run on. This gives you access to IDE features such as type checking, autocomplete, and debugging tools.

​

Assumed knowledge

To get the most out of this guide, you should have:

• An Astro project on your computer. See Run Airflow locally.

​

Dev containers

A dev container runs your editor’s tooling (language servers, linters, debuggers) inside the same container as your code. There are two ways to get one: attach your editor to the scheduler container that astro dev start is already running, or define the container declaratively in a devcontainer.json file that builds from your project’s Dockerfile. A dev container gives your editor:

• Autocomplete and type checking against the exact classes and provider packages installed in your Airflow project.
• Warnings for deprecated or unused imports before you run the Dag.
• Breakpoints and step-through debugging.

An AI agent running in that container has access to the same information, which lets it check code against the right Airflow and provider versions. See Agentic hooks for feeding those errors back to the agent automatically.

​

VS Code

The Dev Containers extension can attach to a container that’s already running.

1. Start Airflow with astro dev start.
2. Open the command palette (Shift+Cmd+P on macOS, Shift+Ctrl+P on Windows/Linux) and run Dev Containers: Attach to Running Container. Select <project>-scheduler-1.
3. In the new window, open /usr/local/airflow.
4. Install the Python extension inside the container. To skip this step on future attaches, set dev.containers.defaultExtensions in your VS Code settings.

You can also define your container in a devcontainer.json file. This is useful for sharing one VS Code setup across a team and builds a single container from your project’s Dockerfile that you and your AI agent can use for editing, type checking, and dag.test(). Because the dev container doesn’t start all five containers from astro dev start, the following example uses a dedicated, disposable SQLite metadata database for any functionality that requires database interaction.

1. In your Astro project, create a .devcontainer folder with a devcontainer.json file:
   {
     "name": "Astro Runtime Devcontainer",
     "build": {
       "context": "..",
       "dockerfile": "../Dockerfile"
     },
     "remoteEnv": {
       "AIRFLOW_HOME": "${containerWorkspaceFolder}",
       "AIRFLOW__DATABASE__SQL_ALCHEMY_CONN": "sqlite:////tmp/airflow.db"
     },
     "postCreateCommand": "rm -rf /tmp/airflow.db && airflow db migrate",
     "customizations": {
       "vscode": {
         "settings": {
           "python.defaultInterpreterPath": "/usr/local/bin/python"
         },
         "extensions": [
           "ms-python.python",
           "ms-python.vscode-pylance",
           "ms-vscode.live-server"
         ]
       }
     }
   }
   
2. Open the command palette and run Dev Containers: Reopen in Container.

The extensions list decides which tooling your agent can see. For example, ms-python.vscode-pylance adds type checking and import resolution. See the VS Code Extension Marketplace for more information.

​

PyCharm

The PyCharm Dev Containers feature can use the same .devcontainer/devcontainer.json file from the preceding VS Code section.

1. Connect PyCharm to Docker: open Settings, go to Build, Execution, Deployment > Docker, click +, and connect to your Docker daemon.
2. Add a new folder called exactly .devcontainer to your Airflow project’s root and create a devcontainer.json file in it, with the same contents as in the preceding VS Code section. Open the file.
3. Click the Dev Container icon in the editor’s left gutter, next to the file’s first line, and select Create Dev Container and Mount Sources…, then choose your backend IDE.
   
   If the icon doesn’t appear, restart PyCharm. It might not detect a devcontainer.json file created while the project was already open.
4. Watch the build progress in the Services tool window (View > Tool Windows > Services), then click Open Project after it finishes.

​

Debug with dag.test()

dag.test() runs every task in a Dag inside a single Python process, without requiring a running Airflow environment. Because it runs as regular Python code, you can set breakpoints and step through task logic with your IDE’s debugger. For more information, see Debug interactively with dag.test().