Skip to main content

Work with assets

When you work with Airflow, you create and consume different resources to craft DAGs, whether it's specific tasks that your DAG performs, or tables and local files where your input and DAG output data is stored. These resources, tasks, datasets, warehouse tables, and local files, can all be data product assets.

You can find a list of assets in your Organization in the Asset Catalog page of Observe.

Adding assets to Observe​

After your DAGs run at least once and datasets have at least one dataset event, Observe automatically stores information about the tasks, datasets, warehouse tables, and local files you use in your DAGs as Assets. You can then find these assets in the Asset Catalog.

Access asset information​

You can access detailed information about a specific asset by looking it up in the Asset Catalog.

  1. Open Observe in the Astro UI. Then click Asset Catalog.

  2. Select a specific asset from the list shown in the Asset Catalog.

After you open a specific asset, you can see detailed information about the Asset and its performance in each tab.

Event timeline​

The event timeline view allows you to see a record of events associated with your asset. These are categorized into Success, Neutral, and Failure events.

An asset timeline showing success or failure and the task duration.

Success events​

  • SLA Success
  • Task Success
  • Task Start

Neutral events​

  • Airflow dataset write
  • OpenLineage dataset write

Failure events​

  • Alert notification
  • SLA Breach
  • Task failure

Graph preview​

A graph of the lineage metadata generated from your Asset or DAG. This graph shows the upstream or downstream assets relative to the asset you want to examine. The following example shows how the current_astronauts dataset is updated by a task inside the example_astronauts DAG.

A diagram of upstream assets, displayed as a dynamic graph.

Metrics​

Assets in Observe have their own metrics. Airflow task assets report the following metrics by default:

  • Task Retries
  • Task Failures
  • Task Duration
tip

You can also access asset metrics from the Data Products details view. Open your data product, then click the Assets tab. Select your asset from the list.

Graphical representation of the events associated with the asset.

Details​

The Details tab includes metadata and information about your asset, organized in the following sections:

  • Basic: information about Asset metadata
  • Task metadata (Airflow tasks only): Specific task metadata, such as the DAG ID, owner, operator class, and task arguments
  • Source: The last accessor of the asset

Was this page helpful?