Skip to main content

​
Overview

Sentiment analysis with an LLM works the same way as any other single-call use case: extract unstructured text, format it, call the model, get a score back. To be useful, the sentiment score gets combined with other indicators of customer happiness such as recent tickets or purchases. This architecture combines the output of two independent Dags into one report. A deterministic ETL pipeline fetches metrics. A separate AI pipeline scores unstructured customer signals for sentiment and aggregates them per customer. Once both Dags have finished, a downstream Dag combines the aggregated sentiment score with the deterministic metrics into a churn risk report, which is sent to customer support as a weekly summary email.
For general information on how to orchestrate LLM calls with Airflow, see LLM orchestration with Apache Airflow®.

​
Architecture

Sentiment analysis reference architecture diagram. A deterministic ETL pipeline extracts and normalizes product usage and telemetry metrics into a table. A second pipeline extracts Slack messages, customer tickets, and social media mentions, scores each for sentiment with a mapped @task.llm call, and weights the scores. A downstream LLM call combines both into a churn risk report sent to customer support.
This architecture consists of five main components:
  • Deterministic ETL pipeline: Extracts product usage and telemetry metrics, normalizes them, and writes them to a table.
  • Unstructured source data: The customer signals sentiment is scored from, such as Slack messages, support tickets, and social media mentions.
  • Sentiment scoring: One @task.llm call per signal, mapped over the list, returning a score for that individual piece of text as part of a structured output.
  • Weighting and aggregation: A deterministic task weights the individual scores and aggregates them into a single number per customer. The weighting math is defined as code, so the same inputs always produce the same aggregate and the formula is reviewable.
  • Report generation: A second LLM call combines the weighted sentiment score with the deterministic metrics into a natural language churn risk report, which is sent to customer support in a weekly summary email.

​
Airflow features

  • Conditional asset scheduling: The deterministic Dag emits an asset event when it writes its metrics table, the AI Dag emits one when its sentiment scores are ready, and the report Dag is scheduled to run once both assets have a new event. You can combine assets with logical conditions using & and | to fit your use case.
  • Dynamic task mapping: Sentiment scoring is mapped over the list of customer signals, so the task count follows how much feedback arrived that week.
  • @task.llm: Runs both the per-signal scoring and the report generation in separate tasks.
  • Structured output: The scoring call returns a score in a fixed schema.

​
Considerations

  • Aggregate the scores using deterministic code. Asking a model to combine twenty sentiment scores into one number gives you a different answer on each rerun, with low explainability.
  • Define how missing data is handled. A customer with no Slack messages and no tickets has no sentiment score at all. Determine how the sentiment part of the weighting task handles an input like this.
  • Score each signal on its own. One call per signal gives a distribution you can weight by recency, channel, or severity.
  • Treat the score as an input to a human decision. The output is a churn risk report for customer support to act on, and the pipeline takes no action by itself. If you add tasks that send information to the customer directly, you also need to add review steps as described in Review AI output.

​
Next steps