Introduction

The Pipeline Cost Analysis Agent helps teams track and optimize data pipeline spend.

Tools & Integrations

• BigQuery – SQL execution and runtime insights

• dbt – Models and transformations

• Airflow – DAG orchestration and pipeline lineage

• DataHub / Redash – Metadata and dashboards

• GitHub – Source control & pull request workflow

• ACP API – Deployment mechanism for Chicory agents

---

Problem: Cloud data warehouse queries can be costly. Without attribution, it’s unclear which dbt models or stages drive spend.

What You’ll Build

An agent that:

• Collects BigQuery job costs from billing exports

• Attributes costs to dbt models/stages

• Aggregates total per pipeline run

• Stores history in a BigQuery table

• Publishes results to a Redash dashboard

---