XCom (EN)

Definition

XCom (Cross-communications) is a mechanism in Apache Airflow that enables the exchange of small data amounts between tasks within a DAG. With XCom, tasks can pass information to other tasks without needing to write temporary files. This improves performance and simplifies communication between tasks. XCom is particularly suitable for exchanging configuration parameters or small result sets between tasks.

Architecture

flowchart TD     A[DAG] --> B[Task 1]     A --> C[Task 2]     A --> D[Task 3]     B --> E[XCom Backend]     C --> E     D --> E     E --> F[Task 1]     E --> G[Task 2]     E --> H[Task 3]          subgraph XCom Backend         direction LR         I[Key-Value Store]         J[Metadaten]     end          B -- "push(key, value)" --> I     C -- "push(key, value)" --> I     D -- "push(key, value)" --> I     F -- "pull(key)" --> I     G -- "pull(key)" --> I     H -- "pull(key)" --> I 

In Context

• Typically used together with DAGs (Directed Acyclic Graphs) in Apache Airflow
• Related to: Apache Airflow, DAG, Task, Data pipelines
• Example use case: Passing calculated parameters from one task to a subsequent task without file I/O