事件驱动 CDC（变更数据捕获）架构

Architecture

变更数据捕获架构图，使用 Debezium 读取数据库事务日志并将变更事件发布到 Kafka，供下游消费者更新搜索索引、使缓存失效、加载数据仓库和写入审计日志。该模板展示 CDC 如何通过在数据库级别捕获变更而无需修改应用代码来消除双写问题。对于异构系统间的数据同步至关重要。

下载 .fz 文件

完整 FlowZap 代码

SourceDB { # Source Database
n1: circle label:"Data Write Operation"
n2: rectangle label:"Transaction Log (WAL)"
n3: rectangle label:"Commit to Database"
n1.handle(right) -> n2.handle(left)
n2.handle(right) -> n3.handle(left) [label="Persist"]
n2.handle(bottom) -> CDC.n4.handle(top) [label="Log Tail"]
}
CDC { # CDC Engine (Debezium)
n4: rectangle label:"Read Transaction Log"
n5: rectangle label:"Detect Change Event"
n6: rectangle label:"Serialize to Avro/JSON"
n7: diamond label:"Schema Changed?"
n8: rectangle label:"Update Schema Registry"
n9: rectangle label:"Publish Change Event"
n4.handle(right) -> n5.handle(left) [label="INSERT/UPDATE/DELETE"]
n5.handle(right) -> n6.handle(left) [label="Before + After"]
n6.handle(right) -> n7.handle(left)
n7.handle(right) -> n9.handle(left) [label="No"]
n7.handle(bottom) -> n8.handle(top) [label="Yes"]
n8.handle(right) -> n9.handle(left) [label="Registered"]
n9.handle(bottom) -> Consumers.n10.handle(top) [label="Kafka Topic"]
}
Consumers { # Downstream Consumers
n10: rectangle label:"Search Index Updater"
n11: rectangle label:"Cache Invalidator"
n12: rectangle label:"Data Warehouse Loader"
n13: rectangle label:"Audit Log Writer"
n10.handle(right) -> n11.handle(left) [label="Elasticsearch"]
n12.handle(right) -> n13.handle(left) [label="Snowflake"]
}
TargetDB { # Target Systems
n14: rectangle label:"Elasticsearch Index"
n15: rectangle label:"Redis Cache"
n16: rectangle label:"Data Warehouse"
n17: rectangle label:"Audit Database"
n10.handle(bottom) -> TargetDB.n14.handle(top) [label="Upsert Doc"]
n11.handle(bottom) -> TargetDB.n15.handle(top) [label="Invalidate"]
n12.handle(bottom) -> TargetDB.n16.handle(top) [label="Load"]
n13.handle(bottom) -> TargetDB.n17.handle(top) [label="Append"]
}

为什么需要这个工作流？

Application-level dual writes (updating a database AND publishing an event) are inherently unreliable—if either fails, data becomes inconsistent. CDC captures changes directly from the database transaction log, guaranteeing that every committed write is captured and published without modifying application code.

工作原理

Step 1: Application writes data to the source database as normal.
Step 2: Debezium tails the database transaction log (WAL) to detect changes.
Step 3: Each INSERT, UPDATE, or DELETE is captured as a change event with before/after state.
Step 4: Events are serialized and published to Kafka topics.
Step 5: Downstream consumers update search indexes, invalidate caches, and load data warehouses.
Step 6: Schema changes are detected and registered in the Schema Registry.

替代方案

Application-level event publishing requires code changes and risks dual-write inconsistency. Polling-based CDC is simpler but adds latency. Debezium with Kafka Connect provides the most reliable log-based CDC. This template shows the complete CDC pipeline.

Key Facts

Template Name	事件驱动 CDC（变更数据捕获）架构
Category	Architecture
Steps	6 workflow steps
Format	FlowZap Code (.fz file)