无缝流：使用 DLT 从批处理数据流演变为流数据流.pdf

1、2024 Databricks Inc.All rights reserved1Seamless Flow:Seamless Flow:Evolving from Evolving from Batch to Batch to Streaming Data Streaming Data Flows using DLTFlows using DLTScott Gordon,LScott Gordon,Le ead Data Engineer at 84.51 ad Data Engineer at 84.51 Alli Hanlon,Data Engineer at 84.51 Alli Han

2、lon,Data Engineer at 84.51 2024 Databricks Inc.All rights reserved2Introductions Introductions Scott GordonScott GordonLead Data Engineer at 84.51Lead Data Engineer at 84.51Alli HanlonAlli HanlonData Engineer at 84.51Data Engineer at 84.512024 Databricks Inc.All rights reserved3AgendaAgendaAbout 84.

3、51 Migrating from On-prem Airflow to Azure Databricks DLTSeamless evolution from batch to streaming with DLT2024 Databricks Inc.All rights reserved4About 84.51About 84.51“We are a retail data science,insights and media company.We help The Kroger Co.,consumer packaged goods companies,agencies,publish

4、ers and affiliates create more personalized and valuable experiences for shoppers across the path to purchase.Powered by cutting-edge science,we utilize first-party retail data from over 62 million U.S.households sourced by the Kroger Plus loyalty program to fuel a more customer-centric journey usin

5、g 84.51 Insights,84.51 Loyalty Marketing and our retail media advertising solution,Kroger Precision Marketing.”2024 Databricks Inc.All rights reserved5Feature:Once FlowFeature:Once FlowSets apply_changes()to run only one time on a static source2024 Databricks Inc.All rights reserved6Use case:Once Fl

6、owUse case:Once FlowMigrate legacy data flow from on-prem Hadoop to Azure Databricksutilizing apply_changes()and Once Flow functionality2024 Databricks Inc.All rights reserved7Online OrdersOnline Orders2024 Databricks Inc.All rights reserved8Sample DataSample DataididstatusstatusshoppingContextshopp