揭秘 Data Lakehouse 数据建模 101.pdf

1、Data Modeling 101 for Data Data Modeling 101 for Data Lakehouse DemystifiedLakehouse DemystifiedLuan Moreno M.MacielLuan Moreno M.MacielTuesday,June 10th10:20 AM-11:00 AM PDTTaming The Chaos:Practical Data Modeling for Data LakehouseAbout MeAbout MeLuan Moreno M.Maciel Lead Data Engineer at Pythian

2、Spark&Databricks Expert Microsoft&Databricks MVP Databricks Product Advisory BoardYou currently have a Medallion Architecture(Bronze,Silver,Gold)implemented in your organization?You have Gold tables in your lakehouse that youre not sure anyone actually uses?Youve ever debated whether something shoul

3、d go in Silver or Gold based on whether its operational or analytical data?The FoundationThe FoundationIn God we trust.All others must bring data.W.Edwards DemingData Lakehouse 101:The FundamentalsData Lakehouse 101:The FundamentalsThe Bronze LayerThe Bronze LayerIn Bronze,we preserve the raw truth.

4、Every byte,every timestamp,every detail-because you never know which data point will unlock tomorrows breakthrough.The Bronze LayerThe Bronze LayerThe Silver LayerThe Silver LayerIn Silver,we transform chaos into clarity.This is where disparate data sources unite to tell the real story of your enter

5、prise.The Silver LayerThe Silver LayerThe Gold LayerThe Gold LayerIn Gold,we refine information into rocket fuel.Purpose-built,performance-optimized,and ready to power critical business decisions at the speed of thought.The Gold LayerThe Gold LayerThe Universal StruggleThe Universal StruggleWe built

6、 it by the book.We have all three layers.So why isnt it working?The answer isnt in the patterns.Its in yourmaturity stage.Stage 1:CopyStage 1:Copy-Paste ArchitecturePaste Architecture“The Starting Point(and Trap)for Many Data Lakehouse Journeys”Stage 1:CopyStage 1:Copy-Paste ArchitecturePaste Archit