掌握 Unity Catalog：Zalando 采用之旅的经验教训.pdf

1、2024 Databricks Inc.All rights reserved1Mastering Unity Mastering Unity CatalogCatalogLessons from Zalandos Adoption JourneySebastian HeroldSebastian Herold20242024-0606-11112024 Databricks Inc.All rights reserved#Sr.Principal Engineer Zalando#6y Zalando#ML Productivity#Streaming DWH#Data Platform#7

2、y (Immo-)Scout24#DataDevOps Manifesto#Data Platform2WHO AM I?WHO AM I?Sebastian HeroldSebastian Herold#Linkedin2024 Databricks Inc.All rights reserved3This is ZalandoThis is ZalandoWe are the leading multiWe are the leading multi-brand brand fashionfashiondestination in Europe14bn Euro GMVactive cus

3、tomersaround 50m20082008-20092009201020102011201120122012-201320132018201820212021202220222024 Databricks Inc.All rights reserved 950 Monthly Active Users 300 Teams 15,000 Jobs per day4DATABRICKS ADOPTIONDATABRICKS ADOPTIONZalandoZalando2024 Databricks Inc.All rights reserved LakehouseIQ LLM Trainin

4、g Description generation Serverless Databricks SQL Warehouse System Tables5Center of Innovation DatabricksCenter of Innovation DatabricksUNITY CATALOG UNITY CATALOG-Our Motivation Our Motivation 2024 Databricks Inc.All rights reserved Instead of 200(see later)More transparency across teams Better da

5、ta discovery Data sharing based on tables,not S3 paths6One Central Technical Data CatalogOne Central Technical Data CatalogUNITY CATALOG UNITY CATALOG-Our Motivation Our Motivation 2024 Databricks Inc.All rights reserved Column-level Security Row-level Security Automatic Access Provisioning Integrat

6、ion with our Scalable Data Access Control program7FineFine-grained Data Access Controlgrained Data Access ControlUNITY CATALOG UNITY CATALOG-Our Motivation Our Motivation 2024 Databricks Inc.All rights reserved8EXCURSION EXCURSION-Scalable Data Access Scalable Data Access Control(SDAC)Control(SDAC)T