HelloFresh 的 Apache Iceberg 与 Unity Catalog.pdf

1、Apache Iceberg with Unity Catalog at HelloFreshAdam Komisarek Max SchultzeData Engineering at HelloFreshApache Iceberg at HelloFreshUnity Catalog for Apache IcebergTable of contentCatalog Federation Director of Data Engineering at worlds leading meal kit company and global integrated food solutions

2、group Productionized large scale data infrastructure and data management platforms Data Mesh Advocate&Content Creator Data Master CoachMax Schultze Senior Staff Data Engineer at HelloFresh Expanding Data Platform offerings to internal users Experienced Software Engineer decided to pursue Data couple

3、 years agoAdam Komisarek6We operate eight different meal kit and ready-to-eat brands across our territories with ample room for further internationalisation7Where we stand in numbers7,661,300,000Annual revenue in FY 2024Over 12,000Recipes in our Database 20,000Employees WorldwideOperating in 18Count

4、ries#1Worldwide(1)965,500,000 Meals Delivered Worldwide in FY 2024Source:Technomic1:In terms of meals delivered per monthData Engineering at HelloFreshData Engineering at HelloFresh-Batch1.Understand SourcesData Engineering at HelloFresh-Batch1.Understand Sources2.Infrastructure Setup3.Build Data Pi

5、peline4.Orchestration5.Move to DWH6.Data Vault Modeling7.Data ActivationData Engineering at HelloFresh-Batch1.Understand Sources2.Infrastructure Setup3.Build Data Pipeline4.OrchestrationoptionalData Engineering at HelloFresh-Streaming1.Stream to Stream modificationsData Engineering at HelloFresh-Str

6、eaming1.Stream to Stream modifications2.Operational monitoring3.AnalyticsData Engineering at HelloFresh-Streaming1.Stream to Stream modifications2.Operational monitoringCurrent Challenges1.Permission management across enginesCurrent Challenges1.Permission management across engines2.Data management o