使用 Apache Iceberg 和 Amazon S3 构建经济高效的数据湖 [重复].pdf

1、 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Sofie ZilbermanShe/HerSr.Solution Architect Specialist,Streaming&AnalyticsAWSRan PergaminHe/HimSr

2、Solution Architect Specialist,Data&AIAWSBuilding cost-effective data lakes with Apache Iceberg and Amazon S3S T G 4 1 5-R 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Iceberg IkigaiOperationalsustainabilityEconomicefficiencyBusinessvalueTechnicalfitIkigai is the Japanese concep

3、t of finding purpose and fulfillment at the intersection of what you love,what youre good at,what the world needs,and what you can be paid for.2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Building a data lake on AWSStorageFile formatTable FormatEnginesMetadataGlue data catalogA

4、mazon EMRAmazon RedshiftUnity catalogHive meta storeAmazon S3Amazon Athena 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Apache Iceberg tables on AWSSelf-managed Iceberg tables in general purpose S3 bucketManaged Iceberg tables with Amazon S3 Tables 2025,Amazon Web Services,Inc.

5、or its affiliates.All rights reserved.AgendaInteraction patterns with Iceberg tablesDesign for performance and efficiency Maintaining and operating your tablesTakeaways 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.WriteRead&WriteCompact&OptimizeIngestionProcessingMaintenanceRea

6、dAnalyticsappend/upsert/deletes,batch or streamingETL,time-travel backfillsrewrite data/manifests,expire snapshots,remove orphans,S3 intelligent tiering metadata pruning,hidden partitioning,time travelPrimary interaction patterns with Iceberg tables 2025,Amazon Web Services,Inc.or its affiliates.All