Pinterest 规模化：应对 PB 级挑战的冰山解决方案.pdf

《Pinterest 规模化：应对 PB 级挑战的冰山解决方案.pdf》由会员分享，可在线阅读，更多相关《Pinterest 规模化：应对 PB 级挑战的冰山解决方案.pdf（33页珍藏版）》请在三个皮匠报告上搜索。

1、 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Scaling Pinterest:Iceberg solutions for petabyte-scale challengesAshish SinghPrincipal Engineer,PinterestS T G 2 1 1Sachin HollaPrincipal Solutions Architect,AWS 202

2、5,Amazon Web Services,Inc.or its affiliates.All rights reserved.AgendaScale at PinterestTable format evolution at PinterestApache Iceberg adoptionApache Iceberg use casesAmazon S3 specific learnings 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or it

3、s affiliates.All rights reserved.Scale atPinterest600M+Monthly Active Users1.5B+Pins Saved Weekly10B+Boards500+PBsData size on Amazon S3100KTables in HMS20K+Spark/hadoop nodes1KTrino nodes400K+Daily compute jobs 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Servi

4、ces,Inc.or its affiliates.All rights reserved.Table Format Evolution at Pinterest2019Hive Table FormatDe Facto standard for Big Data Table Format2020Alternate Table FormatsAlternate table formats like Iceberg,Hudi and Delta started to gain traction to alleviate issues with Hive Table Format2022Icebe

5、rg Build OutTable format evaluations,design and implementation2023Iceberg ProductionizedData deletion use cases2025Ramped up usage 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Apache Iceberg Adoption 2025,Amazon

6、 Web Services,Inc.or its affiliates.All rights reserved.15K+Number of Iceberg tables200+PBsVolume of data under Iceberg tablesIceberg at Pinterest by numbers 2025,Amazon Web Services,Inc.or its affiliates.All rights reserved.Growth of Iceberg tables at PinterestTable count:324%YoYData volume:122%YoY