Spark 规模优化：为 LinkedIn 节省数千小时的计算时间.pdf

1、Spark Right-SizingSaving Thousands of PBHrs of Compute at LinkedIn05/01/2025Shreyesh ArangathForward-looking StatementThis presentation has been prepared for informational purposes only.The information set forth herein does not purport to be complete or contain all relevant information.Statements co

2、ntained herein are made as of the date of this presentation unless stated otherwise.This presentation and the accompanying oral commentary may contain forward-looking statements.In some cases,forward-looking statements can be identified by terms such as“may”,“will”,“should”,“expects”,“plans”,“antici

3、pates”,“could”,“intends”,“projects”,“believes”,“estimates”,“predicts”,or“continue”,or the negative of these words or other similar terms or expressions that concern Databricks expectations,strategy,plans,or intentions.Forward-looking statements are based on information available at the time those st

4、atements are made and are inherently subject to risks and uncertainties that could cause actual results to differ materially from those expressed in or suggested by the forward-looking statements.Forward-looking statements should not be read as a guarantee of future performance or outcomes.Except as

5、 required by law,Databricks does not undertake any obligation to publicly update or revise any forward-looking statement,whether as a result of new information,future developments or otherwise.3About meShreyesh Arangath Senior Software EngineerApache Spark team LinkedIn Focused on resource optimizat

6、ion,job submission,and user experience efforts Something that is not on my LinkedIn profile:I like lifting,and jiu jitsuMotivationSpark at LinkedInLinkedIn relies heavily on Apache Spark for offline analytics,powering data warehousing,A/B testing,AI/ML,and more.Scale500,000+executions daily150,000+u