数据智能平台上的数据工程指南.pdf

1、2024 Databricks Inc.All rights reserved1Matt Jones,Sr.Product Marketing Manager,DatabricksRaju Mudunuri,Data Solutions&Delivery Sr.Leader,LexmarkYour Guide to Data Engineering on the Data Intelligence Platform2024 Databricks Inc.All rights reservedI.Strategic VisionII.Ingestion,Transformation,Orches

2、trationIII.Streaming and Real-Time ProcessingIV.Delta Lake,Unity Catalog,and Serverless V.Embedded GenAI and Data IntelligenceVI.Lexmarks DE Journey with DatabricksAgendaThis information is provided to outline Databricks general product direction and is for informational purposes only.Customers who

3、purchase Databricks services should make their purchase decisions relying solely upon services,features,and functions that are currently available.Unreleased features or functionality described in forward-looking statements are subject to change at Databricks discretion and may not be delivered as p

4、lanned or at allProduct safe harbor statement2024 Databricks Inc.All rights reserved4“Data problems arethe most likely factorto jeopardize ourAI/ML goals”MIT Technology ReviewInsights survey,2023By 2026,over 80%of enterprises will be using GenAI in production environments,up from less than 5%in 2023

5、2023 Gartner Hype Cycle for Generative AIAI initiatives aretop of mindbut they will failwithout reliable data2024 Databricks Inc.All rights reservedQuality ChecksGovernanceData DiscoveryIts difficult to build and operate data pipelines5your codeDependencyManagementDaily PartitionComputationCheckpoin

6、ting&RetriesBackfill HandlingVersion ControlDeployment InfrastructureData LakeOrchestration&ETLData WarehouseGovernanceStreamingBIData ScienceGenerativeAIMachine Learning2024 Databricks Inc.All rights reservedQuality ChecksGovernanceData DiscoveryIts difficult to build and operate data pipelines6you