在捷蓝航空构建安全且可扩展的 LLM 框架.pdf

1、2024 Databricks Inc.All rights reserved1Building a Building a secure&secure&scalable LLM scalable LLM frameworkframeworkSai Ravuru Sai Ravuru-JetBlue,JetBlue,NavindraNavindra Yadav Yadav-TheomTheomJun 12,2024Jun 12,20242024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved2Te

2、chnology StackTechnology Stack2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedPillars of a successful LLM stackPillars of a successful LLM stackEfficient and promptingOrganizational knowledge repositoryData and access governanceTrainingUser feedbackInternet accessCurate

3、d dataAccuracy testingRole based accessRegular auditsLoggingActive Directory integration2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedGenerative AI maturity curveGenerative AI maturity curveExpectationEnterprise documentsAccess governanceAutomation agentsExpectedRespo

4、nsePrompt templatePrompt templateExpectedResponseData governanceReality2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedGenerative AI scaleGenerative AI scale2024 Databricks Inc.All rights reserved6DsPyDsPy to the rescueto the rescue2024 Databricks Inc.All rights reserve

5、dPYTHONSample Python CodeSample Python Codeclass ToolRetrieverToolRetriever(dspy.Module):def _initinit_(self):self.generate_query=dspy.ChainOfThought(context,question-query)self.choose_tool=dspy.ChainOfThought(ToolChoice)self.generate_answer=dspy.ChainOfThought(GenerateAnswer)self.tools=answer_payro

6、ll_faq,irrelevant_contentdef irrelevant_contentirrelevant_content(self):return Ask something else.DsPyDsPy retriever initialization retriever initialization 72024 Databricks Inc.All rights reservedPYTHONSample Python CodeSample Python Codedef forwardforward(self,question):client=