Lakehouse 到 LLM：使用 LLM 可靠、大规模地解决业务任务.pdf

1、2024 Databricks Inc.All rights reservedLakehouse to Lakehouse to LLM:Automating LLM:Automating Knowledge work Knowledge work at scaleat scaleAlberto Rizzoli Alberto Rizzoli CEO,V7CEO,V7June 2024June 2024122024 Databricks Inc.All rights reserved2018202020222024LabelingAutomatingWhat AI leadership sho

2、uld spend time on32024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved4What do you call it What do you call it when employees when employees produce ground produce ground truth?truth?2024 Databricks Inc.All rights reservedHow do we capture knowledge work and bring it into L

3、LMs?56An“AGI”DatabaseDesigning for knowledge work automationChapter 1740%of knowledge work can be done by well-trained frontier models just as well.So whats missing?Why arent we doing it yet?8Format supportMust adopt text,images,audio calls,documents,natural language instructions.Designing to adapt

4、to any form of knowledge workDiscrete decisionsMust convert the output of LLMs into discrete options,easier to set up than function calls.Reliability at ScaleAfter completing 1,000 reviewed tasks it must succeed and fail reliably.9Compounding improvementsInvesting in AI should improve the business,n

5、ot its model provider.What do companies care about?Freedom to buildNot be locked into restrictive domain-specific SaaS or long model implementations.Choice of modelIf the company already invested into one particular provider.101112Discrete Decisions1314Choice of Model1516171819Reliability at Scale20

6、21PromptingStart with thisFine TuningIf you really have to keep trying222324Context System Prompt-Avoid preamble,format this wayTask Prompt-Does the liability clause raise any issues?Inputs-Target document,guidelines documentGround truth samples-10m liability