告别集群等待时间：你好闪电般的 r py 库加载！.pdf

1、2024 Databricks Inc.All rights reservedFaster Library LoadsFaster Library LoadsMay MerkleMay Merkle-TanTan20242024-0606-1111Goodbye to Cluster Wait Time:Goodbye to Cluster Wait Time:Hello LightningHello Lightning-fast R/fast R/PyPy Library Loads!Library Loads!2024 Databricks Inc.All rights reserved2

2、024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedThere is another There is another way way 2024 Databricks Inc.All rights reserveddemo/videodemo/video2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedSecre

3、t Sauce?Secret Sauce?2024 Databricks Inc.All rights reservedinit.shinit.sh script makes a symbolic link between userscript makes a symbolic link between user-defined defined and corresponding default R or Pythonand corresponding default R or Python ProfileProfile pathpathPrePre-install required proj

4、ect libraries install required project libraries to Cloud Storage:to Cloud Storage:R/Python_LIB_PATH_MOUNTEDR/Python_LIB_PATH_MOUNTED path on path on dbfsdbfs/mntmnt withwith Read OnlyRead Only accessaccessUser specified User specified/./.RprofileRprofileOR OR/./.ipythonipython/profile_demoprofile_d

5、emo/startup/startup/00_demo.py00_demo.pyDefaultDefault/root/./root/.RprofileRprofileOR OR/root/./root/.ipythonipython/profile_defaultprofile_default/startup/startup/.pypyAppend R/Python_LIB_PATH_MOUNTAppend R/Python_LIB_PATH_MOUNTto default library search path to default library search path for eith

6、er Rfor either R.libPathslibPaths()()or or PythonPython sys.pathsys.path()()Add corresponding VolumesAdd corresponding Volumes init.shinit.sh script to R/Python Clusters Advanced Optionsscript to R/Python Clusters Advanced Options2024 Databricks Inc.All rights