1、 0 Anna Knack,Richard J.Carter and Alexander Babuta Human-Machine Teaming in Intelligence Analysis Requirements for developing trust in machine learning systems December 2022 Anna Knack,Richard J.Carter and Alexander Babuta 1 About CETaS.2 Acknowledgements.2 Executive Summary.3 1.Introduction.7 1.1

2、The black box problem.7 1.2 Implications of the black box problem for analytic tradecraft.9 1.3 Research aims and objectives.12 1.4 Methodology.13 2.Cross-cutting Findings.15 2.1 The importance of context.15 2.2 Embedding ML into the intelligence analysis pipeline.17 3.Technical Considerations.20 3.

3、1 Determining thresholds:false positives and false negatives.20 3.2 Different explanations for different audiences.21 3.2.1.Explanations for the senior responsible owner.21 3.2.2.Explanations for analysts.23 3.2.3.Explanations for oversight bodies.24 3.3 Factors influencing the required granularity

4、of explanation.26 4.Management Processes and Structures.28 4.1 Diversity of thought.28 4.2 Standardisation of terminology and explainability practices.29 4.3 The importance of user testing.30 4.4 Resource requirements.30 4.5 Training and support.31 5.Further Research.33 5.1 Human-machine teaming.33

5、5.2 Automated decision-making.34 5.3 Human-centred engineering of ML/AI applications.34 5.4 Technical requirements for users from different backgrounds.35 About the Authors.36 Human-Machine Teaming in Intelligence Analysis 2 About CETaS The Centre for Emerging Technology and Security(CETaS)is a poli

6、cy research centre based at The Alan Turing Institute,the UKs national institute for data science and artificial intelligence.The Centres mission is to inform UK security policy through evidence-based,interdisciplinary research on emerging technology issues.Connect with CETaS at cetas.turing.ac.uk.T