Asia-24-Stott-The-Fault-in-Our-Metrics.pdf

1、The Fault in Our MetricsRethinking How We Measure Detection&ResponseBoD meeting is coming up.Gonna need updated program metrics.Lets chat tomorrowyou got it bossBossmanTeam ChatBoD metrics.what have we presented in the past?oh nobad news,our last manager pretty much just made those upgood news,youre

2、 here and gonna do so much better;-)detection response metricsWhy should I care about metrics?Metrics:You Are What You Measure!Hauser&KatzThat which is measured,improves Karl PearsonMetrics reveal data.Edward TufteMetrics are an annoying powerpoint I need to update every month.AllynHi Im AllynIve ma

3、de mistakes.5 Terrible Mistakes Ive Made When Creating MetricsLosing Sight of the GoalMistake#1JanFebMarAprMayJunJulAugSepOctNovDecJanFebMarfor illustrative purposes onlySecurity AlertsJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarTPFPfor illustrative purposes onlySecurity AlertsWhat do I measure?Vig

4、ilanceAwarenessExplorationReadinessStreamlinedSAVER CategoriesVigilanceAwarenessExplorationReadinessStreamlinedOperational efficiency accuracy,and automation.Context and intelligence about existing and emerging threats,vulnerabilities,and risks.Visibility and detection coverage for known threats.Pro

5、active hunts and investigations into the unknown.Preparation for the next big incident.JanFebMarAprMayJunJulAugSepOctNovDecJanFebMarTPFPfor illustrative purposes onlySecurity Alertsfor illustrative purposes onlyJan Mar May Jul Sep Nov Jan MarTPFPSecurity AlertsOutcome Theres always time and effort f

6、or TPs Question Are we spending time investigating impactful alerts?Category Streamlined Metric control Alert tuning Risk reward Over-tuning alerts,prioritizing based on volume Data requirements Alert resolution Effort cost Medium Metric cost Low Metric expiration Automation and enforced detection q