CRF

LOGIN

Architecting Data Analytics for Continuous Risk Management

CRF Web Assessment Tool

Speaker: James Tarala
Event: RSAC 2025
Date: May 1, 2025
Watch on YouTube: https://www.youtube.com/watch?v=Q2-39LjjKnQ&t=1s

Introduction

At RSA Conference 2024, cybersecurity veteran James Tarala returned to explore one of today’s most urgent but misunderstood topics: how to architect meaningful data analytics and business intelligence programs that support continuous cybersecurity risk management. In his characteristically practical and conversational style, Tarala walked attendees through a roadmap for transforming raw telemetry and vendor dashboards into insights that drive leadership-level decisions.

While many organizations aspire to provide clean, quantifiable dashboards to demonstrate risk reduction, few have laid the groundwork to make those dashboards meaningful. This talk provided a thoughtful framework to help teams mature their analytics efforts, reduce uncertainty, and support their organizations’ operational goals.

 

Key Takeaways

  1. Most Organizations Want Dashboards—But Aren’t Ready for Them
    The common request from business leaders is: “Give me a dashboard that shows improvement.” But many teams are not prepared to produce actionable insights. Dashboards often start with basic metrics like phishing click rates, which may be easy to collect but lack business relevance unless tied to decision-making.
  2. Measurement Should Reduce Uncertainty, Not Just Count Things
    Tarala highlighted Doug Hubbard’s advice that metrics should “quantitatively reduce uncertainty” for better decision-making. Metrics without context or follow-through waste time and resources.
  3. Risk Management Is About Supporting Business Objectives
    The goal of cybersecurity isn’t security for its own sake. It’s ensuring that technology systems do what they were designed to do. Risk management should align with mission enablement, not only threat reduction.
  4. CRF GRC Roadmap as a Maturity Model
    Tarala introduced the Cybersecurity Risk Foundation’s 7-step GRC roadmap:
  • Initiate the program
  • Inventory systems and data
  • Select safeguards
  • Educate the organization
  • Implement controls
  • Validate implementation
  • Communicate status

This roadmap ensures measurement happens only after solid foundational work has been done.

  1. Introducing the CRF Business Intelligence Model
    The session offered a detailed walkthrough of the CRF Business Intelligence Model—a practical framework for:
  • Selecting safeguards based on legal, contractual, and best-practice obligations
  • Mapping safeguards to technology systems
  • Identifying existing tools and their data collection capabilities
  • Aggregating data (via APIs or exports)
  • Analyzing and normalizing that data for key stakeholders
  • Creating reports that support specific decision-making use cases
  1. Don’t Skip Steps—Crawl Before You Run
    The desire to jump straight to KPIs and dashboards can backfire. Teams must first define what they are trying to achieve, what safeguards are in place, and what data supports meaningful measurement.
  2. Actionable Metrics Beat Vanity Metrics
    Phishing click rates, firewall alerts, and funnel charts may look impressive but offer little if they don’t guide next steps. Start instead with operational coverage metrics like endpoint agent status—something that can be improved through real action.
  3. Use What You Already Have
    Start small with tools like Excel or Power BI. Tarala outlined how to create a “bad agent” report using available API integrations from platforms like CrowdStrike or Defender. This report can show agent coverage across compute assets and provide Help Desk teams with tangible action items.
  4. Automate Where It Makes Sense
    While many organizations rush to buy CMDBs or GRC platforms, success depends on using the tools—regardless of brand. Simpler tools like runZero, Axonius, or Balbix may offer faster wins with lower overhead.
  5. Define Goals and Audience for Reporting
    Executives and boards need simplified, aggregated summaries that reflect business risk. Technical teams need actionable detail. Use a red/green light system, Sigma thresholds, or project timelines to show progress toward clearly defined objectives.

Final Thoughts

The promise of business intelligence for cybersecurity isn’t in prettier dashboards—it’s in data that leads to better, faster, and more defensible decisions. Tarala emphasized that effective data analytics starts with defining goals, mapping safeguards, and understanding how existing tools can support risk reduction.

If you’re overwhelmed by the idea of building a continuous monitoring program, start with one measure. Start with EDR coverage. Start with something you can track, validate, and improve.

Resources

  • Watch the webcast: Architecting Data Analytics for Continuous Risk Management
  • Explore the CRF Business Intelligence Model: org
  • Get free policy templates: SANS Security Policy Project
  • Suggested Books:
    • How to Measure Anything in Cybersecurity Risk by Hubbard & Seiersen
    • Storytelling with Data by Cole Nussbaumer Knaflic
    • The Information Diet by Clay Johnson

If you’d like help implementing this model or customizing your dashboards, reach out to James Tarala or explore additional CRF resources at crfsecure.org.