Speaker: James Tarala
Event: SANS Webcast
Date: November 10, 2022
Watch on YouTube: https://www.youtube.com/watch?v=ADLfKTanxCM
Introduction
The Cybersecurity Standards Scorecard provides an annual review of leading security frameworks, assessing their effectiveness, scope, and applicability. James Tarala, a senior instructor at SANS, presents the 2022 edition, offering insights into how cybersecurity standards compare and which are most relevant for organizations today.
This webcast builds on the 2021 Scorecard, evaluating new versions of major security frameworks, including ISO 27002:2022, PCI DSS 4.0, and CIS Controls Version 8. Tarala provides a quantitative and qualitative comparison of each standard to help organizations make more informed decisions about their security programs.
Key Takeaways
- Cybersecurity standards vary widely in scope and effectiveness. Not all frameworks are equally useful for every organization.
- Recent updates to standards show a shift toward governance and risk management. Frameworks like ISO 27002:2022 and CIS Controls v8 place more emphasis on policy and governance.
- Technical security controls are being streamlined. New versions of security standards are reducing the number of prescriptive technical controls.
- Mapping threats to controls remains a gap. Most frameworks do not explicitly map security controls to real-world cyber threats.
- Prioritization of security controls is disappearing. Organizations are now responsible for determining which controls to implement first.
The Growing Complexity of Cybersecurity Standards
The cybersecurity landscape has seen an explosion of standards, with 50-100 widely used frameworks in existence today. Many organizations struggle to choose which standard to follow, often selecting frameworks based on popularity rather than effectiveness.
James Tarala challenges the belief that all cybersecurity standards are essentially the same. His research shows that frameworks have distinct strengths and weaknesses, making it essential for organizations to evaluate them based on their specific needs.
Evaluation Criteria
James Tarala’s methodology for evaluating standards includes:
- Governance, Operational, and Technical Controls – Does the framework balance security policy, implementation, and technical defenses?
- Recent Updates – Is the framework actively maintained to address modern threats?
- Community-Driven Development – Can organizations provide feedback and contribute to improvements?
- Threat Mapping – Does the standard link security controls to known cyber threats?
- Applicability to IT Environments – Can the framework be applied to cloud security, SaaS, industrial control systems, and DevOps?
- Prioritization of Controls – Does the standard provide guidance on which controls to implement first for maximum impact?
- Metrics and Measurement Guides – Does it include tools to evaluate implementation effectiveness?
Each standard is graded using a five-point system, with letter grades assigned based on performance across these categories.
Notable Framework Comparisons
CIS Controls (Version 7.1 & 8)
- Strengths: Strong focus on technical security measures, clear actionable guidelines.
- Weaknesses: Limited coverage of governance and privacy.
- Changes in Version 8: More focus on policy-driven controls, fewer prescriptive technical controls.
- Final Score: B+ (Version 7.1), B (Version 8 due to missing supplementary guidance at the time of evaluation).
NIST Cybersecurity Framework (CSF)
- Strengths: Well-regarded for governance and risk management.
- Weaknesses: Lacks clear technical security measures.
- Adoption Trend: Slight increase in interest as discussions around CSF Version 2 gain traction.
- Final Score: C+
Cybersecurity Maturity Model Certification (CMMC)
- Strengths: Designed for government contractors, integrates NIST 800-171.
- Weaknesses: Ongoing regulatory changes, uncertainty about certification requirements.
- Adoption Trend: Initial excitement has cooled as certification requirements evolve.
- Final Score: B-
ISO 27002:2022
- Strengths: Strong governance and compliance focus, best for regulatory alignment.
- Weaknesses: Fewer technical security guidelines, some outdated concepts.
- Improvement Over 2013 Version: More emphasis on identity management and software security.
- Final Score: B-
PCI DSS 4.0
- Strengths: Clear compliance requirements for financial data security.
- Weaknesses: Still compliance-driven, minimal focus on emerging threats.
- Final Score: C+
HIPAA (U.S. Healthcare Security Rule)
- Strengths: Foundational regulation for healthcare security.
- Weaknesses: Outdated and vague, provides only basic security hygiene.
- Final Score: D+
COBIT (ISACA)
- Strengths: Governance-heavy framework, strong IT risk management principles.
- Weaknesses: Lacks prescriptive technical controls, difficult to implement as a standalone security program.
- Final Score: C
MITRE ATT&CK & Enterprise Mitigations
- Strengths: Explicitly maps controls to real-world threats, strong for technical defense.
- Weaknesses: Not a full security framework—designed for reference rather than policy implementation.
- Final Score: B
Collective Controls Catalog (Research Initiative)
- Strengths: Aggregates over 40 security frameworks into a comprehensive control baseline.
- Weaknesses: Still new, not widely adopted outside of research communities.
- Final Score: A-
Actionable Insights
- Choose a framework based on your needs, not just compliance. Not all frameworks are equally effective for every organization.
- Balance governance, operational, and technical security. The best security programs integrate all three.
- Demand better threat mapping in security frameworks. Organizations should push for explicit control-to-threat alignment.
- Use prioritization to drive security investments. Organizations should focus first on high-impact controls.
- Continuously update security strategies. The threat landscape evolves, and security frameworks must evolve with it.
Conclusion
The Cybersecurity Standards Scorecard highlights the diverse approaches to security frameworks and the importance of choosing the right one for your organization.
Rather than defaulting to compliance mandates, security leaders should prioritize effectiveness, governance, and technical security measures to build resilient security programs.
By understanding the strengths and weaknesses of cybersecurity standards, organizations can align their security programs with business objectives and real-world risks.
For more insights on this topic, watch the full webcast here.