Introduction
The rapid deployment of AI systems across critical sectors has prompted governments worldwide to develop regulatory frameworks that address the unique risks these technologies pose. For security engineers, understanding the regulatory landscape is no longer optional—compliance requirements directly shape how AI systems must be designed, tested, deployed, and monitored.
This section examines the major regulatory frameworks governing AI security, from the comprehensive EU AI Act to US-specific guidance from NIST, executive orders, and CISA. Each framework takes a different approach, but they share common themes: risk-based classification, transparency requirements, human oversight mandates, and accountability mechanisms.
The EU AI Act (2024)
The EU AI Act, which entered into force in 2024, is the world's first comprehensive AI regulation. It establishes a risk-based framework that classifies AI systems into four categories: unacceptable risk (banned), high risk (heavily regulated), limited risk (transparency obligations), and minimal risk (largely unregulated).
For cybersecurity applications, the high-risk category is most relevant. AI systems used in critical infrastructure protection, law enforcement, and border security are classified as high risk and must meet stringent requirements including risk management systems, data governance, technical documentation, human oversight, and robustness against adversarial attacks.
- Risk management: High-risk AI systems must implement a continuous risk management system throughout the entire lifecycle
- Data quality: Training, validation, and testing datasets must meet quality criteria including relevance, representativeness, and freedom from errors
- Technical documentation: Comprehensive documentation of system design, development process, and testing results must be maintained
- Robustness: High-risk systems must be resilient against attempts by unauthorized third parties to alter their use or performance through exploitation of vulnerabilities
Key Insight: The EU AI Act explicitly requires robustness against adversarial attacks for high-risk AI systems. This regulatory mandate transforms adversarial ML defense from a best practice into a legal requirement for organizations operating in or serving the European market.
NIST AI RMF and US Policy
The NIST AI Risk Management Framework (AI RMF) provides a voluntary, flexible framework for managing AI risks. Unlike the EU AI Act's prescriptive approach, the NIST AI RMF offers principles-based guidance organized around four core functions: Govern, Map, Measure, and Manage.
US Executive Order 14110, issued in October 2023, directed federal agencies to develop AI safety and security standards. The order mandated red-teaming requirements for the most powerful AI models, established reporting requirements for companies developing frontier models, and directed NIST to develop guidelines for AI system evaluation and auditing.
- Govern: Establish organizational governance structures, policies, and processes for AI risk management
- Map: Identify and categorize AI risks by understanding the context, capabilities, and intended uses of AI systems
- Measure: Assess, analyze, and track identified AI risks using quantitative and qualitative methods
- Manage: Prioritize and act on AI risks, implementing mitigations and monitoring their effectiveness
The NIST framework has become the de facto standard for AI risk management in the US, and its principles are increasingly referenced by industry standards bodies, insurance companies, and procurement requirements. Security engineers should be familiar with its structure even if compliance is not legally mandated.
CISA AI Guidance and GDPR Intersections
The Cybersecurity and Infrastructure Security Agency (CISA) has published specific guidance on securing AI systems, emphasizing the need to apply existing cybersecurity best practices to AI deployments while also addressing AI-specific vulnerabilities. CISA's guidance covers secure development practices, supply chain risk management for AI components, and incident response planning for AI-related security events.
GDPR intersects with AI security in critical ways. AI systems that process personal data must comply with GDPR principles including data minimization, purpose limitation, and the right to explanation. Membership inference attacks (discussed in Chapter 13) are not just security threats but potential GDPR compliance violations, as they can reveal whether specific individuals' data was used in training without consent.
The right to erasure presents particular challenges for AI systems. When an individual requests deletion of their data under GDPR, the organization must consider whether the data has been "learned" by a model—and if so, whether the model itself must be retrained. This intersection of data protection law and machine learning practice remains an area of active regulatory development.
Why This Matters: AI governance is not a single regulation but an interconnected web of requirements spanning AI-specific laws, existing data protection regulations, sector-specific rules, and evolving standards. Security engineers who understand this landscape can design AI systems that meet compliance requirements by default, rather than retrofitting controls after deployment.