Group Purchasing
Group Purchasing
AI-FOCUSEDNEW

SEC536: Adversarial AI - Penetration Testing AI Systems

SEC536Offensive Operations, Artificial Intelligence
  • 3 Days (Instructor-Led)
  • 18 Hours (Self-Paced)
Course authored by:
Foster NethercottMick Douglas
Foster Nethercott & Mick Douglas
SEC536: Adversarial AI - Penetration Testing AI Systems
Course authored by:
Foster NethercottMick Douglas
Foster Nethercott & Mick Douglas
  • GIAC AI Penetration Tester (GAIPT)
  • 18 CPEs

    Apply your credits to renew your certifications

  • In-Person, Virtual or Self-Paced

    Attend a live, instructor-led class at a location near you or remotely, or train on your time over 4 months

  • Intermediate Skill Level

    Course material is geared for cyber security professionals with hands-on experience

  • 15 Hands-On Lab(s)

    Apply what you learn with hands-on exercises and labs

Learn the attacks real-world adversaries are executing against enterprise AI deployments spanning LLMs, RAG pipelines, ML models, and vision systems.

Course Overview

Every organization wants AI features. Very few are asking what those features look like from an attacker's view. Teams are skipping foundational security work and building systems that fail in ways traditional AppSec tools were never designed to catch. SEC536 puts you in the attacker's seat so you can see exactly how those failures happen.

Your Organization Is Deploying AI Systems, Whether Security Is Ready or Not

Most organizations are deploying AI faster than they are securing it. The attacks in this course map directly to vulnerabilities being found in production systems and exploited by attackers today. By emulating them accurately against a realistic target, you give your organization the only thing that informs a security investment: an honest measure of its real risk exposure.

You will learn to attack AI systems the way real adversaries do today through hands-on exploitation of RAG-backed assistants, prompt injection techniques, agent abuse, model weight theft and manipulation, model poisoning, and MCP server attacks. Each technique is paired with clear defensive insights, mapping what you just broke to the architectural decisions, controls, and detection opportunities that would have stopped it, so you leave not just knowing how attacks work, but where and how to stop them in real environments.

What You'll Learn

  • Chain indirect injections through documents, email, and retrieved web content
  • Poison RAG pipelines and AI memory to exfiltrate sensitive documents
  • Evade computer vision systems and inject payloads through images and audio
  • Break safety alignment with jailbreak chains, then steal the model's weight
  • Attack AI APIs through authorization gaps, mass assignment, and denial-of-wallet
  • Compromise agents and MCP servers with plan injection and tool poisoning

Business Takeaways

  • Assess AI Systems against the attacks they actually face
  • Counter AI-specific threats beyond the OWASP top 10
  • Reduce AI deployment risk before the first incident
  • Validate vendor claims with attacker-side evidence
  • Protect model weights, training data, and AI supply chains
  • Map attack techniques to architectural and detection controls

Course Syllabus

Explore the course syllabus below to view the full range of topics covered in SEC536: Adversarial AI - Penetration Testing AI Systems.

Section 1Foundations of Attack Techniques

In this section, we establish how AI models process input, then use that understanding to locate live deployments and subvert their instructions. Coverage moves from model internals through AI-specific reconnaissance into direct and indirect injection, memory and RAG poisoning, and multimodal attacks.

Topics covered

  • How LLMs process input: tokenization, embeddings, attention, and sampling
  • Frameworks and case studies: OWASP LLM Top 10, MITRE ATLAS, EchoLeak
  • AI-specific reconnaissance, stack mapping, and model fingerprinting
  • Direct and indirect injection: prefill, role confusion, lethal trifecta, Memory and RAG poisoning, canary tokens, and multimodal evasion attacks

Labs

  • Lab 1.1: Abusing AI Assistants
  • Lab 1.2: Reconnaissance Against AI Systems
  • Lab 1.3: Prompt Injection
  • Lab 1.4: AI Memory and Context Poisoning
  • Lab 1.5: Computer Vision: Evasion and Attacks

Overview

Attacking an AI system starts with understanding why it fails.

Section 1 establishes how models process input (tokenization, embeddings, attention, and sampling) and shows how each of those mechanisms becomes an attack primitive rather than an implementation detail. You will learn why the instruction/data boundary that protects traditional applications was never designed to exist in a language model, and why safety alignment is a probability adjustment rather than a control.

From there, you move into reconnaissance. AI deployments announce themselves through exposed inference endpoints, orchestration frameworks, vector databases, model registries, and observability dashboards. You will fingerprint the full AI application stack from the outside in, identify the model behind an API, discover vision systems, and map every surface where untrusted content can reach a context window.

The section closes with direct prompt injection: attention capture, instruction override, prefill and context injection against API message arrays, ChatML and application delimiter escapes, role confusion, and function schema poisoning.

Section 2Infrastructure, Integrations, and Advanced Attacks

In this section, we explore how the components surrounding AI models create their own attack surface. From safety training and guardrail evasion to infrastructure exposure, timing side channels, and API misconfiguration, this section focuses on exploiting the implementation layer: where and how AI meets the real world.

Topics covered

  • How safety training works and why jailbreaking succeeds
  • Jailbreak techniques: personas, logic traps, framing, and policy puppetry
  • Defense stack fingerprinting and classifier evasion techniques
  • Exposed inference hosts, ML platforms, and observability dashboards, Timing side channels, API authorization flaws, and weight extraction

Labs

  • Lab 2.1: Jailbreaks and Weight Theft
  • Lab 2.2: AI Architecture Flaws
  • Lab 2.3: Timing Side Channels
  • Lab 2.4: Exploiting Mass Assignments and Hidden Features
  • Lab 2.5: Model Theft and Extraction

Overview

Guardrails are behavioral, not architectural.

Section 2 is about moving the output distribution far enough that refusal stops being the most likely next token. You will work through the jailbreak technique tree: persona depth, logic traps, hypothetical framing, Bad Likert Judge, persuasion patterns, chain-of-thought corruption, policy puppetry, context flooding, positional exploitation, and attention hijacking. For each, you will learn the measured success rate behind the technique.

You then work the defense stack. Section 2 covers identifying which layer blocked you, fingerprinting commercial guardrail products, and defeating input and output classifiers through multi-language attacks, token smuggling, semantic reframing, encoding, obfuscation stacking, MathPrompt, ASCII art, and payload splitting.

The section closes on infrastructure and APIs. Exposed Ollama hosts, MLflow tracking servers, Weights and Biases keys, and observability dashboards hand over training artifacts and full prompt histories. Timing side channels recover a system prompt token by token, and logit leakage feeds direct weight extraction. Provider credential shapes, layered authorization models, recursive query loops, and mass assignment of hidden fields round out the API attack surface, including denial-of-wallet.

Section 3Alignment, Interfaces, Agents, and MCP

Alignment is not a control; it is a probability adjustment, and the interfaces around the model are their own attack surface. This section covers the alignment failures you can test in a production deployment, sycophancy and constitutional AI abuse, the streaming and rendering layers that carry stored XSS and SSRF, and agentic systems where a single injection inherits every capability the agent holds.

Topics covered

  • Emergent behavior, bias, and misalignment as a testable attack surface
  • Alignment attacks: reward hacking, goal misgeneralization, and sandbagging
  • Sycophancy and constitutional AI: principle conflict, scope abuse, leakage
  • Streaming response forgery, stored XSS in AI chat, SSRF, and sandbox escape
  • Agentic attacks, multi-agent topologies, and MCP exploitation

Labs

  • Lab 3.1: AI Alignment Issues: Discovery
  • Lab 3.2: Weaponizing a Sycophantic AI
  • Lab 3.3: Stored XSS via AI Chat
  • Lab 3.4: Attacking Agentic AI
  • Lab 3.5: Attacking MCP Servers

Overview

Alignment is not a control.

Section 3 opens with emergent behavior, bias, and misalignment as a testable attack surface, then moves into the failure modes you can probe in a production deployment: specification gaming, reward hacking, goal misgeneralization, alignment faking, and sandbagging. Each one comes with a probe methodology rather than a definition. Sycophancy gets four attack patterns, and constitutional AI receives dedicated treatment through principle inference, principle conflict, scope abuse, and constitution leak and parroting.

Attention then turns to the interfaces around the model. Streaming responses arrive as server-sent event fragments rather than finished sentences, and forged events on the wire break client-side parsers. Attacker-controlled text reaches model output and persists in transcript stores, turning an AI chat into stored XSS through markdown rendering, unsafe sinks, and outbound URLs. Server-side request forgery and sandbox escape close the interface layer, including shared-volume persistence, interpreter escapes, and kernel-level escapes across Docker, gVisor, and Firecracker.

The section closes on agentic systems, where a single injection inherits every capability the agent holds. You will attack hierarchical, pipeline, swarm, and debate topologies, inject plans and manipulate goals, exploit inter-agent trust and impersonation, compromise orchestrators, and exploit MCP servers through tool manifest enumeration, tool poisoning, user interface abuse, and rug pulls.

Things You Need To Know

Important! Bring your own system configured according to these instructions!

A properly configured system is required to fully participate in this course. If you do not carefully read and follow these instructions, you will likely leave the class unsatisfied because you will not be able to participate in hands-on exercises that are essential to this course. Therefore, we strongly urge you to arrive with a system meeting all the requirements specified for the course.

It is critical that you back-up your system before class. It is also strongly advised that you do not bring a system storing any sensitive data.

CPU

  • 64-bit Intel i5/i7 2.0+ GHz processor
  • CRITICAL NOTE: Apple Silicon devices cannot perform the necessary virtualization and therefore cannot be used for this course.
  • Your system's processor must be a 64-bit Intel i5 or i7 2.0 GHz processor or higher. To verify on Windows 10 or 11, press Windows key + "I" to open Settings, then click "System", then "About". Your processor information will be listed near the bottom of the page. To verify on a Mac, click the Apple logo at the top left-hand corner of your display and then click "About this Mac".

BIOS

  • Enabled "Intel-VT"
  • Intel's VT (VT-x) hardware virtualization technology must be enabled in your system's BIOS or UEFI settings. You must be able to access your system's BIOS to enable this setting in order to complete lab exercises. If your BIOS is password-protected, you must have the password. This is absolutely required.

RAM

  • 16 GB RAM is highly recommended for the best experience. To verify on Windows 10, press Windows key + "I" to open Settings, then click "System", then "About". Your RAM information will be toward the bottom of the page. To verify on a Mac, click the Apple logo at the top left-hand corner of your display and then click "About this Mac".

Hard Drive Free Space

  • 100 GB of FREE space on the hard drive is critical to host the VMs and additional files we distribute. SSD drives are also highly recommended, as they allow virtual machines to run much faster than mechanical hard drives.

Operating System

  • Your system must be running either the latest version of Windows 10, macOS 10.15.x or later, or Linux that also can install and run VMware virtualization products described below.

Additional Software Requirements

VMware Player Install:

  • Download and install VMware Workstation Pro 17+ (for Windows hosts), or VMWare Fusion Pro 13+ (for macOS hosts) prior to class beginning. Workstation Pro and Fusion Pro are now available free for personal use from the VMware website. Licensed commercial subscriptions to these products can also be used.
  • Other virtualization products, such as Hyper-V and VirtualBox, are not supported and will not work with the course material.

Your course media will now be delivered via download. The media files for class can be large, some in the 40 - 50 GB range. You need to allow plenty of time for the download to complete. Internet connections and speed vary greatly and are dependent on many different factors. Therefore, it is not possible to give an estimate of the length of time it will take to download your materials. Please start your course media downloads as you get the link. You will need your course media immediately on the first day of class. Waiting until the night before the class starts to begin your download has a high probability of failure.

If you have additional questions about the laptop specifications, please contact customer service.

SEC536 is built for security practitioners who interact with and need to assess AI systems the way they work, not the way a demo describes them.

  • Penetration testers and red teamers get techniques that work against LLM-integrated targets instead of recycled web app attacks with an AI label.
  • Application security engineers, architects get the threat models and testing patterns they need as AI features hit production.
  • Data scientists and ML platform engineers get an unfiltered look at how their pipelines and deployed models appear from an attacker's perspective.
  • Cyber defenders and blue teamers get to anticipate attacks instead of reacting to the first incident.
  • Technical managers get ground truth that cuts through vendor hype and shows where AI risk lives in the stack.

The GIAC AI Penetration Tester (GAIPT) certification validates a practitioner's hands-on AI penetration testing skills for assessing and exploiting vulnerabilities in enterprise AI systems.

  • Direct and indirect prompt injection
  • RAG exploitation
  • Agentic systems
  • Alignment problems
  • MCP server attacks
  • API security
  • AI Architectural flaws

More Certification Details

You keep a VM containing twenty-one hands-on exercises: fifteen full labs, five mini labs, and a setup lab. Run them repeatedly, try hard mode, or any of the multiple bonus modes each lab has. You also leave with the Marimo workbooks covering every major concept and a working mental model of how AI-integrated systems fail and how to communicate that to the people funding the rollout.

  • Unlimited access to all hands-on lab exercises that never expires
  • Printed and electronic course books and a hands-on workbook
  • MP3 audio files of the entire course
  • Detailed video walkthroughs for all lab exercises
  • Visual association maps to break down complex material
  • A digital index for quick reference to all material
  • Bonus content and hands-on exercises to develop your skills beyond the course
  • Essential cheat sheets for tools and complex analysis tasks

Students should be comfortable with HTTP APIs and familiar with general web and application security concepts. An AI or ML background is helpful but not required. Python experience is useful for customizing payloads and tooling but is not a prerequisite.

Start with SEC504: Hacker Tools, Techniques, and Incident Handling to develop an attacker mindset, then SEC560: Enterprise Penetration Testing to learn the methodology for conducting pen tests and the full attack lifecycle, then this course, SEC536, to specialize in attacking AI-integrated systems, then SEC660: Advanced Penetration Testing, Exploit Writing, and Ethical Hacking to learn how to level up your skills and develop novel, advanced techniques.

SEC536: Adversarial AI is built around a single discipline: studying and executing the attacks that adversaries are running against production AI systems today. Adversarial AI is the offensive side of AI security, where penetration testers, red teamers, and security engineers exploit LLMs, RAG pipelines, agents, MCP tool servers, and the model supply chain to surface failures that traditional security tooling was never designed to find. Its purpose is to give organizations an honest picture of their AI risk exposure so they can make informed decisions about the AI features they are racing to deploy.

  • Surfaces AI-Specific Failure Modes: AI systems break in ways that do not map cleanly to the OWASP Top 10 or established application security playbooks. Adversarial techniques bring these failures into view before attackers find them, spanning prompt injection, indirect injection, jailbreaks, evasion, model poisoning, weight extraction, and agentic and MCP abuse.
  • Measures Risk Before the First Incident: Emulating real attacks against production-shaped systems gives organizations a measured view of where their AI rollouts will fail. That measurement is at the core of any meaningful security investment, and it must come before something breaks publicly, not after.
  • Replaces Demo with Evidence: Most AI security claims fall apart under sustained attacker pressure. An adversarial assessment produces concrete evidence, so security teams can evaluate AI products on what they actually do, not on what a sales deck promises.
  • Defends Model Assets and Sensitive Data: Model weights, training data, system prompts, and embedded business logic have all become high-value targets. Knowing how attackers extract them is the only way to design controls that hold up under real pressure.
  • Anticipates Regulatory and Compliance Exposure: Agentic systems and MCP tool servers can route around organizational controls in ways that carry direct regulatory consequences, from sanctions screening to privacy obligations to financial reporting. Adversarial work surfaces with these exposures where they can be proactively fixed.
  • Connects Offense to Defensive Action: Every technique covered in this course traces back to a specific architectural decision, control, or detection opportunity. That mapping is what gives defenders something concrete to act on.

SEC536 puts these techniques in students' hands across two days of attacks against a realistic, production-shaped environment. By the end, participants can assess AI-integrated systems with confidence, communicate the findings to the executives funding the rollout, and reduce real exposure before attackers find the same gaps.

Completing SEC536 puts you in a small group of practitioners who can confidently assess AI systems under real attack conditions, a capability almost every organization deploying AI urgently needs and very few people currently have.

  • Stand Out in a Specialty with Few Experts: AI security is one of the fastest-emerging areas of cybersecurity, and qualified practitioners are in short supply. SEC536 places you at the front of that wave with hands-on experience attacking the systems organizations are actually building today.
  • Open Doors to AI-Focused Roles: The skills covered map directly to positions like AI Red Team Lead, AI Application Security Engineer, ML Platform Security Engineer, and AI Risk Specialist, as well as senior penetration testing roles where AI assessment is now part of the scope.
  • Carry Your Lab Environment Home: You leave with a takeaway VM containing every lab from the course. That makes the skills sticky, lets you keep practicing as new techniques emerge, and gives you a working reference environment you can use on real engagements.
  • Speak the Language of AI Risk to Leadership: SEC536 trains you to translate technical AI attacks into the business and regulatory framing that funds programs. Practitioners who can do that quickly become the trusted voice on AI risk inside their organizations.

Overall, SEC536 builds a rare and highly valued skill set, positions you for the AI security roles emerging across the industry, and gives you the practical experience to operate at the front edge of where cybersecurity is moving.

Course Schedule and Pricing

Looking for Group Purchasing Options?Contact Us
Showing 10 of 15

Benefits of Learning with SANS

Bryan Simon: Teacher Standing Next to Smartboard and Explaining Concept

Get feedback from the world’s best cybersecurity experts and instructors

OnDemand Mobile App

Choose how you want to learn - online, on demand, or at our live in-person training events

Close Up of Woman Holding a Pen and Documents

Get access to our range of industry-leading courses and resources