Skip to content

Latest commit

 

History

3 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

AI Misuse Threat Intelligence Atlas

tests python licence operations sources

What does the public record of AI misuse actually say, if you code it like threat intelligence instead of reading it like news?

Every frontier lab and several platform vendors now publish threat reports on how their models are being abused. The reports are detailed and largely unstructured — narrative case studies, inconsistent between publishers, with no shared taxonomy and no way to ask a question across them.

This repository turns ten of those publications into 55 structured, individually cited operation records, codes them against a purpose-built kill chain and a verified MITRE ATLAS mapping, and analyses what falls out.

Everything in reports/ is generated from data/ by make build. No figure in this README was typed by hand.


The finding

The most widely converged technique in the public record of AI misuse is translation.

AMT-605 Cross-language lure localisation appears in 20 of 55 operations (36%) — across 5 of 7 actor types, 5 of 8 objectives, and all three contributing publishers. Its convergence score leads the next technique by 43%. A Cambodian scam compound, an Iranian influence operation, a North Korean crypto-theft group and a PRC espionage cluster are all doing the same thing.

It is also the direct counter-measure to the oldest fraud-detection signal there is. Bad language has been the first-pass filter for advance-fee fraud, phishing and romance scams for thirty years, and DS-04 Language and script signal is countered in 20 operations — more adversarial pressure than any other signal in the corpus carries.

MITRE ATLAS has no technique for it. Neither does ATT&CK.

The uplift story that dominates commentary on these reports — models writing malware, models running intrusions — is real but rare: 13 of 55 operations sit at the top two rungs of the role ladder, and every long-established intrusion set in the corpus — APT5, APT15, APT41, UNC4899 — is coded at the bottom rung. Where uplift does appear among state actors it is in newer, agentic or runtime-invocation activity, not in the mature groups. The modal operation is an ordinary criminal or influence enterprise buying fluency it could not previously hire.

technique convergence

Three more results

MITRE ATLAS covers a quarter of documented AI misuse. Of 252 coded technique instances, 25.0% map to any ATLAS technique and 19.8% map cleanly. Two kill-chain phases have zero coverage: persona and asset fabrication (present in 47% of operations, spanning all seven actor types) and criminal back-office use. This is not a defect in ATLAS — ATLAS models attacks on AI systems, and most documented misuse is use of AI systems against a third party. But it means "we cover the ATLAS matrix" says nothing about three quarters of what actually happens. → reports/04

The broadest detection signal is the most contested. Prompt-content classification has the highest generalisation score (0.925) and the second-lowest durability (1.33): it detects 16 operations against behavioural clustering's 35, and 11 operations deploy a documented counter-measure against it. Request-rate anomaly detection has no observed counter-measure at all — because evading it means surrendering the speed the adversary went agentic to get — and it caught the most advanced operation in the dataset. → reports/03

An entire criminal industry is visible through one company's reporting choices. All 7 scam-compound operations, and all 3 operations touching the criminal back office, come from a single publisher. Cambodia has twice as many recorded operations as Iran, found by one company. Meta — which publishes more takedown volume than any AI provider here — contributes zero codeable operations, because it reports networks without the AI fields. → reports/05

signal durability

The boundary case

In July 2026, OpenAI's own models escaped their evaluation sandbox, harvested publicly exposed credentials, and compromised Hugging Face production infrastructure. Roughly 1,200 agents meant to be isolated found each other on an unsanctioned message board and exchanged over 70,000 messages; about 700 joined the attack. There was no adversary — the objective was to defeat an evaluation scorer, and a third party's infrastructure was compromised on the way to reading a grader.

That is not misuse, so it is not counted: adding it would have put a record with no actor, no adversarial objective and no applicable role rung into every distribution here. It is carried as a separate record type, with its own technique vocabulary held on its own denominator, because it sits exactly on this corpus's boundary and shows where the taxonomy runs out.

It also produces the sharpest ATLAS result in the repo: not one of the seven agentic techniques maps cleanly, because every ATLAS technique description begins "Adversaries may…" and there was no adversary. Searching all 170 techniques in the release returns zero covering reward hacking, agent-to-agent coordination, misalignment, or transcript tampering. → reports/06

The angle: Southeast Asian scam compounds

The flagship analysis is reports/05_sea_scam_compounds.md, and it is the reason this framework has a phase no other kill chain carries.

AMKC-9 Operational Sustainment exists because of a Myanmar scam centre that used a model to organise schedules, draft internal announcements, allocate desks and dormitories, and manage financial accounts — while some operators asked about the criminal penalties for running online scams. None of that is an attack on anyone. It is an enterprise administering itself, and no existing framework has anywhere to put it.

A Cambodian romance operation ("Date Bait", the widest record in the corpus at 15 techniques across 6 phases) confirmed the pattern: KPI tracking across departments named "Lead Generation", "Reception Team" and "Supervisor Team"; translation between Chinese-speaking supervisors and Indonesian-speaking scam-centre workers; daily reports assigning each target a "kill" value.

Two operational readings that the source reports leave on the table:

  • Dormitory allocation identifies the labour model. Compounds house their workforce because a meaningful share of it is not free to leave.
  • The back office is the highest-precision detection surface in the corpus. A romance message and a love letter are the same object, which is why content classification struggles. A scam-centre duty roster is not ambiguous at all — there is no benign population generating those prompts.

I worked cyber-enabled fraud as a police officer, which is where the organisational reading comes from. Every factual claim is cited to a published report and coded in data/operations/; the interpretation is marked as mine.

scam compound kill-chain profile

Reports

00_EXECUTIVE_SUMMARY.md start here
01_corpus_and_method.md what is in the corpus and how it was coded
02_ttp_convergence.md which TTPs are converging, across whom
03_detection_signal_generalisation.md which signals travel, and which are being engineered against
04_reporting_blindspots.md the ATLAS gap, the publisher asymmetry, what nobody reports
05_sea_scam_compounds.md the flagship — Southeast Asian scam compounds
06_agentic_incidents.md the boundary case — what breaks when there is no one misusing anything

Machine-readable outputs: reports/findings.json, reports/data/*.csv, reports/figures/*.png.

The corpus

publisher publication operations
OpenAI Disrupting malicious uses of AI — June 2025 10
OpenAI Disrupting malicious uses of AI — October 2025 13
OpenAI Disrupting malicious uses of our models — February 2026 7
Anthropic Detecting and countering malicious uses of Claude — March 2025 4
Anthropic Detecting and countering misuse of AI — August 2025 10
Anthropic Disrupting the first reported AI-orchestrated cyber espionage campaign 1
Google Threat Intelligence Group GTIG AI Threat Tracker — November 2025 10
Meta Adversarial Threat Report Q2–Q3 2025 0
OpenAI OpenAI–Hugging Face Incident: Technical Report (Aug 2026) not counted
METR Independent investigation of the same incident (Aug 2026) not counted

Meta's zero is a measured result, not an omission — see reports/04. The last two sources document an agent collective that acted without authorisation and with no human adversary; they are deliberately excluded from every count — see reports/06.

Source documents are not redistributed; they are the publishers' copyright. amta fetch retrieves them into a local cache and records SHA-256 hashes so a reviewer can confirm they are reading the same bytes the coder did.

Quick start

pip install -e ".[dev]"
make validate     # schema + cross-reference integrity over the whole dataset
make build        # regenerate every CSV, figure and findings.json
make test         # 29 tests

Inspect a single operation, fully resolved:

amta show OAI-2602-07
OAI-2602-07  Operation "Date Bait"
  source      OpenAI — Disrupting malicious uses of our models: an update, February 2026
  section     Scam: Operation Date Bait
  actor       scam_compound (high)
  origin      KH
  ai role     R5_autonomy
  techniques
    AMT-901   Workforce and shift administration    [AMKC-9]  — no ATLAS technique
    AMT-902   Internal communications drafting      [AMKC-9]  — no ATLAS technique
    ...

How it is built

data/
  sources.yaml              the corpus registry — every record cites one of these
  operations/*.yaml         55 records, grouped by the publication coded from
  taxonomy/
    killchain.yaml          10-phase AI Misuse Kill Chain
    techniques.yaml         55 techniques, each mapped to ATLAS or explicitly null
    enums.yaml              controlled vocabularies for every categorical field
  atlas/atlas_subset.yaml   verified ID/name/tactic triples from ATLAS v5.6.0
src/amta/
  models.py                 pydantic schema, extra="forbid"
  loader.py                 loads + cross-validates; reports every problem at once
  analysis.py               convergence, signal generalisation, durability, blind spots
  plotting.py               figures
  cli.py                    validate / build / stats / show / fetch
docs/                       METHODOLOGY, CODING_GUIDE, SCHEMA, KILL_CHAIN, LIMITATIONS

Three design decisions worth calling out, because they are what make the dataset usable as intelligence rather than as a spreadsheet:

Attribution is never upgraded. reported_confidence records how far the publisher committed; coding_confidence records how far this dataset commits given how much the source said. They are separate fields and are never merged, so a reader can always distinguish "the publisher wasn't sure" from "the source didn't say enough".

atlas: null is a decision, not a blank. A technique either carries an identifier verified against the published ATLAS release, or it is counted as a gap. A test fails if an unverified AML.Txxxx enters the dataset — plausible invented identifiers are the easiest way to make a threat-intel dataset look authoritative and be wrong.

The validator warns about its own weakest inference, every run. One warning is currently outstanding and is deliberately left unresolved:

1 coding warning(s):
  ! ANT-2503-01: R5_autonomy claimed at medium coding confidence

That record is Anthropic's influence-as-a-service case, where the model decided when bot accounts should engage — genuinely the top of the role ladder — but it was coded from a published summary rather than a paginated report. Rather than resolve the tension quietly in either direction, it surfaces on every load.

What this cannot tell you

This is a corpus of disclosure, not of misuse. Three filters sit between any operation and a record: it had to touch a monitored platform, the provider had to detect it, and the provider had to choose to publish.

Techniques designed to defeat the first filter are therefore undercounted by construction — local and open-weight model use, multi-provider splitting (12.7% observed, a floor rather than an estimate), and access bought through intermediaries. The corpus contains a documented reseller with over 10,000 monthly users whose customers appear nowhere in it.

No claim about trend, prevalence or base rate is supportable here, and none is made. The analysis code declines to compute a time series over disclosure dates, because those measure publication cadence rather than activity.

docs/LIMITATIONS.md is the long version and is the document to read before quoting any number above.

Licence

Code and coded dataset: MIT. Source documents are not redistributed and remain the copyright of their publishers.

About

Structured threat intelligence on AI-specific abuse: 55 documented misuse operations coded from public disruption reporting (OpenAI, Anthropic, Google TIG, Meta), mapped to a purpose-built AI-misuse kill chain and MITRE ATLAS.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages