Frontier Model Forum

AI agents can help cyber defenders act at greater speed and scale, from simulating adversarial behavior and analyzing threats to triaging alerts and finding and fixing vulnerable code. Building on FMF’s 2024 issue brief on AI for cyber defense, our latest publication explores where agents can provide value across the defensive lifecycle and what organizations need to deploy them effectively. The brief highlights five use cases in particular: 🔹 Simulating adversarial behavior in security testing 🔹 Accelerating threat intelligence research and analysis 🔹 Augmenting security operations and alert triage 🔹 Detecting and patching vulnerabilities 🔹 Improving code security upstream With emerging practices and operational norms still nascent, the issue brief closes with key considerations for using agents in cyber defense, from the importance of  cybersecurity fundamentals to matching agent autonomy to risk. You can read the full publication here: https://lnkd.in/ey-fCKcU

Frontier AI agents and AI-enabled biological tools hold significant potential for scientific research, but their combination may also introduce novel risks and threat scenarios in biosecurity. Our latest issue brief outlines two preliminary categories of concern – lowering barriers to entry for advanced biological capabilities, and raising the ceiling of harm – and also sets out considerations for evaluation, safeguards, and monitoring. You can find the issue brief here: https://lnkd.in/ehYB-aTg

As frontier AI capabilities advance, developing evaluations for CBRN and advanced cyber risks that work across languages, regions, and cultural contexts will be critical. Yet, multilingual evaluation coverage remains limited. Our latest issue brief highlights emerging practices for building multilingual CBRN and cyber evaluations, including key safety gaps and failure patterns, considerations for evaluation development and operationalization, and directions for future work: https://lnkd.in/eQuZRh6z

By combining advanced reasoning with the ability to act and call tools, frontier AI agents hold significant economic and scientific promise – yet also pose potential risks. Our new issue brief identifies emerging practices for securing the agent stack: https://lnkd.in/eviUZ5Mw

Information sharing, incident reporting, and incident response will each play an important role in managing frontier AI risks. Yet they are often treated interchangeably, when they are distinct and complementary tools that serve different purposes. Our new issue brief provides a high-level overview of these concepts, clarifies their respective roles, and highlights the Frontier Model Forum's ongoing information-sharing efforts: https://lnkd.in/eaKBFWcY

New Issue Brief: Adversarial Distillation Adversarial distillation is a growing concern for frontier AI safety and security. Although authorized distillation has legitimate use cases, such as the creation of more efficient and specialized models in the life sciences, adversarial distillation can enable malicious actors to extract core capabilities from frontier models while bypassing built-in safety measures. Our latest issue brief provides an overview of adversarial distillation methods and target categories. By raising awareness of adversarial distillation without discouraging legitimate use cases, the brief aims to foster greater capacity across the frontier AI ecosystem to identify the risks related to adversarial distillation. Read the full brief here: https://lnkd.in/eW8SJCCR

The Frontier Model Forum is pleased to share a progress update on our information-sharing efforts.  As we announced last March, the member firms of the FMF signed a first-of-its-kind voluntary agreement designed to facilitate information-sharing about vulnerabilities, threats, and concerning capabilities unique to frontier AI. Since then, FMF member firms have successfully shared covered information, from security vulnerabilities to jailbreak repositories and enhanced safety protocols. With our core infrastructure in place, we look forward to building on the agreement in ways that continue to strengthen the collective capacity of the frontier AI ecosystem to anticipate and mitigate large-scale risks. Read our progress update: https://lnkd.in/edRekT5E

𝗡𝗲𝘄 𝗥𝗲𝗽𝗼𝗿𝘁: 𝗠𝗮𝗻𝗮𝗴𝗶𝗻𝗴 𝗖𝘆𝗯𝗲𝗿𝘀𝗲𝗰𝘂𝗿𝗶𝘁𝘆 𝗥𝗶𝘀𝗸𝘀 𝗶𝗻 𝗙𝗿𝗼𝗻𝘁𝗶𝗲𝗿 𝗔𝗜 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸𝘀 Frontier AI offers significant promise for cybersecurity, including accelerating vulnerability discovery and patching, optimizing defensive systems, and enhancing threat detection capabilities. However, these same capabilities create dual-use risks, potentially lowering barriers for malicious actors to exploit known vulnerabilities or discover new attack vectors. As AI capabilities advance, it is crucial to develop robust risk management frameworks that maximize security benefits while proactively addressing emerging risks. Building on the Frontier Model Forum's Technical Report Series, this new report focuses on how frontier AI frameworks can be implemented in the cyber domain. Based on expert discussions among FMF member firms, this report highlights:  • Identification and threat modeling for severe AI-cyber risks  • Capability thresholds for advanced cyber risks   • Frontier capability assessments for advanced cyber risks  • Frontier mitigations for cyber risks as models approach or breach critical cyber capability thresholds Read the full report: https://lnkd.in/ekYfpgaC

As part of the upcoming AI Impact Summit in India, Frontier Model Forum and MLCommons are pleased to invite you to a panel discussion on “The Future of Frontier AI and Cybersecurity” on February 16th at 2:30PM (IST) at the Bharat Mandapam (West Wing Room 6). As AI capabilities advance, robust risk management and testing frameworks are essential to maximize security benefits while proactively addressing emerging threats. Our speakers will explore current offensive cyber threats from AI, evaluation methods and benchmarks, and best practices for mitigating AI-driven cyber risks. Featured speakers will include Amanda Craig Deckard, Amlan Mohanty, Jonas Kgomo, Supheakmungkol Sarin, PhD, Peter Mattson and Rebecca Finlay. To register for the event, please RSVP using the link below. We look forward to seeing you there! https://lnkd.in/gvyYV7TK

As AI capabilities advance, we've been examining potential dual-use risks to civilian nuclear materials and facilities. Over the past year, the Frontier Model Forum collaborated with nuclear security experts to conduct preliminary research into these risks. In our latest research update, we outline potential frontier AI capabilities of concern, challenges for AI-nuclear risk management, and areas for future work. This research update aims to inform greater understanding of these risks and establish a foundation for collaboration between the frontier AI and nuclear security communities. Read more here: https://lnkd.in/eBNq9Nbk