Adam Gleave

Is the AI-nuclear weapons analogy valid? I joined the Verifiably Authentic podcast from SAS to talk about all things generative AI, from deepfakes to cyberwarfare. Kimberly Nevala gave me the chance to share some recent thinking on the root causes of AI regulatory paralysis. While AI could enable existential threats in the future—whether at the scale of nuclear warfare or not—I argue that there are more immediate harms here and now that need addressing. Full episode available below. https://lnkd.in/gTVR2WKX

68

3 Comments

I've just returned from #EMNLP2025, in China, and I'm still intrigued and thinking a lot about the talks and discussions that unfolded there. It was great to notice the convergence of Natural Language Processing (#NLP) and Human-Computer Interaction (#HCI)—two fields that, when combined, have the potential to unlock and make transparent essential social-technical criteria to build AI systems that make sense to people. We seem to be moving toward AI systems that are: ➡️ Understandable: Designed with transparency and user control in mind, despite the challenges to do so nowadays with GenAI.  ➡️ Configurable: Tailored to human needs, requiring interfaces for developers and final users to set up and align human values.  ➡️ Collaborative: Built through shared knowledge and complementary skills across domains. ➡️ Evaluated through emergent methodologies: Going beyond benchmarks to assess real-world data, traceability, and societal impact. It was also amazing to meet in person so many brilliant researchers from both inside and outside #IBM. I've included a list of some of our IBM work, published at #EMNLP25, below for those interested in exploring it further. Synthetic Data for Evaluation: Supporting LLM-as-a-Judge Workflows with EvalAssist https://lnkd.in/dzwsFk7F Martín Santillán Cooper, Zahra Ashktorab, Hyo Jin Do, Erik Miehling, Werner Geyer, Jasmina Gajcin, Elizabeth Daly, Qian Pan, Michael Desmond Collaborative Co-Design Practices for Supporting Synthetic Data Generation in Large Language Models: A Pilot Study https://lnkd.in/ddeaf-Ms Heloisa Candello, Raya Horesh, Aminat Adebiyi, Ph.D. , Muneeza Azmat, Rogério de Paula, Ph.D., Lamogha C. Effective Red-Teaming of Policy-Adherent Agents https://lnkd.in/d6u5eBPp Itay Nakash, George Kour, Koren Lazar, Matan Vetzler, Guy Uziel, Ateret Anaby-Tavor FactReasoner: A Probabilistic Approach to Long-Form Factuality Assessment for Large Language Models https://lnkd.in/ddhKMH3a Radu Marinescu, Debarun Bhattacharjya, Junkyu Lee, Tigran Tchrakian, Javier Carnerero Cano, Yufang Hou, Elizabeth Daly, Alessandra Pascale Towards Enforcing Company Policy Adherence in Agentic Workflows https://lnkd.in/dWVDjUei Naama Zwerdling, David Boaz, Ella Rabinovich, Guy Uziel, David Amid, Ateret Anaby-Tavor Quality Assessment of Tabular Data using Large Language Models and Code Generation https://lnkd.in/d8hJcV2r ashlesha akella, Akshar Kaul, Krishnasuri Narayanam, Sameep Mehta #EMNLP2025 #NLP #HCI #HumanCenteredAI #IBMResearch

117

Can you build an AI Safety startup in 5 days? Participants from v4 of Incubator Week have already tracked down previously unknown Chinese data centers, pushed new research on AI personas, written fieldstrategy for the AI for epistemics field and massively accelerated AI safety grantmakers (me included). They have also raised pre-seeds totaling more than $500k from Coefficient Giving, Longview Philanthropy, and of course BlueDot Impact. If you too want to start a high-impact AI safety org, applications for v5 close on August 14th.

246

20 Comments

Very excited to share that Plural is leading a $60 million Series A in CoMind, the startup building breakthrough optical sensing technology to revolutionise brain monitoring and treatment. For decades, doctors treating critically ill patients have been forced to compromise when monitoring the brain. Today's approaches use risky, expensive, highly invasive procedures that require drilling a hole into a patient's skull, or rely on inaccurate non-invasive monitoring that can compromise treatment decisions. To take just one example of why this matters, in the US alone 3 million patients suffer traumatic brain injuries (TBIs) every year, but only 5% receive an intracranial pressure test, which requires drilling a hole in the skull and carries a 15% complication rate. The rest are treated with limited information, leading to worse data, worse outcomes and higher costs. London-based CoMind is changing that. Its optical neuromonitoring platform can non-invasively measure three core brain health indicators - cerebral blood flow, autoregulation and intracranial pressure - something that has never been done before. It’s a breakthrough that could help tens of millions of patients worldwide and redefine how we understand and treat the brain. The company is led by a true outlier of a young founder, James Dacombe. He started programming when he was 13 and founded CoMind when he was just 17. His work ethic and rapid learning rate have led him to build a working product and take it through two successful clinical research studies, with the help of a stellar leadership team of medical and industry veterans from the likes of Medtronic, Tufts Medical Center and UCL.  This is only the beginning. CoMind plans to expand its product portfolio to measure additional physiological parameters and develop CoVision, an AI platform that transforms sensor data into predictive insights, identifying complications early and personalising treatment. The company aims to gain regulatory approval within two years, unlocking access to the huge ICU market in the US. And while this first sensor can revolutionise TBI monitoring, a second device is already in development to measure more parameters and treat more brain conditions. This technology has the potential to dramatically improve how we monitor our brains, giving doctors better information and more choices for treating patients, while building out a whole new market by providing a far more affordable and safe alternative to existing solutions. Read more about why we invested in CoMind here: https://lnkd.in/e-NPgDVp And Tim Bradshaw’s write up in today’s Financial Times here: https://lnkd.in/eAynvqPF

266

13 Comments

Across Britain this week, teenagers are sweating through GCSEs and A-levels, months of revision, sealed papers, invigilators pacing the aisles. We don’t hand out qualifications on self-declared talent; we verify it. Funny, then, that the week’s biggest AI-in-insurance stories were all about what happens when nobody checks the homework: Aviva caught a record £233m in fraud: including deepfaked accident scenes and fake policies sold on TikTok. AI is now sitting both sides of the fraud table. One in five insurers is deploying AI while cutting the training budgets needed to make it work. That’s sending staff into the exam hall having cancelled the revision classes. Even the AI makers want invigilators. Anthropic’s chief called for models to be blocked until independently audited; as ransomware hit near-record highs and Travelers flagged widening AI governance gaps inside organisations. The thread: capability without verification is exactly where the losses are landing. The firms that win this cycle will treat AI like an exam candidate: test it, train for it, audit it; not like a prodigy who marks their own paper. Full wrap, quick infographic and short explainer video: #LondonMarket #InsurTech #ArtificialIntelligence #AIGovernance #FraudPrevention #CyberRisk #Insurance

3

What if we stopped asking LLMs to generate rankings? That's the intriguing idea behind a new paper from Meta: hLLM: Single Pass Decoding for Generative Reranking. LLMs can be very effective at ranking candidates. But there's a fundamental problem with using them as generative rankers: generation is sequential. If the model needs to produce a ranking of N items, it generates N item IDs one after another. Every additional position means another decoding step. The hLLM authors ask a different question: Does ranking actually need to be generated at all? The authors start with an autoregressive LLM as a teacher, generating high-quality rankings one item at a time. They then use teacher ranking distillation to train hLLM to reproduce that ranking behavior without requiring autoregressive decoding at inference time. The key insight is that a ranking is ultimately a permutation: each candidate needs to be assigned to exactly one position. hLLM uses a single prefill pass to produce an item × position score matrix – how well each candidate fits each position – and then treats ranking as a bipartite assignment problem. The Hungarian algorithm finds the permutation with the highest total score. So the distilled hLLM model doesn't generate the ranking. It scores the possible assignments, and a classical optimization algorithm constructs the ranking from those scores. The results are striking: 28 ms end-to-end inference and a reported 64× speedup, while maintaining ranking quality comparable to the teacher model. We've spent a lot of time figuring out how to make LLMs generate things more efficiently. But sometimes the bigger opportunity is to recognize that the problem we're solving has a structure that doesn't require generation at all. For ranking, that structure is a permutation. Once you recognize it, you can replace sequential decoding with a combinatorial optimization problem. Ultimately, the key is knowing when not to generate. #InformationRetrieval #Search #LLMRerankers Paper: https://lnkd.in/ebM_UY4G

250

8 Comments

Our recent paper, "Neural reranking for UK statutory retrieval: Provision-level evaluation and an open distilled model," has been published in the Artificial Intelligence and Law journal (Springer). In this study, we evaluate the effectiveness of neural reranking for retrieving UK statutory texts at the provision level. Alongside our findings, we have also released an open distilled model tailored specifically for legal information retrieval. The paper is Open Access and can be read in full here: https://lnkd.in/eEf7qXR8 #LegalTech #InformationRetrieval #UKLaw #OpenAccess

4

2 Comments

Creativity with RL Trained Reasoning Model? Not quite yet. Despite strong performance (Pass@1) compared to the base LLM, RLVR reduces Pass@k at larger k, meaning the model solves fewer distinct problems given more attempts or the LLM loses diversity after RLVR training — the “reasoning boundary” shrinkage. This limits novel problem-solving and test time scaling. Why does this happen? And can we fix it? Introducing "negative interference", "winner-take-all", and "data curation" for RL Training of LLM in our newest work (highlights below): Phuc Minh Nguyen, Chinh La, Duy Ho Minh Nguyen, Nitesh Chawla, Binh T. Nguyen, Khoa D. Doan. The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models. https://lnkd.in/gf7x4A_f -- our first joint work between the Center for Environmental Intelligence (CEI) at VinUniversity, and Prof. Nitesh Chawla at the University of Notre Dame and Lucy Family Institute for Data & Society (also our Honorary Professor VinUniversity). Highlights: - There exists **negative interference** during RLVL training, where a model update on problems in a batch lowers the log-likelihood of correct solutions for others out of batch, due to correlated gradients across different problems. This strength of neg. interference increases during training. - On-policy of RLVR likely reinforces problems highly solvable under the base LLM while neglecting problems with initially low likelihood of correct solutions. Neg. interference makes it worse: these examples capture most of the learning signals, resulting in winner-take-all learning behavior that narrows the solution diversity. Existing regularization e.g. clipping or reverse-KL cannot help mitigate this reasoning boundary shrinkage. - Solution? We propose SEFL, which selectively learns on problems with low likelihood of generating correct solutions, significantly improving the Pass@k performance of RLVR. Preprint: https://lnkd.in/gf7x4A_f Code: https://lnkd.in/gXg65Wbp

32

4 Comments

Research insights from paper “Toolformer: Language Models Can Teach Themselves to Use Tools” By our ResearcherAhmad Afaneh Stay tuned for the next blogs 🔍 #AIPaperReadingClubs #Researcher #AIPR_GDGHU

5