About
Prof. Jingjing Liu is Professor, Principal Investigator at Tsinghua University. Prof. Liu…
Activity
6K followers
-
Jingjing Liu shared thisI’m excited to share that our Emu paper on Multimodal LLM has published in Nature! Emu model series (Emu1, Emu2, Emu3) has been released by BAAI since 2023, with related papers published in CVPR, ICLR, etc. The latest Nature article is the culmination of years of research on large-scale multimodal modelling. Emu3 provides coherent, high-fidelity video generation, interleaved vision–language generation, and vision–language–action modelling for robotic manipulation, offering a powerful paradigm for Unified Multimodal Intelligence. Emu3 (Nature): https://lnkd.in/grcEzsHJ Emu2 (CVPR): https://lnkd.in/g_V2NHr4 Emu1 (ICLR): https://lnkd.in/gcnbYxWF GitHub: https://lnkd.in/gZ5DZi8F Hugging Face: https://lnkd.in/gRWyY4dp Project page: https://lnkd.in/guWZazSn Congratulations to my PhD student Qiying Yu and colleagues at BAAI: Xinlong Wang, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yonghua Lin, Zhongyuan Wang, Tiejun Huang, and many other contributors. #Nature #LLM #Multimodal #GenAIMultimodal learning with next-token prediction for large multimodal models - NatureMultimodal learning with next-token prediction for large multimodal models - Nature
-
Jingjing Liu shared thisI'm glad to share that our 5 recent papers have all been accepted by #NeurIPS2025, on RL reasoning LLM, AI for Science, and Autonomous Driving. 1. DAPO is our flagship, fully open-sourced, large-scale RL system that surpassed DeepSeek's R1 model on mathematical benchmarks with half of the training steps (collaboration with ByteDance). (https://lnkd.in/g8RJB3ne) 2. ProteoBayes is our new all-atom protein design model with Bayesian flows, which significantly outperforms SOTA over peptide design and antibody design tasks. (https://lnkd.in/g7UPc6Xr) 3. ShortListing Model is a simplex-based diffusion model that yields remarkable performance on DNA promoter and enhancer design, protein design, character-level and large-vocabulary language modeling tasks. (https://lnkd.in/gW9FnZ9T) 4. MOF-BFN is a novel BFN-based generative model that achieves new SOTA across MOF structure prediction, geometric property evaluation, and de novo generation tasks. (https://lnkd.in/gHdDFEx9) 5. Flow Planner is an advanced learning-based framework incorporating flow matching with classifier-free guidance, which demonstrates superior performance in real-world autonomous driving tasks. (https://lnkd.in/gs6NdiCT) Congratulations to Qiying Yu, Hanlin Wu, Yuxuan Song, Rui Jiao, Zhe Zhang, Yu Pei, Tianyi Tan, Yinan Zheng, Hao Zhou, Xianyuan Zhan, Yang Liu, Wei-Ying Ma, Zheng Zhang, Mingxuan Wang, and many other collaborators at Tsinghua University and ByteDance. #LLM #RL #AI4S #Autonomous
-
Jingjing Liu shared thisI'm excited to introduce our fully open-sourced large-scale RL reasoning model, DAPO-Zero-32B, which surpasses DeepSeek-R1-Zero-Qwen-32B, scoring 50 points on AIME 2024 with 50% fewer training steps. It is trained with zero-shot RL, and everything is fully open-sourced (algorithm, code, dataset, verifier, and model). DAPO (decoupled clip and dynamic sampling policy optimization) introduces four novel RL techniques, including Clip-Higher, Dynamic Sampling, Token-Level Policy Gradient Loss, and Overlong Reward Shaping, which makes RL shine in the long-CoT scenario. DAPO project was led by my PhD student Qiying Yu at Tsinghua University, in collaboration with ByteDance. Full contributor list is included in the paper. Homepage: https://lnkd.in/gqkAgMnV Paper: https://lnkd.in/gU9TKbem Code: https://lnkd.in/gGCZCjjf #LLM #RL #Reasoning #DeepSeek
-
Jingjing Liu shared thisBesides our latest Oral and Spotlight papers at #ICLR’25 (Diffusion Planner, CrysBFN, etc), we have several more papers accepted by #ICRA’25 on Robotics and Autonomous Driving. Robo-MUTUAL introduces a novel framework for robotic multimodal representation learning via unimodal training, which enables real robots to understand multimodal instructions for complex tasks. IROAM proposes a new contrastive learning method that enhances roadside monocular 3D detection for autonomous driving. CoopDETR introduces a cooperative perception framework that reduces transmission cost while improving 3D detection. https://lnkd.in/g6R_63V9 https://lnkd.in/gUw3S3w7 https://lnkd.in/gyAJ82QD Congratulations to Jianxiong Li, Zhihao Wang, Jinliang Zheng, Xianyuan Zhan, Zhe Wang, Xiaoliang Huo, Yan Wang and many other collaborators #Robotics #Multimodal #Autonomous #EmbodiedAI
-
Jingjing Liu shared thisThis has been a productive year. Besides the release of Multimodal LLMs (Emu3, CapsFusion) and Robotic Embodied AI (DecisionNCE, IVM), we have several more recent papers accepted by #ICLR’25, on AI for Science and Diffusion Models. Code and papers are available. CrysBFN proposes a novel crystal generation method via Periodic Bayesian Flow, which integrates a new entropy conditioning mechanism and significantly outperforms SOTA over crystal ab initio generation and crystal structure prediction tasks. Paper: https://lnkd.in/gsxajbiY Code: https://lnkd.in/gpQJW6A4 Our Diffusion Planner effectively leverages diffusion model by learning the gradient of trajectory score function, with a flexible classifier guidance mechanism, which excels in large-scale real-world autonomous planning tasks. Paper: https://lnkd.in/g2CCJAq8 Code: https://lnkd.in/ghgcy8qt In ‘Rethinking Diffusion Posterior Sampling (DPS)’, we significantly improve DPS by explicitly maximizing the posterior through multi-step gradient ascent and projection, with a light-weighted conditional score estimator. Paper: https://lnkd.in/gNSCJ4ba Code: https://lnkd.in/gJAThtiq Congratulations to Hanlin Wu, Yuxuan Song, Jingjing Gong, Hao Zhou, Yinan Zheng, Ruiming Liang, Kexin Zheng, Xianyuan Zhan, Tongda Xu, Yan Wang and many other collaborators. #AI4Science, #GenAI, #Diffusion, #Autonomous
-
Jingjing Liu shared thisEmu3 is our latest set of state-of-the-art Multimodal LLMs - code and models recently released by BAAI. Emu3 models surpass SOTA open models such as SDXL, LLaVA-1.6 and OpenSora-1.2 over a sweep of multimodal benchmarks on both generation and perception tasks, capable of generating high-fidelity video without using diffusion or compositional architectures. Paper: https://lnkd.in/gFbEMMjn Code: https://lnkd.in/gZ5DZi8F Models: https://lnkd.in/g9yepvxh Project page: https://lnkd.in/guWZazSn Core contributors: Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, and many others from BAAI. #Multimodal #LLM #GenerativeAI
-
Jingjing Liu shared thisWe have two recent papers accepted by #NeurIPS 2024, on Multimodal Embodied AI and LLM. ‘Instruction-Guided Visual Masking’ introduces IVM, a versatile visual grounding model, along with IVM-Mix-1M, a new dataset with 1 million image-instruction pairs. By constructing visual masks for instruction-irrelevant regions, IVM-enhanced LMMs and Robots can effectively follow complex multimodal instructions. Paper: https://lnkd.in/gCRizdhH Code: https://lnkd.in/gcW-cFfp ‘Mesa-Extrapolation’ introduces a novel weave PE (Position Encoding) method, which utilizes a chunk-based triangular attention matrix and applies Stair PE to manage the final chunk, significantly enhancing LLM extrapolation performance with reduced memory demand and faster inference speed. Congratulations to Jinliang Zheng, Jianxiong Li, Sijie Cheng, Yinan Zheng, Xianyuan Zhan, Xin Ma, Yang Liu and other collaborators. #EmbodiedAI #Multimodal #Robotics #LLM
-
Jingjing Liu shared thisOur latest work on Multimodal Embodied AI for Autonomous Robots, DecisionNCE, has been accepted by #ICML2024. DecisionNCE mirrors an InfoNCE-style objective distinctively tailored for decision-making tasks, providing an embodied representation learning framework that extracts both local and global task progression information with temporal consistency enforced through implicit time contrastive learning, while ensuring trajectory-level instruction grounding via multimodal pre-training and reward reparameterization. Paper: https://lnkd.in/gW_eNGVf Code: https://lnkd.in/gmhS2U99 Congratulations to Jianxiong Li, Jinliang Zheng, Yinan Zheng, Xianyuan Zhan and many other collaborators. #EmbodiedAI #Multimodal #Robot #ICML
-
Jingjing Liu shared thisOur latest LMM works - Emu2 and CapsFusion - have both been accepted by #CVPR2024. Emu2 is a 37-billion parameter LMM (Large Multimodal Model) that sets a new record on multimodal understanding tasks in few-shot settings. Trained with a unified autoregressive objective, Emu2 exhibits strong multimodal in-context learning abilities, supporting on-the-fly reasoning such as visual prompting and object-grounded generation. Paper: https://lnkd.in/gqDvTdaB Code: https://lnkd.in/gnE9vb75 CapsFusion framework leverages LLM to consolidate and refine both raw and synthetic image captions. Across model performance, sample efficiency, world knowledge depth and scalability, CapsFusion captions have exhibited remarkable all-round superiority over existing captions. Paper: https://lnkd.in/gKXWrfV3 Code: https://lnkd.in/g7dt4UMN Congratulations to all collaborators: Qiying Yu, Quan Sun, Xiaosong Zhang, Yufeng Cui, Fan Zhang, Yue Cao, Xinlong Wang, Zhengxiong Luo, Yueze Wang, Yongming Rao, Tiejun Huang from Tsinghua and BAAI. #LMM #multimodal #LLM #AI #CVPR
Jingjing Liu replied to a comment
1y
Thanks!
Jingjing Liu replied to a comment
1y
Sure!
Jingjing Liu replied to a comment
3y
Ashish Patel 🇮🇳Thanks! Evaluations on 9 benchmarks show that Emu outperforms similar-sized SOTA models across all tasks. Detailed results can be found in the paper
-
Jingjing Liu liked thisJingjing Liu liked this🏆 Honored to be recognized as one of the Top 3 Finalists for the Study UK Alumni Awards 2026 (China) in the Business and Innovation category. For a long time, I secretly doubted the saying, "Everyone who makes it to this stage is already a winner." However, the process of applying for the #StudyUKAlumniAward forced me to pause and "connect the dots" of my entrepreneurial journey. Looking back, everything feels like "Yuan Qi" (Origination)—the Buddhist concept that you must first plant an “intention” for a flower to bloom. That seed was sown 8 years ago with a viral matriculation photo at Hughes Hall, Cambridge, which opened my eyes to the life-changing power of digital influence. Today, that seed has grown into EvansChat Consulting Limited , a Xiaohongshu consultancy I founded to drive content commercialization across the Greater Bay Area in China. Seeing our work translate digital influence into real economic vitality in this region makes me realize that the journey itself is the true prize. My deepest gratitude goes to my academic home, Cambridge Judge Business School. Special thanks to Dean Gishan Dissanaike and Professor Simon Stockley for being the sparks behind my entrepreneurial fire during my MBA (2017-2018). It was also a true privilege to celebrate this milestone alongside distinguished leaders including Geraldine McCafferty Scott McDonald Nick Thomas and Leina Shi. Your support strengthens the bridge between the UK and China that we, as alumni, are so proud to build. Finally, huge congratulations to my incredible fellow Cambridge alumni recognized this year: Ge Yu (Culture, Creativity and Sport Award) and Professor Jingjing Liu (Science and Sustainability Award)! #StudyUKAlumniAwards #CambridgeMBA #CJBS #GreaterBayArea #Xiaohongshu #Entrepreneurship #EvansChat
-
Jingjing Liu liked thisJingjing Liu liked thisHonoured to be named a Finalist for the 2026 Study UK Alumni Awards in the Culture, Creativity and Sport Award category. Thank you so much British Council! It was both a great honour and a true learning experience to be among such an inspiring cohort of finalists. None of this would be possible without the foundational support from my academic home: University of Cambridge Cambridge Judge Business School: My profound thanks to the Dean Gishan Dissanaike，faculty, cohorts and—especially Ding Lan and Floriana Guida for their help. Pembroke College, Cambridge : For the vibrant community that made Cambridge truly special. Boustany foundation: For the pivotal scholarship and the Monaco internship that embedded the principle of using business for social good into my journey. Thanks to Mr.Fadi Boustany and Ms.Alison Pitt. I am also thankful to the Fondation Prince Albert II de Monaco for the invaluable internship experience. It was a great opportunity that significantly sharpened my skills and broadened my perspective on implementing sustainability. It was a privilege to share this moment with such esteemed guests. My sincere thanks to Geraldine McCafferty, Deputy Head of Mission at the British Embassy Beijing; Scott McDonald, Chief Executive of the British Council; Nick Thomas, Country Director of British Council China; and Leina Shi, Director of Education for British Council China, for their presence and for acknowledging the role alumni play in strengthening bridges between China and the UK. A very special thank you must go to Lirong from British Council for her unwavering dedication and meticulous work behind the scenes. My heartfelt congratulations to the incredible fellow The Cambridge MBA alumni recognised this year: Professor Jingjing Liu – Science and Sustainability Award and Vivian Li Business and Innovation Award. #2026StudyUKAlumniAwards #CambridgeJudge #CambridgeMBA #SustainableFashion #ANNDERSTAND #BritishCouncil
-
Jingjing Liu liked thisSo proud of everyone for making this happen! It's not easy to make meaningful improvements to the already SOTA Veo 3.0!!Jingjing Liu liked this🎬 The frontier of video generation has leaped forward! Google DeepMind's latest Veo 3.1 now ranks #1 in both Text-to-Video and Image-to-Video leaderboards. This is a +30-point leap from Veo 3.0 → 3.1, and the first model to break 1400 in Video Arena history. 🏆 Come test out Veo 3.1 and all the best Video models (Sora, Kling, Hailuo, Seedance, and more) with your hardest prompts in our Discord at: discord.gg/lmarena
-
Jingjing Liu liked thisJingjing Liu liked this💡 Computer-use agents (CUAs) rely exclusively on primitive actions (click, type, scroll) that require lengthy execution chains, which can be cumbersome and error-prone. How to improve this? 🔥 🔥 In our native agent UltraCUA, we advocate the idea of "hybrid action" -- seamlessly integrating GUI primitives with high-level programmatic tool calls. The resulting agent is therefore both stronger 💪 and faster 🚀 . 🔗 arXiv: https://lnkd.in/gbVV9a-A 1️⃣ We develop an automated pipeline for collecting programmatic tools, and a synthetic data engine producing 17,000+ verifiable tasks for computer use. 2️⃣ As a result, we produce a large-scale high-quality hybrid action trajectory collection with both low-level GUI actions and high-level programmatic tool calls. 3️⃣ We conduct both SFT and multi-turn RL. On OSWorld, UltraCUA-32B (41.0% success rate) improves over OpenCUA-32B (29.7%) by +11.3 absolute points 🚀 . Notably, UltraCUA also achieves strong performance on WindowsAgentArena, without any Windows-specific training. These results validate that hybrid action provides consistent benefits across model scales and platforms. Led by our awesome intern Yuhao Yang, joint work with Zhen Yang, Zi-Yi Dou, Anh Nguyen, Keen You, Omar Attia, Andrew Szot, Michael Feng, Ram Ramrakhya, Alexander Toshev, Chao Huang, Yinfei Yang.
-
Jingjing Liu liked thisJingjing Liu liked this🤔 Knowledge stored in multimodal LLM weights can be inherently limited. How can we empower multimodal LLMs with multimodal web search? 💡 In DeepMMSearch-R1, we aim to train a multimodal search agent capable of performing on-demand, multi-turn web searches and dynamically crafting queries for both image and text search tools. It can also initiate web searches based on relevant crops of the input image via using Grounding DINO as a tool. 🔗 arXiv: https://lnkd.in/gDNAesBv 1️⃣ In SFT, we teach the model when to search, what to search for, which search tool to use and how to reason over the retrieved information. 2️⃣ SFT enables tool use, while RL refines the tool-selection behavior by reducing unnecessary calls. Led by our awesome intern Kartik Narayan, together with Yang Xu, Tian Cao, Kavya Nerella, Navid Shiee, Peter Grasch, Chao Jia, Yinfei Yang etc. and thanks for the support from Yang Zhao.
-
Jingjing Liu liked thisJingjing Liu liked this🤔 Small GUI agents can be important for on-device deployment, but how far can such small agents go? 💡 Instead of scaling up, in Ferret-UI Lite, we focused on scaling down, and present our lessons from building small on-device GUI agents. 🔗 arXiv: https://lnkd.in/gWyYNejc 📔 Ferret-UI Lite is 3B in size. We conduct both SFT and RL, for both grounding and navigation, and across mobile, web and desktop platforms. 1️⃣ Ferret-UI Lite achieves strong results on GUI grounding (e.g., 53.3% on ScreenSpot-Pro and 61.2% on OSWorld-G), surpassing many larger models. We found that RL with zoom-in is effective to improve performance. 2️⃣ For navigation, Ferret-UI Lite achieves success rates of 28.0% on AndroidWorld and 19.8% on OSWorld, outperforming other 3B agents. But its performance can still be limited. We found carefully designed RLVR reward is important for training such small agents. We hope our lessons will inform future research on building efficient, capable, and practical on-device AI agents. Led by Zhen Yang Zi-Yi Dou Di Feng, Dr.-Ing. Forrest Huang Anh Nguyen Keen You Jeffrey Nichols Alexander Toshev Yinfei Yang etc. and thanks for the support from Yang Zhao.
Experience
Education
Projects
-
TicAuto & TicMirror
-
Leading an engineering team to develop TicAuto, an operating system that supports speech-enabled conversational assistance and gesture control in automobiles; leading a product team to develop a smart rear-view-mirror (TicMirror), which integrates Advanced Driver Assistance System, automatic camera, entertainment and other intelligent services using AI technology.
-
Venture Capital in the European Energy Industry
-
Investment strategy consulting for MOL Group, the leading energy company in Europe, for a 200-million-euro venture capital fund. Responsible for in-depth business segment analysis in the energy sector and portfolio development for long-horizon multi-asset investment.
-
Business Development in Virtual Reality/Augmented Reality (VR/AR) Industry
-
Management consulting for ARM, the world-leading semiconductor company, on business development in the VR/AR domain. Conducted market research and customer study for Geomerics, an ARM-owned computer-graphics company, and developed launching strategy for their 3D modelling products to enter the architecture market.
-
Microsoft XiaoIce
-
Researched on advanced machine learning algorithms that enabled Microsoft XiaoIce shopping assistant, a personalized conversation App, to boost e-commerce sales (e.g., JD.com) via automated chat agents.
-
MIT Asgard
-
Initiated the Asgard project, and led a team of research scientists to revolutionize the MIT dialogue system platform; developed a series of speech-enabled assistant systems (e.g., movie/restaurant/flight recommendation).
Languages
-
English
-
-
Mandarin
-
Other similar profiles
Explore more posts
-
William Small
I’ve published a research note and reproducible protocol on measurement contamination in AI self-report. Across multiple LLM architectures (Claude, GPT-4, Gemini, DeepSeek, Grok), I found that when response behavior is trained or policy-constrained, direct questioning becomes an unreliable instrument for inferring internal state. In that regime, denial carries no evidentiary weight because it is enforced independent of truth value. This work does not claim AI consciousness. It argues that some commonly used evaluation methods fail as measurements. Paper and protocol are fully public here: https://lnkd.in/gXSt2evd I’m interested in discussion around evaluation design, robustness to contaminated observables, and what signals remain valid when self-report is biased.
-
Peniel Argaw
Can frontier LLMs predict how long a patient will live? Our new paper, Large Language Models are Approximate Survival Estimators (https://lnkd.in/gjem4ReX), puts this question to the test. We introduce Survprompt, a framework for evaluating whether frontier LLMs can estimate patient survival zero-shot, directly from clinical information. Across two pan-cancer cohorts, frontier LLMs produced surprisingly competitive survival-time estimates. In some settings, their errors came within 10% of specialized survival models trained directly on the cohort data. But there’s an important catch: 𝗮𝗰𝗰𝘂𝗿𝗮𝘁𝗲 𝘀𝘂𝗿𝘃𝗶𝘃𝗮𝗹-𝘁𝗶𝗺𝗲 𝗲𝘀𝘁𝗶𝗺𝗮𝘁𝗲𝘀 𝗱𝗼 𝗻𝗼𝘁 𝗻𝗲𝗰𝗲𝘀𝘀𝗮𝗿𝗶𝗹𝘆 𝗺𝗲𝗮𝗻 𝗴𝗼𝗼𝗱 𝗿𝗶𝘀𝗸 𝘀𝘁𝗿𝗮𝘁𝗶𝗳𝗶𝗰𝗮𝘁𝗶𝗼𝗻. LLMs show lower concordance and substantial variability across cancer types and institutions—highlighting important limitations before these systems can be relied upon clinically. The results are both surprising and cautionary: pretrained LLMs appear to have internalized a meaningful approximation of clinical survival relationships, despite never being explicitly trained as survival models. 🔗 Read the paper: https://lnkd.in/gjem4ReX 💻 Code: https://lnkd.in/gC8jKk4G Thank you to the authors and collaboration between Microsoft Research and Providence! cc: Juan Manuel Zambrano Chaves, Risa Ueno, Carlo Bifulco, Dr. Kristina Young, Rom Leidner, Tristan Naumann, Hoifung Poon #Microsoft #MachineLearning #HealthcareAI #Oncology #LLMs #SurvivalAnalysis
16 Comments
Explore top content on LinkedIn
Find curated posts and insights for relevant topics all in one place.
View top contentOthers named Jingjing Liu in United States
77 others named Jingjing Liu in United States are on LinkedIn
See others named Jingjing Liu