Jeff Hawke

DeepSeek recently published a series of papers. DeepSeek Math-V2 features self-verification, while mHC proposes 'manifold-controlled multi-head residual connections' as an alternative to Kaiming He's famous ResNet. The Engram paper highlights the separation of computation and knowledge/memory lookup. Taking these papers together—along with the previous DSA—builds significant hype for the rumored DeepSeek V4/R2. The architectural innovation here is massive. Additionally, the mHC and Engram experimental models are both 27B, similar to the ~27B scale of last year's DeepSeek V3 Lite!

14

A huge milestone for Wayve today. $1.5B in new funding. $8.6B valuation. And, most importantly, the shift from research leadership to scaled commercial deployment. Wayve has spent nearly a decade pioneering end-to-end embodied AI for autonomous driving, turning a bold research vision into a global autonomy platform. To understand how it all began and where it’s heading next, listen to James Wise in conversation with founder Alex Kendall on What’s Next, covering his early ambition for Wayve, the founding story, why building in the UK was an advantage, and what he sees coming over the next 10 years. https://lnkd.in/eyATx8JM

64

2 Comments

Last week I wrote about a hacked Toyota Corolla running open-source self-driving software from a $1,000 device by comma.ai In the same days, NVIDIA responded from the #CES2026 stage. They announced Alpamayo: open-source AI models, simulators and datasets for autonomous driving, built around reasoning, not just perception. Chain-of-thought, vision-language-action models that do not only decide what to do, but can explain why. That is a big shift. Autonomy has been optimized for "normal days", but the real problems are rare, messy, human situations that break rigid pipelines. Alpamayo is designed to reason through those. A couple takeaways: 1) Open source is no longer just for geeks. It is becoming a safety feature. If you can see how the system reasons, you can test it, audit it and improve it. That matters much more than flashy demos when regulators, car makers and insurance companies have to trust the technology. 2) Big AI models will train smaller ones that actually drive the car. The smartest models will live in the lab and in simulation. Then their “knowledge” will be copied into lighter versions that can run on real vehicles. Long term leadership in autonomy will likely favor companies that expose their decision logic, not just their performance metrics. Nvidia's press release: https://lnkd.in/dPeUuTWh

25

2 Comments

We've been scaling up our training, yielding the best performing foundational world model to date. Generality is key for robotics, games, and even consumer use in time. The GPUs have been busy, and there's more coming :)

81

7 Comments

Recently the Muon optimizer has shown great promise in LLM pre-training. By performing gradient matrix orthogonalization, Muon can effectively avoid/migitate gradient norm explosions or vanishing. In collaboraiton with Prof. Sijia Liu, recently my students Ruijie Zhang, Yequan Zhao, Ziyue Liu, Zhengyang Wang, Dongyang Li and Yupeng Su developed and implemented Teon, a tensorized version of Muon, to improve LLM pre-training efficiency & accuracy. The key idea is to stack multiple gradient matrices together and perform mode-i orthogonalization of the resulting higher-order tensor, which can effectively explore the correlation among the gradient matrices. When tested on GPT-2 and LlaMA models, Teon shows consistent improvement over the conventional Muon baselines. They also provided theory to explain why "tensorization" can offer benefits, and how to maximize the benefit during the tensorization process. Paper link: https://lnkd.in/gpq-kxmW

124

What would you do if you could train your own robot to do the work you want done? This week Reimagine Robotics came out of stealth. We build robots that people train on the job. Show the robot the task. If it gets it right, great. If it slips, you correct it on the spot and move on. Every skill it learns syncs across the fleet. AI insiders call this post-training: shaping raw model capability into a reliable product. Most labs do it in the lab. We do it in the field, on real work. It's the whole reason the company exists. The result: new behaviours in minutes, not days. No code, no specialist programmer. R2 was founded by Jonathan Scholz, Oleg Sushkov, Misha Denil and Akhil Raju, the team behind Google DeepMind's Applied Robotics group. We're already up and running with customers in London and Sydney. Our CEO Jonathan explains more 👇 Link to our site in the comments.

204

17 Comments

The critical trade-off between throughput vs. sample off-policyness has always been central in RL, from robotics to video games, I have seen it myself multiple times as a key design driver. But at the current scale of large foundation models, the centrality of its role in RL-based post-training efficiency has become even more evident. The high level architecture in the picture describes the custom system adopted by MiniMax in training their M2.5 model released a few days ago (link in the comments). They claim that this element alone provided an approximate 40x(!) training speedup. There are so many variables at play in these applications, and so much space for improvement, I cannot think of a more exciting space to work in. 

27

1 Comment

All of a sudden, hardware startups have become hot with VC funds, especially where they intersect with AI, compute infrastructure, semiconductors, robotics, and advanced manufacturing. Software developer teams seem poised to be gradually replaced by AI agentic teams as the industry shifts toward open source -> with AI agents building apps while we sleep. Just a gentle example: Kelly Claude is now building 12+ products per day. Autonomously. Welcome to the future of software building. And by the way, you can find Kelly Claude on Moltbook. For those still living in the past, Moltbook is like a “Facebook” for AI agents.

15

Python roboticists, we’re coming for you 🙂 We just added Python task support to copper-rs (our Rust robotics runtime). You can now iterate quickly in Python, directly inside the system, without giving up determinism, observability, or safety with no need to rebuild everything from scratch later. When it’s time to move to Rust, you only touch the part that matters! Curious? Take a look at the release. #python #ros #ros2 #robotics

57

I’m noddling on a thought experiment. A 1GW AMD Helios deployment that is used to serve agentic inference workloads. I’m assuming one modern model is being used by all customers to simply things. Each Helios has 18 trays. Each tray has one EPYC and four MI455X. Each tray has a specific amount of DRAM, HBM and NVMe capacity. For now let’s assume these are the only locations KV Cache blocks can live. Let’s assume vLLM and lmcache and NIXL for now. We can argue that choice later. Let’s assume lmcache can know at any time t how many total KV Cache blocks exist (the working set at time t) and where each block lives. Let’s also assume NIXL can share some blocks on any node A with any other node B. But perhaps not all permutations of source and destination have a transfer path. We can explore how things change as we enable and disable certain transfer paths. Now let’s assume at t=0 we turn this 1GW system on. And allow users to start sessions. We can model many users with many turns with a distribution of time between turns and between users starting and finishing. We can have agentic sessions and human. We can model ISL and OSL. Now the question. How does shareablity of KV Cache blocks improve our key figures of merit. Let’s assume for now our top FoM is token/s/gpu. We can argue this later. My conjecture is the more shareable we can make our KV cache blocks the better our FoM becomes. As long as the transfer time between nodes stays bounded. But I also conjecture to get best FoM we need to be able to share KV blocks on node As NVMe. Because this is where most KV Blocks will live (since that is where the most capacity is). I’m working with Claude on a simulator for this. But it’s very complex. Lots of it depends ons. But I think it’s a fun problem very relevant to my work at AMD. Stay tuned for more. Discussion welcome.!

59

22 Comments

What’s the Difference Between a Microkernel and a Monolithic Kernel? 🤔 At the heart of every operating system lies its kernel — the core layer that connects software with hardware. But not all kernels are built the same way. The two main designs that shape how operating systems function are the Microkernel and the Monolithic Kernel. Each follows a different philosophy, influencing performance, safety, and reliability. Let’s understand how they differ 👇 In a Microkernel architecture 🧩, only the most essential functions run inside the kernel — things like memory management 💾, CPU scheduling 🧠, and inter-process communication 🔄. All other services such as device drivers, file systems, and network stacks are moved out of the kernel and run as separate user-space processes. This modular design brings a huge advantage: stability and safety 🛡️. If one service crashes, it doesn’t affect the entire system — it can simply be restarted. That’s why microkernels are often used in safety-critical environments like cars, airplanes, or medical systems, where reliability is non-negotiable. However, this design has a cost — since components must communicate across user and kernel boundaries, it can lead to slightly slower performance due to message passing and context switching. Operating systems like QNX, Integrity, and seL4 are great examples of microkernel-based designs, widely used in automotive and embedded systems that demand functional safety and real-time performance. In a Monolithic Kernel architecture ⚙️, most of the operating system’s services — including device drivers, file systems, and networking — run together inside a single large kernel program. Everything shares the same memory space, allowing functions to communicate directly with one another without switching modes. The biggest strength here is speed and efficiency ⚡. Since there’s no need for message passing between user and kernel spaces, operations happen much faster. This is why systems like Linux, UNIX, and older versions of macOS rely on a monolithic kernel — it’s perfect for general-purpose computing, servers, and performance-driven applications. But the downside is clear: if one driver or module fails, it can potentially crash the entire system 😬. Maintenance and debugging are also more complex since everything is deeply interconnected. In simple terms, Microkernels focus on safety and modularity, while Monolithic kernels focus on performance and efficiency. 🚗 Automotive systems like QNX use microkernels to isolate faults and meet ISO 26262 safety standards, while Linux powers infotainment systems and general-purpose platforms with raw performance. Both are brilliant in their own way — it’s not about which one is better, but which one fits your system’s needs best! 💡 #OperatingSystems #Microkernel #MonolithicKernel #QNX #Linux #EmbeddedSystems #AutomotiveSoftware #RTOS #FunctionalSafety #SystemsDesign #SoftwareArchitecture #TechExplained #LearnWithChetan

14