Gyuho Lee

“Today’s glory began in the back alleys of yesterday.” Jensen Huang didn’t simply ride waves of AI— he spent 30 years building the ship capable of navigating it. From the electronics markets of Yongsan in Seoul to standing at the pinnacle of the AI era, this article from Kyungjeilbo examines NVIDIA’s journey and the essence of its success through the philosophical lens of “Go (古) · Go (枯) · Go (孤) · Go (高).” This is not a story of a fleeting spark, but of a steady light that quietly illuminates the world. Explore the full story at the link below: 🔗 https://nvda.ws/48VNqLl

10

The news about Nvidia acquiring Groq’s assets and IP triggered a wave of speculation about it's inference roadmap. Tempted to add my 2 cents as well... 😊 Groq’s LPU architecture gives up hardware generality in exchange for a compiler-scheduled, deterministic streaming machine fed by high bandwidth on-chip SRAMs. It is a tiled architecture where interleaved vertical columns of SRAM and compute operate as a unified, synchronous grid. The on-chip SRAM has ~230MB capacity and aggregate bandwidth of ~80 TB/s (compared to ~8TB/s of HBM bandwidth in GB200). At the system level, a Groq rack typically integrates 72 LPUs connected via a switchless, low-diameter fabric (dragonfly topology). But the full rack provides only ~15GB of SRAM. In contrast, an NVL72 rack provides ~900x more memory capacity! The compiler owns instruction scheduling, data movement, and resource allocation across the entire cluster a model maps to - no dynamic scheduling, no cache misses, no network congestion. By treating memory bandwidth and deterministic dataflow as first-class design constraints, this architecture can deliver stable, low-variance latency during the decode phase, as long as the memory capacity does not become a limiting factor! But memory capacity does matter when scaling to frontier models. Storing a TB-class model would require ~70 racks worth of SRAM storage, and this does not include the additional memory needed for KV cache growth at runtime. Thus, the system-level picture changes quickly at scale. Inter-rack cabling, retimers and physical footprint all compound TCO. This is why it makes little sense for Nvidia to "replace" GPUs entirely with LPUs for inference. The memory density needed for the scale is simply not there. The pragmatic, near-term option that Nvidia could embrace, which does not require new silicon, is to absorb Groq’s static scheduling philosophy directly into it's inference compiler and runtime stack. More aggressive graph-level scheduling across kernels, tighter lifetime planning for KV cache and activation buffers, and deterministic fast paths for latency-critical decode all directly attack tail latency on existing GPUs. Medium term, Groq LPUs could connect to GPUs over NVLink (either within a tray or across the scale-up fabric). LPUs could offload speculative decoding (using draft models that reside in SRAMs) and other processing during the decode phase that can benefit from deterministic execution and ultra-low latency. Longer term, Nvidia's inference platforms will increasingly be heterogeneous. With specialized CPX-class accelerators for prefill and GPUs with tightly coupled streaming tiles for decode. Integrating Groq's LPU tiles closer to the GPU, within the package, strengthens the case for advanced 3D SRAM integration - deterministic dataflow can fully exploit the proximity and bandwidth of 3D SRAM? In short: Groq IP and SW stack don’t redefine Nvidia’s inference stack - but they can supercharge it! Any thoughts? 🤔

366

18 Comments

A few months ago our team at Dria was training Mem Agent, a 4B parameter model designed to give any LLM persistent memory through an Obsidian style markdown system. We tried a dozen combinations of base models and RL algorithms before landing on GSPO, a variant of the reinforcement learning methods that are quietly reshaping how every reasoning model gets built today. That experience gave me a deep appreciation for how much the RL training stack for LLMs has evolved in just two years, and how most of the gains came from places nobody expected. The field started with PPO, an algorithm borrowed from robotics that required four separate neural networks in memory just to train one model. Then DeepSeek had a deceptively simple idea: instead of training an entire “critic” model to judge responses, just sample multiple answers to the same question and compare them against each other. That single change (called GRPO) cut memory usage roughly in half and became the backbone of nearly every reasoning model since. But here is what I find most fascinating. The biggest improvements after GRPO were not new algorithms at all. They were subtle bugs hiding inside normalizations that everyone assumed were harmless. One team found that dividing loss by sequence length was secretly rewarding longer wrong answers. Another discovered that normalizing by standard deviation was making models obsess over problems they had already solved. A third noticed that less than 0.5% of gradient updates were causing all the training instability, and blocking just those was enough to fix everything. The pattern is clear: we went from “we need bigger, more complex algorithms” to “we need to understand what our existing algorithms are actually doing at the token level.” This is exactly the philosophy we have been building on at Dria. With Agent Alpha we showed that a 3B model trained with the right synthetic data and RL recipe can match GPT-4o on function calling benchmarks. With Mem Agent we used GSPO to teach a 4B model to read, write and reason over structured memory, rivaling models 50x its size. And with Kai Evolve we took the idea further: instead of a single RL loop, we run evolutionary optimization across hundreds of autonomous iterations, using verification as the fitness function, achieving a 192x speedup on NVIDIA B200 CUDA kernels across 540 generations. The common thread across all of this is that the bottleneck is rarely the absence of a sophisticated solution. It is almost always a subtle misunderstanding hiding inside something that already works. Alexander Weers wrote an excellent technical walkthrough of the full RL evolution from REINFORCE through GRPO and everything that followed. If you work anywhere near LLM training, it is one of the clearest overviews out there: https://lnkd.in/e_ysV-UD

11

The AI Industry Has Been Doing Reasoning Completely Wrong. I Spent 48 Hours Digging Into ByteDance’s Secret Paper… What I Found Will Completely Change How We Build AI Agents Overnight. A couple of days back I said ByteDance Seed is criminally underrated. Today I’m delivering exactly what I promised: the full SMELT teardown + code walkthrough + graphical explanation. 𝗣𝗮𝗽𝗲𝗿: SMELT -- Scaling Laws for Compute-Matched MoE Looped Transformers arXiv: 2609.01343 Authors: Shaowen Wang & team (ByteDance Seed) 𝗧𝗵𝗲 𝗰𝗼𝗿𝗲 𝗿𝗲𝘀𝘂𝗹𝘁 (𝗶𝗻 𝗽𝗹𝗮𝗶𝗻 𝗘𝗻𝗴𝗹𝗶𝘀𝗵) Can you get deeper reasoning from a looped Transformer while keeping: ● FLOPs per token identical ● Total parameters identical ● KV cache size identical Answer: 𝗬𝗲𝘀. And the advantage grows with scale. SMELT delivers: ● 18 executed layers of reasoning depth on a 12-layer physical budget ● ≤3.9% error on active FLOPs ● ≤1.0% error on stored parameters (up to 54B) ● ≤3.6% error on serving KV cache ● Up to +20.4% compute efficiency on code & complex in-context learning 𝗪𝗵𝘆 𝘁𝗵𝗶𝘀 𝗵𝗶𝘁𝘀 𝗮𝗴𝗲𝗻𝘁𝗶𝗰 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 𝗵𝗮𝗿𝗱 Autonomous agents don’t just need longer context. They need 𝗿𝗲𝗰𝘂𝗿𝘀𝗶𝘃𝗲 𝗺𝘂𝗹𝘁𝗶-𝘀𝘁𝗲𝗽 𝗱𝗲𝗽𝘁𝗵 without exploding your serving bill. SMELT gives models ~50% more execution depth while keeping the exact same GPU footprint in production. That changes the economics of agent loops, tool-use chains, and long-horizon planning. I wrote the complete technical deep-dive (mechanistic breakdown + scaling surface + executable PyTorch block-recurrence code) here: 🔗 https://lnkd.in/gYJ7h4Y3 If you’re building agents, reasoning models, or serving infrastructure, this is worth 8-10 minutes. - Ranil Mukesh M J

9

Anysphere is considering investment offers at a $30 billion valuation. The company, known for its coding assistant Cursor, has seen its valuation triple since mid-year, driven by fast-growing revenue. Read more: https://lnkd.in/gjKGd3EJ 📰 Subscribe to the Daily AI Brief: https://lnkd.in/eq56P3bj #aistartup #aifunding #ainews

5

Spotify's best engineers haven't written a single line of code since December. Meanwhile, AT&T's CDO just said fine-tuned small models will be the big trend of 2026. Here's what most people are getting wrong: Everyone is chasing GPT-5.3 and Claude Opus 4.6. But the companies actually winning with AI? They're fine-tuning 7B-14B parameter models that run 10x cheaper and outperform general-purpose LLMs on their specific tasks. I've seen this firsthand. I've fine-tuned domain-specific models for FinTech and HR that beat GPT-4 on task accuracy — at a fraction of the cost. The secret isn't model size. It's: → Training data quality > quantity → LoRA/QLoRA for efficient adaptation → Chain-of-thought prompting baked into fine-tuning → RAG as a complement, not a crutch → Evaluation frameworks that test real-world performance, not benchmarks 2026 is the year "smart beats big." The engineers who know how to make a small model do one thing extremely well will be worth more than the ones chasing the next frontier release. Agree or disagree? Drop your take below. #AI #LLM #FineTuning #MachineLearning #SmallLanguageModels #RAG #AIEngineering #FinTech #DeepLearning

1

1 Comment

India just hosted one of the world's largest AI summits, announcing a staggering ₹10 lakh crore investment in AI over the next seven years 📈. With GPU capacity expanded to 58,000 units and compute priced at just ₹65 per hour, the infrastructure is finally catching up to ambition. The event also showcased homegrown breakthroughs: Sarvam AI's Indic‑language LLMs that outperform global rivals, the launch of BharatGen – India's own multimodal model, and even PM Modi sporting Made‑in‑India AI glasses 👓. The "Create in India" mission promises to build world‑class talent and position the nation as a tech powerhouse. Yet the headlines were dominated by a controversy over a robot dog, turning a historic summit into a gossip cycle. While the world watched India pledge massive AI funding, we were busy debating a professor’s unauthorized comment. This misplaced focus reveals a deeper issue: a collective attention calibrated for scandal, not substance. The uncomfortable truth is clear – a nation gets the media it feeds. If we continue to reward noise over innovation, we will keep lagging behind. Let’s redirect our energy toward the real breakthroughs happening right now. What will you do to support India’s AI ascent? #AI #IndiaAI #TechInnovation #FutureReady #CreateInIndia #BharatGen #SarvamAI Reference: [https://lnkd.in/grE-GRun] 🔄 Share 👍 React 🌐 Visit www.aravind-r.com #AravindRaghunathan

33

3 Comments