Wanderson Ferreira

Cloud Network is a Foundation team at Nubank. We own the networking layer every Nubank product runs on, to name a few things: VPC design, cross-realm and hybrid connectivity, transit and egress, global traffic. What makes it interesting: at this scale the question is never "does this route work," it's whether the addressing and routing model still holds as the company adds countries and products on top of it. Compliance lands on the network layer, so we codify it instead of firefighting audits. And everything is declarative and in code, meaning that click-ops is gone and it isn't coming back. The role is a lead level IC. It's for someone who sets technical direction rather than picks up finished designs: leads complex, ambiguous work spanning several teams, owns the operational health of the platform end to end, and raises the engineers around them. If that's the kind of problem you want to spend the next few years on, the opening is linked below. https://lnkd.in/diaqswa5

103

1 comentário

🔥 FineTranslations just dropped: 1 Trillion tokens of parallel text across 500+ languages! Created by Guilherme Penedo and team at Hugging Face, this dataset tackles a major gap: most models struggle with English→X translation, especially for low-resource languages. 💡 The insight: Instead of translating English content into other languages, they started with authentic non-English web pages from FineWeb2 and translated TO English. This creates the perfect parallel data for training better English→X models. 🎯 Key features: - 500+ languages with document-level translations - Smart chunking preserves context across sentences - Fully open & reproducible (ODC-By license) Use it for translation model fine-tuning, multilingual LLMs, or even English pre-training.

54

4 comentários

Apache Airflow 3.x continues its steady evolution 💪 The new 3.1.4 release delivers a focused set of bug fixes and stability improvements. #ApacheAirflow #DataEngineering #MLOps #Orchestration #ETL #Airflow

7

New blog post! How Would You Like Your Iceberg Sir? Stream or Batch Ordered? Many teams today are trying to unify their streaming and batch pipelines over Apache Iceberg. But physical unification comes with trade-offs: 🔹 Flink streaming mode and other stream processors want stream-order (temporal order). 🔹 Spark and analytical engines want batch-order (value clustering for pruning). In my latest post, I explore why these layouts are fundamentally at odds and how Apache Fluss and Confluent Tableflow each navigate this tension in very different ways. https://lnkd.in/dwKf8Fua

71

6 comentários

🐍🎧 What bottlenecks were preventing NumPy from scaling on free-threaded Python? Christopher Trudeau is back with another batch of PyCoder's Weekly articles and projects. https://lnkd.in/drGyvEux

1

Built a weekly Research Assistant Agent for the <Awesome Agentic Machine Learning> repo so that no more manual repo maintenance is needed: It now: (1) scouts Grok social signals + arXiv RSS (2) uses gpt-5.3-codex (xhigh) with live web search for validation (3) opens a draft PR with README.md updates when high-confidence items are found (4) creates an issue log when there’s nothing worth adding Repo Agent: https://lnkd.in/gXEjsMVK

33

🧠 SQL-of-Thought: Multi-Agent Reasoning for Databases (NeurIPS 2025 DLC Workshop) 🔗 https://lnkd.in/gDZM6NyW? The work, linked above, tackles a long-standing challenge in Text-to-SQL: turning natural language into executable queries that are both correct and explainable. Instead of relying on a single model, SQL-of-Thought builds a multi-agent pipeline where each agent has a defined architectural role: • Schema Linking Agent extracts relevant tables, keys, and relationships • Subproblem Agent breaks the query into clause-level tasks (WHERE, GROUP BY, etc.) • Query Plan Agent uses chain-of-thought reasoning to design the logical plan • SQL Agent executes that plan • Correction Agents analyze failures through a guided error taxonomy to fix logical bugs (not just syntax errors) The result? A self-correcting, interpretable system that hits 91%+ execution accuracy on Spider, setting a new state of the art for NL2SQL models. What’s really elegant here is how they merge reasoning (chain-of-thought) with structure (taxonomy-guided correction). Instead of retrying blindly, the agents diagnose and repair errors systematically. A nice glimpse into what agentic AI for databases could look like in the near future. ⚙️ And if you want to see querying done differently (in production!) From OLTP to OLAP: Supercharging Queries on SingleStore 📅 November 3, 2025 | 🕤 9:30 PM IST In this joint session by the Presto and SingleStore teams, you’ll see how federated query engines and distributed SQL architecture can make real-time analytics on live operational data a reality, no ETL, no lag. Think of it as taking the spirit of SQL-of-Thought, smarter orchestration, zero-latency insights, and applying it to your enterprise data stack. Join for the deep dive, stay for the live demo: https://lnkd.in/g3VjKKq3

53

1 comentário

Importance sampling is a concept that appears constantly in RL research (e.g., PPO objective, training / inference mismatch, and more). Here’s how it works… In RL, the policy used to generate rollouts does not always perfectly match the current policy we are optimizing. For example, algorithms like PPO may perform multiple policy updates over the same rollouts, while recent asynchronous RL infrastructure may lead to the incorporation of mildly stale or off-policy data into the training process. Importance sampling is a general technique in probability theory that can help to correct mismatches of this kind. Formally, importance sampling allows us to estimate an expectation under a target distribution f(x) using samples drawn from a different proposal distribution g(x). Instead of sampling directly from f(x), we can sample from g(x) and correct for the discrepancy between these distributions using the importance ratio f(x) / g(x). Intuitively, samples that are more likely under f(x) than g(x) receive a larger importance ratio, while samples that are less likely receive a smaller ratio. Importance ratios appear constantly in research on RL for LLMs. For example, PPO uses an importance ratio to compare the probability of a sampled token under the current policy and the policy that sampled the rollout—this ratio is a core component of the PPO loss function.  Additionally, if rollouts are generated using a policy that is different than the policy being optimized—or a separate inference engine that produces slightly different token distributions—we can use importance ratios to account for this mismatch. In practice, the importance ratio can become large when the distributions differ substantially, yielding unstable and high-variance estimates. For this reason, RL algorithms may choose to clip or truncate the importance ratio, which introduces bias in order to reduce variance and improve stability. For example, both PPO and truncated importance sampling (TIS) adopt a clipped version of the importance ratio.

308

9 comentários