---
title: "Carnegie Mellon just open-sourced a Blackwell GPU programming book. Free."
url: https://stacklist.com/card/82a37bc5-6c75-4816-b867-32f0d2532a52
source_url: "https://www.linkedin.com/posts/paoloperrone_carnegie-mellon-just-open-sourced-a-blackwell-share-7476106428521504768-hSuI/?utm_source=share&utm_medium=member_ios&rcm=ACoAAAI21ZsBNnZPaKuTab7nquKLCveUW7o-1DE"
stack: https://stacklist.com/stack/4aae218c-38d7-4c05-b7ae-f2db0029a2e8
summary: "Carnegie Mellon has open-sourced a Blackwell GPU programming book created by Tianqi Chen that covers modern GPU architecture, data layout, kernel writing, and hands-on examples for high-performance computing. The resource addresses the gap between outdated GPU programming practices and current hardware capabilities, emphasizing that kernel-level optimization is becoming essential for production AI systems."
tags: "gpu-programming, blackwell-architecture, kernel-optimization, high-performance-computing, machine-learning-systems, nvidia, open-source"
key_entities: "Tianqi Chen (person), Paolo Perrone (person), Carnegie Mellon University (organization), NVIDIA (organization), Blackwell GPU (technology), XGBoost (technology), Flash Attention (technology), kernel-optimization (concept), data-swizzling (concept), TMA (concept)"
classification: "reference"
content_hash: "sha256:f88bf8f385e921c1d2823cc1c629c7bfd429d194115a13cdac4aee92f62571c8"
acp_version: "0.2"
token_counts_approximate: 1053
visibility: public
agent_accessible: true
status: "final"
---

# Carnegie Mellon just open-sourced a Blackwell GPU programming book. Free.

Paolo Perrone 1w Report this post Carnegie Mellon just open-sourced a Blackwell GPU programming book. Free. Most GPU courses still teach decade-old material. Blackwell is a completely different architecture. Tianqi Chen (XGBoost creator, NVIDIA Distinguished Engineer, CMU professor) built this with his course staff. What it covers: → Data layout and data swizzling → 3D TMA for one-shot tiling and swizzling → High-performance kernel writing for modern GPUs → Hands-on examples with a minimal compiler This started as a crash course in CMU's ML Systems class. Now it's a free online book with interactive materials. The teams treating kernel work as optional are the ones paying 10x more for inference and wondering why their agents feel slow. Free book https://lnkd.in/d3_ZEHTf ♻️ Repost for the engineer still writing kernels like it's 2019. 1,783 59 Comments Like Comment Share Copy LinkedIn Facebook X Paolo Perrone 1w Report this comment 📌 The free book is the door. The career is behind it. I cover the path twice a week. Free → theaiengineer.substack.com Like Reply 2 Reactions 3 Reactions Deependra Verma 1w Report this comment Great read. High-performance LLM inference isn't just about better models—it's about understanding how attention maps to GPU primitives. This guide bridges that gap brilliantly. Flash Attention. Understanding the pipeline, synchronization, and online softmax rescaling makes the kernel design far less "magical." Definitely bookmarking this. Like Reply 9 Reactions 10 Reactions Ashish Patel 🇮🇳 1w Report this comment This is a critical resource. We've reached a point where 'model-agnostic' optimizations are becoming the primary driver of production ROI. For the last two years, the focus has been on scaling up parameters; for the next two, the focus is undoubtedly scaling down latency and cost through hardware-aware software engineering. The gap between generic implementations and custom kernel-level optimizations is the difference between a project that is a 'cost center' and one that can scale to millions of users profitably. Building at the intersection of Flash Attention 4 and modern GPU primitives isn't just an 'engineering exercise' anymore—it's a core business strategy. Does anyone see this level of hardware-aware optimization becoming an 'optional' skill, or is this the new baseline for production AI teams? Like Reply 3 Reactions 4 Reactions Franck Hermann Debah ZROUAMA 1w Report this comment Isaac emmanuel YAO Like Reply 1 Reaction 2 Reactions Ignazio De Santis 1w Report this comment Bookmarked. But the "learn kernels or pay 10x" framing is true only at a certain scale. For most teams the cheaper win is still batching, caching, and right-sizing the model before anyone hand-writes a kernel, and Blackwell-specific tricks rot fast as the architecture moves. The durable skill the book actually teaches isn't the swizzle pattern, it's learning to think in terms of memory movement instead of FLOPs. Like Reply 1 Reaction Manny Ko 1w Report this comment Tianqi is one of the handful of academics that also is a low level programmer. Truly rare. Like Reply 5 Reactions 6 Reactions Tom Sullivan 1w Report this comment This looks like an amazing resource, thanks for sharing! Like Reply 1 Reaction Dr. Martin Schiele 1w Report this comment Great initiative! Staying up-to-date with GPU advancements is critical. Always amusing to see those sticking to outdated methods and then complaining about inefficiencies. Thanks, CMU! Like Reply 1 Reaction Alex Cinovoj 1w Report this comment This is where the next cost advantage shows up. Everyone talks about model choice, but kernel-level efficiency is what separates fast agents from expensive demos. Like Reply 3 Reactions 4 Reactions Michał Piszczek 1w Report this comment Good resource, but kernel syntax is the part that expires every generation. What transfers from Hopper to Blackwell and beyond is roofline intuition, knowing whether you are memory bound or compute bound before you write a line. Learn that and the API churn stops mattering Like Reply 1 Reaction See more comments To view or add a comment, sign in