Saman Keon on LLM efficiency, quantization, inference, and GPU kernels.https://samankeon.com/https://samankeon.com/blog/rope-the-phase-geometry-behind-long/https://samankeon.com/blog/rope-the-phase-geometry-behind-long/How rotating query and key vectors turns absolute position into relative attentionSat, 06 Jun 2026 00:00:00 GMThttps://samankeon.com/blog/from-greedy-to-nucleus-how-llms-choose/https://samankeon.com/blog/from-greedy-to-nucleus-how-llms-choose/Decoding algorithms and sampling techniques explainedMon, 22 Dec 2025 00:00:00 GMThttps://samankeon.com/blog/matmul/https://samankeon.com/blog/matmul/A tiled CUDA matmul kernel with shared-memory caching, benchmarked against Torch.Mon, 01 Dec 2025 00:00:00 GMThttps://samankeon.com/blog/2d-workloads/https://samankeon.com/blog/2d-workloads/Mapping images onto CUDA thread blocks: row-major memory, 2D indexing, and a grayscale conversion kernel.Sat, 22 Nov 2025 00:00:00 GMThttps://samankeon.com/blog/softmax-kernel-3/https://samankeon.com/blog/softmax-kernel-3/A CUDA softmax kernel taken from a naive per-row loop to shared memory, float4 loads, and warp-shuffle reductions.Sun, 16 Nov 2025 00:00:00 GMThttps://samankeon.com/blog/softmax-kernel-2/https://samankeon.com/blog/softmax-kernel-2/Debugging a Triton softmax that falls off a cliff at large row sizes, and what block size has to do with it.Sun, 09 Nov 2025 00:00:00 GMThttps://samankeon.com/blog/softmax-kernel-1/https://samankeon.com/blog/softmax-kernel-1/Softmax in Torch and a first Triton kernel, starting from the numerically stable formulation.Sun, 02 Nov 2025 00:00:00 GMThttps://samankeon.com/blog/add-kernel-4/https://samankeon.com/blog/add-kernel-4/Vectorized memory access and a grid-stride loop bring the CUDA vector-add kernel up to Triton and Torch speed.Sun, 26 Oct 2025 00:00:00 GMThttps://samankeon.com/blog/add-kernel-3/https://samankeon.com/blog/add-kernel-3/Benchmarking the CUDA, Triton, and Torch vector-add kernels on a GTX 1080, RTX 5060, A100, and H100.Sun, 12 Oct 2025 00:00:00 GMThttps://samankeon.com/blog/add-kernel-2/https://samankeon.com/blog/add-kernel-2/The same vector addition written in raw CUDA: grids, blocks, threads, and host-to-device memory transfer.Sun, 05 Oct 2025 00:00:00 GMThttps://samankeon.com/blog/add-kernel-1/https://samankeon.com/blog/add-kernel-1/A first Triton kernel: vector addition, with pointers, blocks, and masking explained line by line.Sun, 28 Sep 2025 00:00:00 GMThttps://samankeon.com/blog/the-command-line-is-learning-to-talk/https://samankeon.com/blog/the-command-line-is-learning-to-talk/A deep dive into Gemini CLI, Claude CLI, and the rise of shell-native AI agents.Tue, 16 Sep 2025 00:00:00 GMThttps://samankeon.com/blog/breaking-the-quadratic-barrier/https://samankeon.com/blog/breaking-the-quadratic-barrier/An Engineer's Guide to the Attention Hacks Behind Million-Token LLMsTue, 09 Sep 2025 00:00:00 GMThttps://samankeon.com/blog/attention-isnt-all-you-need-you-also/https://samankeon.com/blog/attention-isnt-all-you-need-you-also/An Engineer's Guide to GQA and FlashAttention: The Hacks That Tamed the Transformer's Memory ProblemTue, 02 Sep 2025 00:00:00 GMThttps://samankeon.com/blog/from-word2vec-to-bert-the-evolution/https://samankeon.com/blog/from-word2vec-to-bert-the-evolution/How we taught machines to understand not just words, but the meaning behind themTue, 26 Aug 2025 00:00:00 GMThttps://samankeon.com/blog/from-brute-force-to-finesse-the-story/https://samankeon.com/blog/from-brute-force-to-finesse-the-story/How we learned to teach billion-parameter titans new tricks without rebuilding their mindsTue, 19 Aug 2025 00:00:00 GMThttps://samankeon.com/blog/rag-giving-llms-an-external-brain/https://samankeon.com/blog/rag-giving-llms-an-external-brain/From keyword search to semantic retrieval, the journey to grounding models in realityTue, 12 Aug 2025 00:00:00 GMThttps://samankeon.com/blog/how-moe-powers-modern-llms/https://samankeon.com/blog/how-moe-powers-modern-llms/Inside the Architecture That Trains Billion-Scale Models Without Breaking the BankWed, 06 Aug 2025 00:00:00 GMThttps://samankeon.com/blog/continuous-batching-the-secret-sauce/https://samankeon.com/blog/continuous-batching-the-secret-sauce/Why batching in real-time matters, and how vLLM uses it to maximize GPU utilization.Tue, 29 Jul 2025 00:00:00 GMT