Writing · 19 posts
Writing
How generative models work, from the architecture down to the GPU kernels, with code and benchmarks.
- RoPE: The Phase Geometry Behind Long-Context TransformersHow rotating query and key vectors turns absolute position into relative attentionLLMs
- From Greedy to Nucleus, How LLMs Choose the Next TokenDecoding algorithms and sampling techniques explainedLLMs
- MatmulA tiled CUDA matmul kernel with shared-memory caching, benchmarked against Torch.GPU kernels
- 2D WorkloadsMapping images onto CUDA thread blocks: row-major memory, 2D indexing, and a grayscale conversion kernel.GPU kernels
- Fused Softmax::P3::Cuda KernelA CUDA softmax kernel taken from a naive per-row loop to shared memory, float4 loads, and warp-shuffle reductions.GPU kernels
- Fused Softmax::P2::Triton optimizationDebugging a Triton softmax that falls off a cliff at large row sizes, and what block size has to do with it.GPU kernels
- Fused Softmax::P1::Naive & Triton ImplementationSoftmax in Torch and a first Triton kernel, starting from the numerically stable formulation.GPU kernels
- Vector Addition::P4::OptimizingVectorized memory access and a grid-stride loop bring the CUDA vector-add kernel up to Triton and Torch speed.GPU kernels
- Vector Addition::P3::BenchmarkingBenchmarking the CUDA, Triton, and Torch vector-add kernels on a GTX 1080, RTX 5060, A100, and H100.GPU kernels
- Vector Addition::P2::Cuda KernelThe same vector addition written in raw CUDA: grids, blocks, threads, and host-to-device memory transfer.GPU kernels
- Vector Addition::P1::Triton KernelA first Triton kernel: vector addition, with pointers, blocks, and masking explained line by line.GPU kernels
- The Command Line is Learning to TalkA deep dive into Gemini CLI, Claude CLI, and the rise of shell-native AI agents.LLMs
- Breaking the Quadratic BarrierAn Engineer's Guide to the Attention Hacks Behind Million-Token LLMsLLMs
- Attention Isn't All You Need. You Also Need a Memory BudgetAn Engineer's Guide to GQA and FlashAttention: The Hacks That Tamed the Transformer's Memory ProblemLLMs
- From Word2Vec to BERT: The Evolution of Language EmbeddingsHow we taught machines to understand not just words, but the meaning behind themLLMs
- From Brute Force to Finesse: The Story of Parameter-Efficient Fine-Tuning (PEFT)How we learned to teach billion-parameter titans new tricks without rebuilding their mindsLLMs
- RAG: Giving LLMs an External BrainFrom keyword search to semantic retrieval, the journey to grounding models in realityLLMs
- How MoE Powers Modern LLMsInside the Architecture That Trains Billion-Scale Models Without Breaking the BankLLMs
- Continuous Batching: The Secret Sauce of High-Throughput LLM InferenceWhy batching in real-time matters, and how vLLM uses it to maximize GPU utilization.LLMs