Ritesh Yadav
Research Notes
Device Software
Software that runs on the GPU, the "device" in NVIDIA's lingo: the CUDA programming model, PTX/SASS, threads, warps, and on-device memory.
Posts
- Warp Tiling GEMM, dividing an output tile among four warps and reaching 40.85 TFLOPS on H100 with FP32 arithmetic.
- Fused RMSNorm + SwiGLU, why transformer inference benefits from fusion, how my FP16 CUDA kernel works, and what the H100 benchmark measures.
- CUDA Kernel, what a kernel is, how launches and indexing work, and how to move from naive to tiled compute.
- Warp Reduction, how 32 threads collapse values in five steps using register shuffles without shared memory.
- Block Reduction, scaling reductions across hundreds of threads using shared memory and a warp-level finish.
- Tiled Matrix Multiplication, in-depth anatomy of shared memory tiling, arithmetic intensity, memory coalescing, and step-by-step trace of a 2D tiled GEMM kernel.
- 1D Block Tiling, accelerating matrix multiplication with register-level reuse and 1D thread tiling.
About the author
Ritesh Yadav works as an AI/ML Engineer. He writes independent research notes on ML performance, infrastructure, and systems, covering CUDA, low-latency inference, generative AI, distributed training, Kubernetes, and LLMOps.