Ritesh Yadav
Ritesh Yadav
/device-software

Device Software

Software that runs on the GPU, the "device" in NVIDIA's lingo: the CUDA programming model, PTX/SASS, threads, warps, and on-device memory.

Posts

  • Warp Tiling GEMM, dividing an output tile among four warps and reaching 40.85 TFLOPS on H100 with FP32 arithmetic.
  • Fused RMSNorm + SwiGLU, why transformer inference benefits from fusion, how my FP16 CUDA kernel works, and what the H100 benchmark measures.
  • CUDA Kernel, what a kernel is, how launches and indexing work, and how to move from naive to tiled compute.
  • Warp Reduction, how 32 threads collapse values in five steps using register shuffles without shared memory.
  • Block Reduction, scaling reductions across hundreds of threads using shared memory and a warp-level finish.
  • Tiled Matrix Multiplication, in-depth anatomy of shared memory tiling, arithmetic intensity, memory coalescing, and step-by-step trace of a 2D tiled GEMM kernel.
  • 1D Block Tiling, accelerating matrix multiplication with register-level reuse and 1D thread tiling.

About the author

Ritesh Yadav works as an AI/ML Engineer. He writes independent research notes on ML performance, infrastructure, and systems, covering CUDA, low-latency inference, generative AI, distributed training, Kubernetes, and LLMOps.