Hand-Written PTX GEMM Kernels on NVIDIA L4: No Speedup for FP16

A recent study published on arXiv (2608.10103) examines the advantages of hand-written PTX Tensor Core kernels versus the WMMA C++ API on an NVIDIA L4 GPU (Ada, SM89). This controlled experiment utilized a single GPU to evaluate double-buffered WMMA baselines against hand-crafted PTX GEMM kernels for FP16, INT8, and INT4 arithmetic, with square matrix sizes ranging from N=512 to N=8192. Profiling was conducted using Nsight Compute, reporting PTX speedups against WMMA baselines of the same precision. The main conclusion indicates that hand-written PTX does not enhance speed for FP16 due to instruction-level limitations. The paper raises a critical question regarding the conditions under which substituting WMMA with hand-written PTX is beneficial. For FP16, the drawbacks surpass the advantages, while potential improvements exist for lower precision (INT8, INT4), although specifics are truncated. This research is significant for high-performance computing and GPU programming, especially in applications utilizing Tensor Cores. The authors have made the complete findings accessible via the provided URL.