X-Stage: A New Pipeline Stage for Communication-Computation Overlap in DiT Inference

Researchers have discovered a previously unnoticed stage in the pipeline called X-Stage, which is part of distributed diffusion transformer (DiT) inference. This software-visible phase takes place after communication is initiated but prior to the completion being visible remotely. Tests conducted on an eight-GPU node utilizing a recent NVIDIA architecture demonstrate that brief remote-store bursts deplete resources as the issuer continues its tasks, while prolonged injection uses up limited outstanding capacity, causing delays in subsequent issues. A lightweight Burst-Gap model, defined by backpressure-free issue time, effective drain rate, and outstanding capacity, forecasts issue overhead, recovery times between bursts, and the onset of backpressure. This model informs the redesign of two communication methods to enhance overlap and minimize inference latency. The study addresses a gap in current systems that manage when communication is sent and when the received data can be used, neglecting progress after issuance, which complicates the prediction of sender backpressure. The results are published in a paper on arXiv (2607.23264).