Racer: Hunting GPU Kernel Bugs with Symbolic Equivalence Checking

Key points

• Today, this obligation depends almost entirely on differential testing: run the new kernel and a trusted reference on randomly sampled tensors, then compare their outputs within a tolerance.1
• For example, a logical bug may produce an error smaller than the tolerance required to accommodate a correct optimized kernel.1
• We propose symbolic equivalence checking, which executes a kernel and its reference on symbolic inputs, turning each output cell into an exact algebraic expression, and then checks whether the paired expressions agree for all inputs.1
• We implement Racer, a PTX-level symbolic equivalence checking engine, with three techniques that make this approach practical for production kernels: (1) Shape Reduction to control checking cost, (2) a lockstep CTA scheduler to handle concurrency, and (3) idiom templates with exact numeric refutation for otherwise inconclusive results.1

Sources (1)

• [1]Racer: Hunting GPU Kernel Bugs with Symbolic Equivalence Checking
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 03:58 PM

  GPU kernels are critical to the performance of today's AI services, yet the same operators are rewritten repeatedly: for each GPU generation, numeric format, serving regime, and kernel language.

  Today, this obligation depends almost entirely on differential testing: run the new kernel and a trusted reference on randomly sampled tensors, then compare their outputs within a tolerance.

Extractive summary: sentences quoted from the sources.