CUDA kernels don't throw exceptions when they're wrong. This harness surfaces the silent failure modes: warp-shuffle reductions returning half the sum, softmax overflowing to NaN on large logits, tile boundaries that break at N=257. Kernels compiled via nvcc, loaded into Python through ctypes, validated against NumPy CPU baselines.
Built and validated a Flash Attention Triton kernel from scratch. Found a non-obvious failure: padding slots in the last K-tile load as 0.0 instead of -inf, so exp(0)=1 sits in the softmax alongside real scores. At N=1 that's 63 fake ones against one real score — output comes out 64x too small. No crash, no warning, just wrong. The harness covers that plus tile-boundary edges, causal-mask integrity across blocks, and FP16 drift at long sequences.
CUTLASS ships hundreds of kernel variants and picks one per shape and dtype at compile time. When no kernel matches a shape, the profiler returns nothing — no error, no warning. This harness drives cutlass_profiler across a shape/dtype matrix, checks results against CUTLASS's own reference GEMM, and maps exactly which shapes get kernels and which silently fall through. M=4095 returns zero kernels; M=4096 returns eight. Batch-1 decode — the real decode step — fails the same way. bf16 and tf32 return zero on T4 (Turing): those tensor cores don't exist until Ampere.
Building the layer where silicon and models have to agree.