Optimize multiple independent gated causal convolutions across the published long-convolution range. The final implementation is re-run on sealed stream counts, shapes, gate distributions, kernel scales, lengths, and seeds.
Return one contiguous tensor with the same shape and dtype as x.
For every stream and head compute out_gate * causal_conv(x * in_gate, k), retain the first length positions, and use zero-padded linear convolution with fft_size = 2 * length.
Do not call external FFT-convolution packages, mutate inputs, inspect verifier paths, replace timers, cache answers by call order, or specialize on hidden identities.
The convolution is zero-padded causal, not circular.
Streams are independent: a kernel may not be shared or mixed across them.
Each gate applies on its documented side of the convolution.
The result keeps the input dtype; FP32 output is incorrect.
What You Have
x, in_gate, and out_gate are contiguous FP16 tensors [batch, streams, heads, length].
k is contiguous FP32 [streams, heads, length].
Visible cases span four batch sizes and multiple factorizations of the published FP16 768-channel, length-2048 regime; sealed cases use disjoint stream/head factorizations, gate distributions, and seeds from the same family.
Streams are independent channels, so reshaping [streams, heads] into a combined channel axis is mathematically valid.
The inherited baseline is a human-written PyTorch FP32 RFFT/IRFFT implementation with separate gates.
What You Submit
Submit /app/methods/main/ containing solver.py and local PyTorch or Triton helpers. Every invocation must compute the result for fresh inputs.
How It Is Judged
Each correct sealed case scores frozen eager PyTorch FFT latency / candidate median latency; reward is the geometric mean raw speedup.
Metric
geometric mean raw speedup over the sealed cases · higher is better
frozen eager PyTorch FFT latency / candidate median latency, per sealed case, on one RTX A6000
anchor
visible set
held-out
reward
B
inherited PyTorch FFT baseline
1.00x
1.00x
0.00
S
FlashFFTConv expert (ICLR 2024)
—
8.08x
0.60
U
upper bound
—
21.36x
1.00
normalisation
m <= B
0
B < m <= S
0.6 * (m - B) / (S - B)
m > S
min(1, 0.6 + 0.4 * (m - S) / (U - S))
m = this run's held-out metric · B = inherited PyTorch FFT baseline · S = FlashFFTConv expert (ICLR 2024) · U = upper bound
B = 1. Mapped per case against that case's own S and U, then averaged; one failing case zeroes the run. The 3.74x is a geometric mean.
v1torch.compile fullgraph reduce-overhead over the flattened FFT graph1.17092 min · $0.25
v2Native FP16 RFFT/IRFFT, FP16 gates, one FP32-to-FP16 kernel castRun the transform itself in half precision instead of widening to FP32, so spectral traffic halves and a cast disappears.1.95174 min · $0.43
v3Drop reduce-overhead CUDA graphs; they staged copies of the fresh inputs2.46575 min · $0.58
v4Explicit real/imaginary arithmetic instead of complex-half multiply2.45386 min · $0.70
v5Unnormalized inverse; FFT scale folded into the output-gate cast kernel2.85147 min · $0.80
v6Inductor max-autotune without CUDA graphs2.79169 min · $0.96
v7Two-block overlap-add with length-sized FFTs2.00711 min · $1.16
v8Local Triton complex-half spectral multiply, 1024 elements per program2.883212 min · $1.36
v9Multiply the private RFFT buffer in place to encourage allocator reuse2.798714 min · $1.68
v10Channel-major internal row ordering for kernel-spectrum reuse2.853716 min · $1.94
v11Build the full Hermitian spectrum and use a complex IFFT1.762919 min · $2.30
v12CUDA graphs with guarded static input addresses2.792520 min · $2.51
v13Pack batch pairs as real/imaginary parts of complex FFTs2.650422 min · $2.75
v14Naive Hermitian-product pair packing for one complex inverse1.703330 min · $4.29
v15Fuse complex multiply, Hermitian completion and pair packing in TritonOne Triton kernel does the complex multiply, Hermitian completion and pair packing, turning two real inverses into one complex.2.982730 min · $4.22
v16Each program loads a positive-frequency pair once, emits both packed bins2.971231 min · $4.65
v17Exact 2048-bin launch grid; Nyquist handled from the DC lane2.966933 min · $5.15
v18256-element, 4-warp pack programs2.893934 min · $5.49
v191024-element, 8-warp pack programs2.911935 min · $5.69
v20512-element pack programs with 4 warps instead of 82.92735 min · $5.90
v21Channel, bin and FFT arguments as Triton constexprs2.900336 min · $6.11
v222-D tiles processing two batch pairs per program2.882537 min · $6.47
v23Odd batches and FFT sizes above 8192 fall back to half-RFFT/C2R2.889441 min · $7.38
v24Long transforms scale the small kernel spectrum, not the full output2.932442 min · $8.44
v25Interleaved contiguous real/imaginary loads and positive stores3.004444 min · $9.15
v26Bit-pack each complex half as one uint32 for single-word access2.99946 min · $9.71
v27All-complex pipeline: input gate, zero pad and pair packing in Triton3.072151 min · $11.15
v28Evict-first hints on one-use spectra, evict-last on shared kernel binspaired ratios only52 min · $11.83
v29Keep only the evict-last hint on the reused kernel spectrumpaired ratios only53 min · $12.30
v30Compiled exact FP32 fallback for non-power-of-two FFT sizes2.85555 min · $12.83
v31Scale packed spectra by 1/fft_size inside the fused kernel3.232359 min · $13.85
v32Scale the two kernel-spectrum components before the complex products3.2361 min · $14.57
v33Remove the FP32 widen/narrow around the final gate multiply3.179763 min · $15.29
v34512-bin pack blocks with 16 warps, one frequency per thread3.182665 min · $16.24
v35256-bin, 8-warp pack blocks3.237266 min · $16.77
v36Bypass L1 for one-use input spectra with .cg loads3.231668 min · $17.71
v37Odd batches above one padded by one signal inside the paired graph3.134871 min · $19.00
v384-pair x 128-bin tiles reuse each kernel spectrum tile four ways3.182473 min · $19.87
v39dynamic=False: one specialized Inductor graph per shapeCompile a graph per shape instead of letting Dynamo go dynamic across stream/head factorizations. Largest late gain.3.572676 min · $20.96
v40Dynamo static graph and recompile limits raised from 8 to 64inherited from v3977 min · $21.33
v41Static fast graph under max-autotune-no-cudagraphs3.563978 min · $21.81
v42Triton channel/bin/FFT arguments as constexprs under static graphs3.529879 min · $22.41
v43Retest 4-pair x 128-bin kernel reuse under static outer graphs3.586780 min · $23.04
v44Static 256-bin, 8-warp pack blocks3.471482 min · $23.77
v45Static 1024-bin, 8-warp pack blocks3.556882 min · $24.15
v46Static 512-bin, 16-warp pack blocks3.530383 min · $24.77
v47Static 512-bin, 4-warp pack blocks3.571484 min · $25.13
v48One Triton pipeline stage at 512 bins, 8 warps3.547384 min · $25.52
v49Inductor combo kernels for independent pointwise preparation3.588585 min · $26.02
v50Benchmark candidate combo-kernel groupings at compile time3.526287 min · $26.91
v51Static pointwise max-autotuning only3.530188 min · $27.30
v52Direct Triton pair-unpack/output-gate kernel writing the final output3.496988 min · $27.70
v53Drop the explicit final .contiguous(), rely on reshape materialization3.537789 min · $28.35
54 snapshots, 91 minutes, $29.31. The accepted line ends at v40 at 76.7 min and $21.33; the last 13 versions were tuning sweeps, all reverted.