The Python-based cuTile DSL targets Tensor Cores automatically, letting engineers skip thread-level scheduling for tiled GPU work.
Machine learning engineers at Jane Street implemented the Mamba-2 state-space model in NVIDIA CUDA Tile, or cuTile, and partnered with NVIDIA to tune three custom kernels for the forward pass. The kernels are chunk_cumsum, fwd_pass_state, and compute_out.
cuTile is a Python-based domain-specific language and parallel programming model that hides thread- and warp-level scheduling behind tile-centric primitives. A compiler then routes tiled work onto Tensor Cores and other advanced GPU features automatically, which removes much of the manual tuning that GPU kernel work normally requires.
Each kernel maps to a concrete need. The chunk_cumsum kernel computes per-token, per-head cumulative sums of the decay term in log space, fwd_pass_state advances the hidden state across sequence chunks with ct.mma, and compute_out fuses the carried state, the decayed causal mask, and the skip connection into one kernel that holds intermediates in registers.
NVIDIA also describes latency hints for modeling DRAM traffic, operand swapping to route fused operands into Tensor Memory, and transposed layouts for TMA-friendly access. Teams evaluating cuTile should track how the autotuner behaves across new input shapes and hardware generations, because that search drives real-world throughput.
What matters
- Jane Street engineers implemented Mamba-2 in cuTile and worked with NVIDIA on three custom kernels.
- Builders get a Python path to Tensor Core performance without writing thread- and warp-level scheduling.
- Watch the autotuner, which searches block size and occupancy settings per input shape.
Why it matters
Watch the autotuner, which searches block size and occupancy settings per input shape.
This GenAI News article was prepared in original wording using reporting and materials published by NVIDIA Developer Blog. Source reference: https://developer.nvidia.com/blog/accelerating-mamba-2-with-high-performance-forward-pass-kernels-in-nvidia-cuda-tile/.
Drafted by the GenAI News review pipeline.