[QST] why can't use ColumnMajor as sparse GEMM output layout?

[shicheng0829]

Recently, I used cutlass Sparse Gemm to implement the pytorch linear layer.

In pytorch, y = x @ w^T + b, as w is a sparse matrix.

Sparse Gemm represents D = A * B + C as A is a sparse matrix.

So I must use (w @ x^T)^T to compute x @ w^T.

However, I found that the sparse GEMM output layout can't be ColumnMajor from this file. So I can only compute the transpose of the real result.

cutlass/python/cutlass_library/generator.py

Line 2006 in 5e1a0a5

def GenerateSM80_SparseTensorOp_16832(manifest, cuda_version):

I want to know how I can solve this problem, Thanks.

@hwu36

[hwu36]

Cutlass 2.x only has row major epilogue for both dense and sparse gemm. If we want to output column major for dense gemm, we do this trick: C:row = A x B => C:col = B' x A'. If we apply this trick to sparse gemm, then B' becomes sparse which means B is sparse which changes the original problem.

What you can do is like use a naive epilogue which is like https://github.com/NVIDIA/cutlass/blob/main/include/cutlass/gemm/warp/mma_tensor_op_tile_iterator.h#L2976 or you can write an efficient column major epilogue like cutlass 1.x did.

[github-actions]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[mnicely]

Closing due to inactivity. Please reopen if needed.