too many blocks in cooperative launch at cudaLaunchCooperativeKernel

泰勒朗斯

已于 2024-09-06 14:40:19 修改

阅读量1.1k

点赞数 8

分类专栏： GPU 文章标签：人工智能

于 2024-09-04 16:11:49 首次发布

本文链接：https://blog.csdn.net/weixin_43360707/article/details/141896670

版权

GPU 专栏收录该内容

21 篇文章 1 订阅

订阅专栏

在使用cudaLaunchCooperativeKernel时出现：

cudaErrorCooperativeLaunchTooLarge (error 82) due to “too many blocks in cooperative launch” on CUDA API call to cudaLaunchCooperativeKernel.

问题如下：

I understand I’m using too many ‘active blocks’ and have no argument with that.

What I don’t understand is how to do the math to know how many blocks and threads I can call beforehand.

在使用cudaLaunchCooperativeKernel时，限制其最大grid_dim和block_dim的元素是什么？

A100的关键参数:
在这里插入图片描述
从上面表格中可以看到，影响cooperative launch的max_grid_dim 和max_block_dim的因素有三个：

maximum number of resident blocks per SM
maximum number of resident warps per SM
maximum number of resident threads per SM

对于A100理论上来说，在cooperative launch的时候，有如下限制（寄存器等先忽略）:

block 不能超过 10832=3456
warps不能超过10864=6912
threads 不能超过 108*2048=221184
blocks内的threads不能跨SM (这个是隐含的条件，很多人忘了这个，所以失败)
按照上面条件获得下表，理论上下面表格数据是能cooperative launch成功的，
计算原则就是：
grid_dim * block_dim == 108 *2048，（这个计算没有考虑blocks内的threads不能跨SM）

grid dim	max block dim	threads sum	warp sum
1	1024	1024	32
2	1024	2048	64
4	1024	4096	128
8	1024	8192	256
16	1024	16384	512
32	1024	32768	1024
64	1024	65536	2048
128	1024	131072	4096
256	864（672）	221184（172032）	6912（5376）
512	432（384）	221184（196608）	6912 （6144）
1024	216（192）	221184（196608）	6912（6144）
2048	108（96）	221184（196608）	6912（6144）
3456	64（64）	221184（221184）	6912（6912）

按照上面计算，遍历跑一遍，结果发现实测值和我们上面的计算值不一样，下面表格中的smx列中的值代表，这个SM中有多少个threads
sm上kernel分布：

grid_dim	block_dim	sm0	sm1	sm2	sm3	sm4	sm5	sm6	sm7	sm8	sm9	sm10	sm11	sm12	sm13	sm14	sm15	sm16	sm17	sm18	sm19	sm20	sm21	sm22	sm23	sm24	sm25	sm26	sm27	sm28	sm29	sm30	sm31	sm32	sm33	sm34	sm35	sm36	sm37	sm38	sm39	sm40	sm41	sm42	sm43	sm44	sm45	sm46	sm47	sm48	sm49	sm50	sm51	sm52	sm53	sm54	sm55	sm56	sm57	sm58	sm59	sm60	sm61	sm62	sm63	sm64	sm65	sm66	sm67	sm68	sm69	sm70	sm71	sm72	sm73	sm74	sm75	sm76	sm77	sm78	sm79	sm80	sm81	sm82	sm83	sm84	sm85	sm86	sm87	sm88	sm89	sm90	sm91	sm92	sm93	sm94	sm95	sm96	sm97	sm98	sm99	sm100	sm101	sm102	sm103	sm104	sm105	sm106	sm107
1	1	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
1	64	64	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
1	128	128	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
1	256	256	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
1	512	512	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
1	1024	1024	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
2	1	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
2	64	64	0	64	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
2	128	128	0	128	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
2	256	256	0	256	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
2	512	512	0	512	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
2	1024	1024	0	1024	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
4	1	1	0	1	0	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
4	64	64	0	64	0	64	0	64	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
4	128	128	0	128	0	128	0	128	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
4	256	256	0	256	0	256	0	256	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
4	512	512	0	512	0	512	0	512	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
4	1024	1024	0	1024	0	1024	0	1024	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
8	1	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
8	64	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
8	128	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
8	256	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
8	512	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
8	1024	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
16	1	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
16	64	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
16	128	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
16	256	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
16	512	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
16	1024	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
32	1	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
32	64	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
32	128	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
32	256	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
32	512	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
32	1024	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
64	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1	0
64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0	64	0
64	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0	128	0
64	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0	256	0
64	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0	512	0
64	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0	1024	0
128	1	2	1	2	1	2	1	2	1	2	1	2	1	2	1	2	1	2	1	2	1	2	1	2	1	2	1	2	1	2	1	2	1	2	1	2	1	2	1	2	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1
128	64	128	64	128	64	128	64	128	64	128	64	128	64	128	64	128	64	128	64	128	64	128	64	128	64	128	64	128	64	128	64	128	64	128	64	128	64	128	64	128	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64	64
128	128	256	128	256	128	256	128	256	128	256	128	256	128	256	128	256	128	256	128	256	128	256	128	256	128	256	128	256	128	256	128	256	128	256	128	256	128	256	128	256	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128
128	256	512	256	512	256	512	256	512	256	512	256	512	256	512	256	512	256	512	256	512	256	512	256	512	256	512	256	512	256	512	256	512	256	512	256	512	256	512	256	512	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256
128	512	1024	512	1024	512	1024	512	1024	512	1024	512	1024	512	1024	512	1024	512	1024	512	1024	512	1024	512	1024	512	1024	512	1024	512	1024	512	1024	512	1024	512	1024	512	1024	512	1024	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512
128	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	2048	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024
256	1	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	3	2	2	2	2	2	2	2	2	2	2	2	2	2	2	2	2	2	2	2	2	2	2	2	2	2	2	2	2	2
256	64	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	192	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128	128
256	128	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	384	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256	256
256	256	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	768	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512	512
256	512	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1536	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024	1024
256	672	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	2016	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344	1344
512	1	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4	5	4
512	64	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256	320	256
512	128	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512	640	512
512	256	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024	1280	1024
512	384	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536	1920	1536
1024	1	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	10	9	9	9	9	9
1024	64	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	640	576	576	576	576	576
1024	128	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1280	1152	1152	1152	1152	1152
1024	192	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1920	1728	1728	1728	1728	1728
2048	1	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	18	19	18	19	18	19	18
2048	64	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1216	1152	1216	1152	1216	1152	1216	1152
2048	96	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1824	1728	1824	1728	1824	1728	1824	1728
3456	1	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32	32
3456	64	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048	2048

另外上面这个表格要注意A100是怎么调度blocks的：
看起来首先是把blocks放置在偶数sm id的SM上，接着奇数 sm id，依次轮询layout。

回到问题，实测值和我们计算值不一样?

比如，
当grid dim为256的时候，按照传统计算：
block_dim=（108 * 32 ）/ 256 = 864（108个SM，每个SM最多32个blocks ）
那么grid_dim , block_dim 为<<<256, 864>>>的时候能把所有硬件资源利用完，但是当你采用cooperative launch的时候，发现错误？why?

grid dim是256，所以我们有256个blocks，那么256/108 = 2.37，说明至少一部分的SM上面要放3个blocks, 其中一个SM上就要放置：
3*864 = 2596个threads
明显可以看到2596 > 2048 所以 CUDA报错了。
因为blocks内部的threads不能跨SM，所以这个时候即使有空闲的资源，也不能完全利用，所以合理选用grid dim和block dim非常重要。

查看cuda文档，看到下面介绍，可以通过函数

cudaOccupancyMaxActiveBlocksPerMultiprocessor()来查询在给定threads per block 的时候，每个SM上能驻留的最大blocks数量：
比如，当numThreads=864的时候，每个SM上能驻留的最大blocks是2.
在这里插入图片描述