Andahl‘s law & Gustafson‘s law

什么是Andahl's law？

（Strong scaling）Amdahl rule 是在固定问题规模的基础上，可以设置我们加速可以达到的上限。

 

P表达可并行的部分，占所有的程序的比率。而N表示processor的个数。

可以看到，随着处理器个数的增加，P/N趋近于0，而1-P越小，即P越大，获得的加速比越大。

如果一个3/4的序列化程序可以被并行，那么随着N的增大，最大可以获得4倍的加速。

现实中，很少有这种随着N的改变获得线性的性能提升的案例。更多的代表一种趋势。所以说，当P很小的时候，即使叠加再多的Core上去，也很难得到很好的并行效果。 也就是说，为了达到更好的并行效果，应该努力去提高可以并行代码的比率来获得更大的加速。

 

什么是Gustafson's law ？

(week Scaling)固定每个处理器的处理大小，增加处理器个数来提高整个问题的处理量。

 

 

Understanding which type of scaling is most applicable to an application is an important part of estimating speedup. For some applications the problem size will remain constant and hence only strong scaling is applicable. An example would be modeling how two molecules interact with each other, where the molecule sizes are fixed.

For other applications, the problem size will grow to fill the available processors. Examples include modeling fluids or structures as meshes or grids and some Monte Carlo simulations, where increasing the problem size provides increased accuracy.

Having understood the application profile, the developer should understand how the problem size would change if the computational performance changes and then apply either Amdahl's or Gustafson's Law to determine an upper bound for the speedup.

 

 

RE:

https://docs.nvidia.com/cuda/cuda-c-best-practices-guide/index.html#understanding-scaling

https://www.inlighting.org/archives/amdahls-law-and-its-proof/

https://blog.csdn.net/qq_34594236/article/details/79674204

https://www.d.umn.edu/~tkwon/course/5315/HW/MultiprocessorLaws.pdf