译（四十九）-Pytorch计算模型参数量

文章首发及后续更新：https://mwhls.top/3662.html
新的更新内容请到mwhls.top查看。
无图/无目录/格式错误/更多相关请到上方的文章首发页面查看。

stackoverflow热门问题目录

如有翻译问题欢迎评论指出，谢谢。

目录

1. 计算Pytorch模型参数量

2. Check the total number of parameters in a PyTorch model

计算Pytorch模型参数量

• Fábio Perez asked:

  • 怎么计算 Pytorch 模型的参数量？类似 Keras 的 model.count_params() 那样的函数。

• Answers:

  • Fábio Perez – vote: 198

  • Pytorch 没有类似 Keras 计算参数量的函数，但可以通过每个参数组的求和得出参数量：

  • pytorch_total_params = sum(p.numel() for p in model.parameters())

• 如果希望只计算可训练的参数：

• pytorch_total_params = sum(p.numel() for p in model.parameters() if p.requires_grad)

• 参考 Pytorch 论坛回答。

• 注：这个回答是自问自答，如果你们有更好的解决方法，请分享一下。

• Fábio Perez – vote: 46

• 为了像 Keras 一样计算每层的参数量，Pytorch 的 model.named_paramters() 能返回一个迭代器，包含参数名与参数本身。

• 举例说明：

• from prettytable import PrettyTable
  #
  def count_parameters(model):
    table = PrettyTable(["Modules", "Parameters"])
    total_params = 0
    for name, parameter in model.named_parameters():
        if not parameter.requires_grad: continue
        params = parameter.numel()
        table.add_row([name, params])
        total_params+=params
    print(table)
    print(f"Total Trainable Params: {total_params}")
    return total_params
  #
  count_parameters(net)

• 输出如下：

• +-------------------+------------+
  |      Modules      | Parameters |
  +-------------------+------------+
  | embeddings.weight |   922866   |
  |    conv1.weight   |  1048576   |
  |     conv1.bias    |    1024    |
  |     bn1.weight    |    1024    |
  |      bn1.bias     |    1024    |
  |    conv2.weight   |  2097152   |
  |     conv2.bias    |    1024    |
  |     bn2.weight    |    1024    |
  |      bn2.bias     |    1024    |
  |    conv3.weight   |  2097152   |
  |     conv3.bias    |    1024    |
  |     bn3.weight    |    1024    |
  |      bn3.bias     |    1024    |
  |    lin1.weight    |  50331648  |
  |     lin1.bias     |    512     |
  |    lin2.weight    |   265728   |
  |     lin2.bias     |    519     |
  +-------------------+------------+
  Total Trainable Params: 56773369

• Thong Nguyen – vote: 12

• 如果希望避免重复计算共享的参数，可以用 torch.Tensor.data_ptr，即：

• sum(dict((p.data_ptr(), p.numel()) for p in model.parameters()).values())

• 下面是更繁琐的实现，添加了一个选项，用于跳过不被训练的参数：

• def numel(m: torch.nn.Module, only_trainable: bool = False):
    """
    returns the total number of parameters used by m (only counting
    shared parameters once); if only_trainable is True, then only
    includes parameters with requires_grad = True
    """
    parameters = list(m.parameters())
    if only_trainable:
        parameters = [p for p in parameters if p.requires_grad]
    unique = {p.data_ptr(): p for p in parameters}.values()
    return sum(p.numel() for p in unique)

• ---

  Check the total number of parameters in a PyTorch model

  • Fábio Perez asked:

    • How to count the total number of parameters in a PyTorch model? Something similar to model.count_params() in Keras.
      怎么计算 Pytorch 模型的参数量？类似 Keras 的 model.count_params() 那样的函数。

  • Answers:

    • Fábio Perez – vote: 198

    • PyTorch doesn\’t have a function to calculate the total number of parameters as Keras does, but it\’s possible to sum the number of elements for every parameter group:
      Pytorch 没有类似 Keras 计算参数量的函数，但可以通过每个参数组的求和得出参数量：

    • pytorch_total_params = sum(p.numel() for p in model.parameters())

    • If you want to calculate only the trainable parameters:
      如果希望只计算可训练的参数：

    • pytorch_total_params = sum(p.numel() for p in model.parameters() if p.requires_grad)

    • Answer inspired by this answer on PyTorch Forums.
      参考 Pytorch 论坛回答。

    • Note: I\’m answering my own question. If anyone has a better solution, please share with us.
      注：这个回答是自问自答，如果你们有更好的解决方法，请分享一下。

    • Fábio Perez – vote: 46

    • To get the parameter count of each layer like Keras, PyTorch has model.named_paramters() that returns an iterator of both the parameter name and the parameter itself.
      为了像 Keras 一样计算每层的参数量，Pytorch 的 model.named_paramters() 能返回一个迭代器，包含参数名与参数本身。

    • Here is an example:
      举例说明：

    • from prettytable import PrettyTable
      #
      def count_parameters(model):
        table = PrettyTable(["Modules", "Parameters"])
        total_params = 0
        for name, parameter in model.named_parameters():
            if not parameter.requires_grad: continue
            params = parameter.numel()
            table.add_row([name, params])
            total_params+=params
        print(table)
        print(f"Total Trainable Params: {total_params}")
        return total_params
      #
      count_parameters(net)

    • The output would look something like this:
      输出如下：

    • +-------------------+------------+
      |      Modules      | Parameters |
      +-------------------+------------+
      | embeddings.weight |   922866   |
      |    conv1.weight   |  1048576   |
      |     conv1.bias    |    1024    |
      |     bn1.weight    |    1024    |
      |      bn1.bias     |    1024    |
      |    conv2.weight   |  2097152   |
      |     conv2.bias    |    1024    |
      |     bn2.weight    |    1024    |
      |      bn2.bias     |    1024    |
      |    conv3.weight   |  2097152   |
      |     conv3.bias    |    1024    |
      |     bn3.weight    |    1024    |
      |      bn3.bias     |    1024    |
      |    lin1.weight    |  50331648  |
      |     lin1.bias     |    512     |
      |    lin2.weight    |   265728   |
      |     lin2.bias     |    519     |
      +-------------------+------------+
      Total Trainable Params: 56773369

    • Thong Nguyen – vote: 12

    • If you want to avoid double counting shared parameters, you can use torch.Tensor.data_ptr. E.g.:
      如果希望避免重复计算共享的参数，可以用 torch.Tensor.data_ptr，即：

    • sum(dict((p.data_ptr(), p.numel()) for p in model.parameters()).values())

    • Here\’s a more verbose implementation that includes an option to filter out non-trainable parameters:
      下面是更繁琐的实现，添加了一个选项，用于跳过不被训练的参数：

    • def numel(m: torch.nn.Module, only_trainable: bool = False):
        """
        returns the total number of parameters used by m (only counting
        shared parameters once); if only_trainable is True, then only
        includes parameters with requires_grad = True
        """
        parameters = list(m.parameters())
        if only_trainable:
            parameters = [p for p in parameters if p.requires_grad]
        unique = {p.data_ptr(): p for p in parameters}.values()
        return sum(p.numel() for p in unique)