[python tools] 今天看到另一个配置工具 YACS，所以做下笔记

YACS

实际上就只是把别人的readme翻译了一下

github: https://github.com/rbgirshick/yacs
样例代码: https://github.com/Wuziyi616/multi_part_assembly/blob/master/docs/config.md

一、使用方法

1. 首先搞一个config的python文件，用来存一下基本的配置信息

比如说显卡的数量、num_works等等。

# my_project/config.py
from yacs.config import CfgNode as CN


_C = CN()

_C.SYSTEM = CN()
# Number of GPUS to use in the experiment
_C.SYSTEM.NUM_GPUS = 8
# Number of workers for doing things
_C.SYSTEM.NUM_WORKERS = 4

_C.TRAIN = CN()
# A very important hyperparameter
_C.TRAIN.HYPERPARAMETER_1 = 0.1
# The all important scales for the stuff
_C.TRAIN.SCALES = (2, 4, 8, 16)


def get_cfg_defaults():
  """Get a yacs CfgNode object with default values for my_project."""
  # Return a clone so that the defaults will not be altered
  # This is for the "local variable" use pattern
  return _C.clone()

# Alternatively, provide a way to import the defaults as
# a global singleton:
# cfg = _C  # users can `from config import cfg`

2. 这个基本的配置信息还可以和yaml文件一起使用

YACS给的使用场景是，你可以给每一个实验写一个yaml文件，然后将这个yaml+前面的config文件一起使用

# my_project/main.py

import my_project
from config import get_cfg_defaults  # local variable usage pattern, or:
# from config import cfg  # global singleton usage pattern


if __name__ == "__main__":
  cfg = get_cfg_defaults()
  cfg.merge_from_file("experiment.yaml")
  cfg.freeze()
  print(cfg)

  # Example of using the cfg as global access to options
  if cfg.SYSTEM.NUM_GPUS > 0:
    my_project.setup_multi_gpu_support()

  model = my_project.create_model(cfg)

# my_project/experiment.yaml
SYSTEM:
  NUM_GPUS: 2
TRAIN:
  SCALES: (1, 2)

3. 还可以通过freeze()函数来冻结 config

# my_project/main.py

import my_project
from config import get_cfg_defaults  # local variable usage pattern, or:
# from config import cfg  # global singleton usage pattern


if __name__ == "__main__":
  cfg = get_cfg_defaults()
  cfg.merge_from_file("experiment.yaml")
  cfg.freeze()
  print(cfg)

  # Example of using the cfg as global access to options
  if cfg.SYSTEM.NUM_GPUS > 0:
    my_project.setup_multi_gpu_support()

  model = my_project.create_model(cfg)

4. 你还可以通过list来merge配置信息

但是我觉得这个看上去 好像不如argsparse好用

cfg.merge_from_file("experiment.yaml")
# Now override from a list (opts could come from the command line)
opts = ["SYSTEM.NUM_GPUS", 8, "TRAIN.SCALES", "(1, 2, 3, 4)"]
cfg.merge_from_list(opts)

The following principle is recommended: “There is only one way to configure the same thing.” This principle means that if an option is defined in a YACS config object, then your program should set that configuration option using cfg.merge_from_list(opts) and not by defining, for example, --train-scales as a command line argument that is then used to set cfg.TRAIN.SCALES.

二、在 Multi_part_assembly这个仓库中是具体使用

https://github.com/Wuziyi616/multi_part_assembly/blob/master/docs/config.md

这个项目是将配置信息分割在不同的python文件中，然后通过继承和组合来获得新的配置。

1. 比如说面这个数据集的配置 可以和 模型结构的配置组合在一起

datasets/partnet.py

from yacs.config import CfgNode as CN

_C = CN()
_C.dataset = 'partnet'
_C.data_dir = './data/partnet'


def get_cfg_defaults():
    return _C.clone()

model/global.py

from yacs.config import CfgNode as CN

_C = CN()
_C.name = 'global'
_C.encoder = 'pointnet'


def get_cfg_defaults():
    return _C.clone()

global-partnet.py

import os
from yacs.config import CfgNode as CN
from multi_part_assembly.utils import merge_cfg

# inheriting configs by specifying `_base_`
# 'data' field will be from `partnet.py`
# 'model' field will be from `global.py`
_base_ = {
    'data': 'datasets/partnet.py',
    'model': 'model/global.py',
}

_C = CN()  # create self in yacs

# to override a field in a parent config
# you need to first create a `CN()` base for it
# then modify the value by setting new values
_C.data = CN()
_C.data.data_dir = './datasets/partnet'  # overriding `data_dir`


# merging code
def get_cfg_defaults():
    base_cfg = _C.clone()
    cfg = merge_cfg(base_cfg, os.path.dirname(__file__), _base_)
    return cfg

2. 这篇文章的命名是这样的

{model}_{batch_per_gpu x gpu}_{schedule}_{dataset}.py

• model 使用的模型
• batch per gpu x gpu 在每个gpu上的batch，总的gpu的数量
• dataset 数据集
• schedule 应该是学习率的配置

可以使用下面的代码看一下这个配置有哪些信息

python scripts/print_cfg.py --cfg_file $CFG