自动化调参NNI学习(一)：官网代码学习

文档

github主页：https://github.com/microsoft/nni

文档：https://nni.readthedocs.io/en/stable/Overview.html

quickstart：https://nni.readthedocs.io/en/stable/Tutorial/QuickStart.html

案例地址：https://github.com/microsoft/nni/tree/master/examples

(sklearn)案例地址：https://github.com/microsoft/nni/tree/master/examples/trials/sklearn

安装

pip install --upgrade nni

调参思路

1. json文件：确定参数的取值范围

   json格式中的search_space配置参考：
   https://nni.readthedocs.io/en/stable/Tutorial/SearchSpaceSpec.html#search-space

   当然，建议把search_space单独写成一个json文件，不过也可以写在yml文件中，可以参考：
   https://nni.readthedocs.io/en/latest/reference/experiment_config.html#local-mode-inline-search-space

2. yml文件：nni框架的配置文件

   yml文件的配置参考：
   https://nni.readthedocs.io/en/latest/reference/experiment_config.html#experimentconfig

3. python文件：进行单次的训练

4. 然后使用NNI框架进行迭代训练，得出一个较好的参数

官网案例与解析

以官方案例sklearn的svm为例：https://github.com/microsoft/nni/tree/master/examples/trials/sklearn/classification

config文件：search_space.json

{
    "C": {"_type":"uniform","_value":[0.1, 1]},
    "kernel": {"_type":"choice","_value":["linear", "rbf", "poly", "sigmoid"]},
    "degree": {"_type":"choice","_value":[1, 2, 3, 4]},
    "gamma": {"_type":"uniform","_value":[0.01, 0.1]},
    "coef0": {"_type":"uniform","_value":[0.01, 0.1]}
}

yml文件：config.yml

searchSpaceFile: search_space.json
trialCommand: python main.py
trialConcurrency: 1
maxTrialNumber: 100
maxExperimentDuration: 1h
tuner:
  name: TPE
  classArgs:
    optimize_mode: maximize
trainingService: # For other platforms, check mnist-pytorch example
  platform: local

注意这里的trialCommand，在部署的时候可能需要使用绝对路径来指定python的解释环境

python代码：main.py

import nni
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_digits
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
import logging

LOG = logging.getLogger('sklearn_classification')


def load_data():
    '''Load dataset, use 20newsgroups dataset'''
    digits = load_digits()
    X_train, X_test, y_train, y_test = train_test_split(
        digits.data, digits.target, random_state=99, test_size=0.25)

    ss = StandardScaler()
    X_train = ss.fit_transform(X_train)
    X_test = ss.transform(X_test)

    return X_train, X_test, y_train, y_test


def get_default_parameters():
    '''get default parameters'''
    params = {
        'C': 1.0,
        'kernel': 'linear',
        'degree': 3,
        'gamma': 0.01,
        'coef0': 0.01
    }
    return params


def get_model(PARAMS):  # 加载模型
    '''Get model according to parameters'''
    model = SVC()
    model.C = PARAMS.get('C')
    model.kernel = PARAMS.get('kernel')
    model.degree = PARAMS.get('degree')
    model.gamma = PARAMS.get('gamma')
    model.coef0 = PARAMS.get('coef0')
    return model


def run(X_train, X_test, y_train, y_test, model):
    '''Train model and predict result'''
    model.fit(X_train, y_train)
    score = model.score(X_test, y_test)
    LOG.debug('score: %s', score)
    nni.report_final_result(score)  # 灵魂代码：nni根据这个值判定模型的好坏


if __name__ == '__main__':
    X_train, X_test, y_train, y_test = load_data()
    try:
        # get parameters from tuner
        RECEIVED_PARAMS = nni.get_next_parameter()  # 这里是使用nni得到参数
        LOG.debug(RECEIVED_PARAMS)
        PARAMS = get_default_parameters()  # 使用一个默认参数
        PARAMS.update(RECEIVED_PARAMS)  # 将默认参数更新为新的参数
        LOG.debug(PARAMS)
        model = get_model(PARAMS)  # 用这个参数训练模型
        run(X_train, X_test, y_train, y_test, model)  # 得到结果
    except Exception as exception:
        LOG.exception(exception)
        raise

然后运行

nnictl create --config config.yml -p 12388 --debug

得到运行结果：

部分nnictl命令

index	commands	description
1	nnictl experiment show	show the information of experiments
2	nnictl trial ls	list all of trial jobs
3	nnictl top	monitor the status of running experiments
4	nnictl log stderr	show stderr log content
5	nnictl log stdout	show stdout log content
6	nnictl stop	stop an experiment
7	nnictl trial kill	kill a trial job by id
8	nnictl --help	get help information about nnictl