python gridsearchcv_如何在python中使用交叉验证执行GridSearchCV

I am performing hyperparameter tuning of RandomForest as follows using GridSearchCV.

X = np.array(df[features]) #all features

y = np.array(df['gold_standard']) #labels

x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

param_grid = {

'n_estimators': [200, 500],

'max_features': ['auto', 'sqrt', 'log2'],

'max_depth' : [4,5,6,7,8],

'criterion' :['gini', 'entropy']

}

CV_rfc = GridSearchCV(estimator=rfc, param_grid=param_grid, cv= 5)

CV_rfc.fit(x_train, y_train)

print(CV_rfc.best_params_)

The result I got is as follows.

{'criterion': 'gini', 'max_depth': 6, 'max_features': 'auto', 'n_estimators': 200}

Afterwards, I reapply the tuned parameters to x_test as follows.

rfc=RandomForestClassifier(random_state=42, criterion ='gini', max_depth= 6, max_features = 'auto', n_estimators = 200, class_weight = 'balanced')

rfc.fit(x_train, y_train)

pred=rfc.predict(x_test)

print(precision_recall_fscore_support(y_test,pred))

print(roc_auc_score(y_test,pred))

However, I am still not clear how to use GridSearchCV with 10-fold cross validation (i.e. not just apply the tuned parameters to x_test). i.e. something like below.

kf = StratifiedKFold(n_splits=10)

for fold, (train_index, test_index) in enumerate(kf.split(X, y), 1):

X_train = X[train_index]

y_train = y[train_index]

X_test = X[test_index]

y_test = y[test_index]

OR

sinceGridSearchCV uses crossvalidation can we use all X and y and get the best result as the final result?

I am happy to provide more details if needed.

解决方案

You should not perform a grid search in this scenario.

Internally, GridSearchCV splits the dataset given to it into various training and validation subsets, and, using the hyperparameter grid provided to it, finds the single set of hyperparameters that give the best score on the validation subsets.

The point of a train-test split is then, after this process is done, to perform one final scoring on the test data, which has so far been unknown to the model, to see if your hyperparameters have been overfit to the validation subsets. If it does well, then the next step is putting the model into production/deployment.

If you perform a grid search within cross-validation, then you will have multiple sets of hyperparameters, each of which did the best on their grid-search validation sub-subset of the cross-validation split. You cannot combine these sets into a single coherent hyperparameter specification, and therefore you cannot deploy your model.