Scikit-learn的ShuffleSplit与GroupShuffleSplit

官方文档说明：

class GroupShuffleSplit(ShuffleSplit):
    '''Shuffle-Group(s)-Out cross-validation iterator

    Provides randomized train/test indices to split data according to a
    third-party provided group. This group information can be used to encode
    arbitrary domain specific stratifications of the samples as integers.

    For instance the groups could be the year of collection of the samples
    and thus allow for cross-validation against time-based splits.

    The difference between LeavePGroupsOut and GroupShuffleSplit is that
    the former generates splits using all subsets of size ``p`` unique groups,
    whereas GroupShuffleSplit generates a user-determined number of random
    test splits, each with a user-determined fraction of unique groups.

    For example, a less computationally intensive alternative to
    ``LeavePGroupsOut(p=10)`` would be
    ``GroupShuffleSplit(test_size=10, n_splits=100)``.

    Note: The parameters ``test_size`` and ``train_size`` refer to groups, and
    not to samples, as in ShuffleSplit.


    Parameters
    ----------
    n_splits : int, default=5
        Number of re-shuffling & splitting iterations.

    test_size : float, int, default=0.2
        If float, should be between 0.0 and 1.0 and represent the proportion
        of groups to include in the test split (rounded up). If int,
        represents the absolute number of test groups. If None, the value is
        set to the complement of the train size.
        The default will change in version 0.21. It will remain 0.2 only
        if ``train_size`` is unspecified, otherwise it will complement
        the specified ``train_size``.

    train_size : float or int, default=None
        If float, should be between 0.0 and 1.0 and represent the
        proportion of the groups to include in the train split. If
        int, represents the absolute number of train groups. If None,
        the value is automatically set to the complement of the test size.

    random_state : int or RandomState instance, default=None
        Controls the randomness of the training and testing indices produced.
        Pass an int for reproducible output across multiple function calls.
        See :term:`Glossary <random_state>`.

    Examples
    --------
    >>> import numpy as np
    >>> from sklearn.model_selection import GroupShuffleSplit
    >>> X = np.ones(shape=(8, 2))
    >>> y = np.ones(shape=(8, 1))
    >>> groups = np.array([1, 1, 2, 2, 2, 3, 3, 3])
    >>> print(groups.shape)
    (8,)
    >>> gss = GroupShuffleSplit(n_splits=2, train_size=.7, random_state=42)
    >>> gss.get_n_splits()
    2
    >>> for train_idx, test_idx in gss.split(X, y, groups):
    ...     print("TRAIN:", train_idx, "TEST:", test_idx)
    TRAIN: [2 3 4 5 6 7] TEST: [0 1]
    TRAIN: [0 1 5 6 7] TEST: [2 3 4]
    '''
    @_deprecate_positional_args
    def __init__(self, n_splits=5, *, test_size=None, train_size=None,
                 random_state=None):
        super().__init__(
            n_splits=n_splits,
            test_size=test_size,
            train_size=train_size,
            random_state=random_state)
        self._default_test_size = 0.2

    def _iter_indices(self, X, y, groups):
        if groups is None:
            raise ValueError("The 'groups' parameter should not be None.")
        groups = check_array(groups, ensure_2d=False, dtype=None)
        classes, group_indices = np.unique(groups, return_inverse=True)
        for group_train, group_test in super()._iter_indices(X=classes):
            # these are the indices of classes in the partition
            # invert them into data indices

            train = np.flatnonzero(np.in1d(group_indices, group_train))
            test = np.flatnonzero(np.in1d(group_indices, group_test))

            yield train, test

    def split(self, X, y=None, groups=None):
        """Generate indices to split data into training and test set.

        Parameters
        ----------
        X : array-like of shape (n_samples, n_features)
            Training data, where n_samples is the number of samples
            and n_features is the number of features.

        y : array-like of shape (n_samples,), default=None
            The target variable for supervised learning problems.

        groups : array-like of shape (n_samples,)
            Group labels for the samples used while splitting the dataset into
            train/test set.

        Yields
        ------
        train : ndarray
            The training set indices for that split.

        test : ndarray
            The testing set indices for that split.

        Notes
        -----
        Randomized CV splitters may return different results for each call of
        split. You can make the results identical by setting `random_state`
        to an integer.
        """
        return super().split(X, y, groups)

实验1：

以上就是group_shuffle_split 类的官方说明，下边我们将用实验对其进行测试，与shuffle——split进行对比。

数据：

 subject classname            img
0    p012        c5  img_41179.jpg
1    p012        c0  img_50749.jpg
2    p014        c1  img_53609.jpg
3    p014        c5  img_52213.jpg
4    p014        c0  img_72495.jpg
5    p024        c0  img_66836.jpg
6    p024        c1  img_32639.jpg
7    p024        c1  img_31777.jpg
8    p024        c2  img_97535.jpg
9    p081        c6   img_1399.jpg

普通的shuffsplit：

import pandas as pd
import numpy as np
from sklearn.model_selection import ShuffleSplit,GroupShuffleSplit
sample = pd.DataFrame({'subject':['p012', 'p012', 'p014', 'p014', 'p014', 'p024', 'p024', 'p024', 'p024', 'p081'],
                         'classname':['c5', 'c0', 'c1', 'c5', 'c0','c0','c1','c1','c2','c6'],
                         'img':['img_41179.jpg','img_50749.jpg', 'img_53609.jpg','img_52213.jpg','img_72495.jpg',
                         'img_66836.jpg','img_32639.jpg','img_31777.jpg','img_97535.jpg','img_1399.jpg']})

x_train_names = np.array(sample['img'])
y_train_names = np.array(sample['classname'])

ss = ShuffleSplit(n_splits=5, test_size=0.2, random_state=111)
n_fold =1
for train_indices, test_indices in ss.split(sample):
    # 将sample进行随机打乱，并且进行划分。
    print('fold{}/5......'.format(n_fold))
    print('train_indices:',train_indices)
    x_train = x_train_names[train_indices]
    print('x_trian:\n', x_train)
    y_train = y_train_names[train_indices]
    print('y_trian:\n', y_train)
    print("test_indices:", test_indices)
    x_test = x_train_names[test_indices, ...]
    print("x_test:\n", x_test)
    y_test = y_train_names[test_indices, ...]
    print("y_test:\n", y_test)
    n_fold += 1

fold1/5......
train_indices: [5 2 1 3 6 8 9 4]
x_trian:
 ['img_66836.jpg' 'img_53609.jpg' 'img_50749.jpg' 'img_52213.jpg'
 'img_32639.jpg' 'img_97535.jpg' 'img_1399.jpg' 'img_72495.jpg']
y_trian:
 ['c0' 'c1' 'c0' 'c5' 'c1' 'c2' 'c6' 'c0']
test_indices: [7 0]
x_test:
 ['img_31777.jpg' 'img_41179.jpg']
y_test:
 ['c1' 'c5']
fold2/5......
train_indices: [3 0 9 2 4 1 7 8]
x_trian:
 ['img_52213.jpg' 'img_41179.jpg' 'img_1399.jpg' 'img_53609.jpg'
 'img_72495.jpg' 'img_50749.jpg' 'img_31777.jpg' 'img_97535.jpg']
y_trian:
 ['c5' 'c5' 'c6' 'c1' 'c0' 'c0' 'c1' 'c2']
test_indices: [6 5]
x_test:
 ['img_32639.jpg' 'img_66836.jpg']
y_test:
 ['c1' 'c0']
fold3/5......
train_indices: [3 9 4 8 0 6 5 2]
x_trian:
 ['img_52213.jpg' 'img_1399.jpg' 'img_72495.jpg' 'img_97535.jpg'
 'img_41179.jpg' 'img_32639.jpg' 'img_66836.jpg' 'img_53609.jpg']
y_trian:
 ['c5' 'c6' 'c0' 'c2' 'c5' 'c1' 'c0' 'c1']
test_indices: [1 7]
x_test:
 ['img_50749.jpg' 'img_31777.jpg']
y_test:
 ['c0' 'c1']
fold4/5......
train_indices: [0 7 3 2 9 6 8 5]
x_trian:
 ['img_41179.jpg' 'img_31777.jpg' 'img_52213.jpg' 'img_53609.jpg'
 'img_1399.jpg' 'img_32639.jpg' 'img_97535.jpg' 'img_66836.jpg']
y_trian:
 ['c5' 'c1' 'c5' 'c1' 'c6' 'c1' 'c2' 'c0']
test_indices: [4 1]
x_test:
 ['img_72495.jpg' 'img_50749.jpg']
y_test:
 ['c0' 'c0']
fold5/5......
train_indices: [4 9 0 8 6 1 3 7]
x_trian:
 ['img_72495.jpg' 'img_1399.jpg' 'img_41179.jpg' 'img_97535.jpg'
 'img_32639.jpg' 'img_50749.jpg' 'img_52213.jpg' 'img_31777.jpg']
y_trian:
 ['c0' 'c6' 'c5' 'c2' 'c1' 'c0' 'c5' 'c1']
test_indices: [2 5]
x_test:
 ['img_53609.jpg' 'img_66836.jpg']
y_test:
 ['c1' 'c0']

Process finished with exit code 0

可以看到随机打乱，并且进行了5次分割，但是有一点，却没有考虑p属性，比如分割的时候没有考虑不同年龄段之类的。解决这个问题就用到了GroupShuffleSplit：

实验2：

driver_ids = sample.subject

_,driver_indices = np.unique(np.array(driver_ids),return_inverse=True)
# 不同的group的索引。
gss = GroupShuffleSplit(n_splits=4, test_size=0.25, random_state=22)
n_fold =1
for train_indices, test_indices in gss.split(x_train_names,y_train_names,groups=driver_ids):
    # 将sample进行随机打乱，并且进行划分。
    print('fold{}/5......'.format(n_fold))
    print('train_indices:',train_indices)
    x_train = x_train_names[train_indices]
    print('x_trian:\n', x_train)
    y_train = y_train_names[train_indices]
    print('y_trian:\n', y_train)
    print("test_indices:", test_indices)
    x_test = x_train_names[test_indices, ...]
    print("x_test:\n", x_test)
    y_test = y_train_names[test_indices, ...]
    print("y_test:\n", y_test)
    n_fold += 1

fold1/5......
train_indices: [0 1 2 3 4 5 6 7 8]
x_trian:
 ['img_41179.jpg' 'img_50749.jpg' 'img_53609.jpg' 'img_52213.jpg'
 'img_72495.jpg' 'img_66836.jpg' 'img_32639.jpg' 'img_31777.jpg'
 'img_97535.jpg']
y_trian:
 ['c5' 'c0' 'c1' 'c5' 'c0' 'c0' 'c1' 'c1' 'c2']
test_indices: [9]
x_test:
 ['img_1399.jpg']
y_test:
 ['c6']
fold2/5......
train_indices: [0 1 5 6 7 8 9]
x_trian:
 ['img_41179.jpg' 'img_50749.jpg' 'img_66836.jpg' 'img_32639.jpg'
 'img_31777.jpg' 'img_97535.jpg' 'img_1399.jpg']
y_trian:
 ['c5' 'c0' 'c0' 'c1' 'c1' 'c2' 'c6']
test_indices: [2 3 4]
x_test:
 ['img_53609.jpg' 'img_52213.jpg' 'img_72495.jpg']
y_test:
 ['c1' 'c5' 'c0']
fold3/5......
train_indices: [0 1 5 6 7 8 9]
x_trian:
 ['img_41179.jpg' 'img_50749.jpg' 'img_66836.jpg' 'img_32639.jpg'
 'img_31777.jpg' 'img_97535.jpg' 'img_1399.jpg']
y_trian:
 ['c5' 'c0' 'c0' 'c1' 'c1' 'c2' 'c6']
test_indices: [2 3 4]
x_test:
 ['img_53609.jpg' 'img_52213.jpg' 'img_72495.jpg']
y_test:
 ['c1' 'c5' 'c0']
fold4/5......
train_indices: [0 1 2 3 4 9]
x_trian:
 ['img_41179.jpg' 'img_50749.jpg' 'img_53609.jpg' 'img_52213.jpg'
 'img_72495.jpg' 'img_1399.jpg']
y_trian:
 ['c5' 'c0' 'c1' 'c5' 'c0' 'c6']
test_indices: [5 6 7 8]
x_test:
 ['img_66836.jpg' 'img_32639.jpg' 'img_31777.jpg' 'img_97535.jpg']
y_test:
 ['c0' 'c1' 'c1' 'c2']

参考：博客