【数据竞赛】Kaggle神技：一项堪比Dropout的NN训练技巧！

最新推荐文章于 2023-05-04 17:18:02 发布
最新推荐文章于 2023-05-04 17:18:02 发布
阅读量1.1k 收藏 3
点赞数 1

↑↑↑关注后"星标"kaggle竞赛宝典

 kaggle竞赛宝典

作者：杰少

Swap Noise: 一种论文中所没有的NN神技

01

背景

本文介绍一种论文中所没有，但是却效果极佳的，拿了N项Kaggle大赛奖项的技巧，俗称Swap Noise，至于为什么有效，大家可以参考kaggle的论文，该技巧是几年前Kaggle GM Michael Jahrer提出。至于效果怎么样，下面一个案例足以说明一切。

常见的Swap Noise一共有三种类型，包括：

1. 列交换噪声(Columnwise swap noise): The idea of columwise noise is to noise only 20% of the batch by touching only full columns (check the animation below).

2. 随机噪声交换(randomized swap noise): Randomized noise ads 20% noisy values in a random way (see animation below)

 

3. 行交换噪声(rowwise swap noise):As the name indicates rowwise noise replaces 20% of data so that each and every row is beeing transformed by a certain amount of noise.

02

案例

1. 导入工具包

1.1 导入使用的工具包

import pandas                as pd 
from sklearn.metrics         import mean_squared_error
from sklearn.model_selection import KFold
import xgboost               as xgb
from   tqdm                  import tqdm
import numpy                 as np
import pandas                as pd 
import tensorflow            as tf 
from lightgbm                import LGBMRegressor
from sklearn.model_selection import KFold
import numpy                 as np
import seaborn               as sns
from sklearn.metrics         import mean_squared_error

def RMSE(y_true, y_pred):
    return tf.sqrt(tf.reduce_mean(tf.square(y_true - y_pred))) 

1.2 数据读取

train = pd.read_csv('./data/train.csv')
test  = pd.read_csv('./data/test.csv') 

2. 数据预处理

2.1 数据拼接

train_test = pd.concat([train,test],axis=0,ignore_index=True)

train_test.head()

idcont1cont2cont3cont4cont5cont6cont7cont8cont9cont10cont11cont12cont13cont14target
010.6703900.8113000.6439680.2917910.2841170.8559530.8907000.2855420.5582450.7794180.9218320.8667720.8787330.3054117.243043
130.3880530.6211040.6861020.5011490.6437900.4498050.5108240.5807480.4183350.4326320.4398720.4349710.3699570.3694848.203331
240.8349500.2274360.3015840.2934080.6068390.8291750.5061430.5587710.5876030.8233120.5670070.6777080.8829380.3030477.776091
350.8207080.1601550.5468870.7261040.2824440.7851080.7527580.8232670.5744660.5808430.7695940.8181430.9142810.2795286.957716
480.9352780.4212350.3038010.8802140.6656100.8301310.4871130.6041570.8746580.8634270.9835750.9004640.9359180.4357727.951046

2.2 用于神经网络预处理的GaussianRank

import numpy as np
from joblib import Parallel, delayed
from scipy.interpolate import interp1d
from scipy.special import erf, erfinv
from sklearn.preprocessing import QuantileTransformer,PowerTransformer
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.utils.validation import FLOAT_DTYPES, check_array, check_is_fitted

class GaussRankScaler(BaseEstimator, TransformerMixin):
    """Transform features by scaling each feature to a normal distribution.
    Parameters
        ----------
        epsilon : float, optional, default 1e-4
            A small amount added to the lower bound or subtracted
            from the upper bound. This value prevents infinite number
            from occurring when applying the inverse error function.
        copy : boolean, optional, default True
            If False, try to avoid a copy and do inplace scaling instead.
            This is not guaranteed to always work inplace; e.g. if the data is
            not a NumPy array, a copy may still be returned.
        n_jobs : int or None, optional, default None
            Number of jobs to run in parallel.
            ``None`` means 1 and ``-1`` means using all processors.
        interp_kind : str or int, optional, default 'linear'
           Specifies the kind of interpolation as a string
            ('linear', 'nearest', 'zero', 'slinear', 'quadratic', 'cubic',
            'previous', 'next', where 'zero', 'slinear', 'quadratic' and 'cubic'
            refer to a spline interpolation of zeroth, first, second or third
            order; 'previous' and 'next' simply return the previous or next value
            of the point) or as an integer specifying the order of the spline
            interpolator to use.
        interp_copy : bool, optional, default False
            If True, the interpolation function makes internal copies of x and y.
            If False, references to `x` and `y` are used.
        Attributes
        ----------
        interp_func_ : list
            The interpolation function for each feature in the training set.
        """

    def __init__(self, epsilon=1e-4, copy=True, n_jobs=None, interp_kind='linear', interp_copy=False):
        self.epsilon     = epsilon
        self.copy        = copy
        self.interp_kind = interp_kind
        self.interp_copy = interp_copy
        self.fill_value  = 'extrapolate'
        self.n_jobs      = n_jobs

    def fit(self, X, y=None):
        """Fit interpolation function to link rank with original data for future scaling
        Parameters
        ----------
        X : array-like, shape (n_samples, n_features)
            The data used to fit interpolation function for later scaling along the features axis.
        y
            Ignored
        """
        X = check_array(X, copy=self.copy, estimator=self, dtype=FLOAT_DTYPES, force_all_finite=True)

        self.interp_func_ = Parallel(n_jobs=self.n_jobs)(delayed(self._fit)(x) for x in X.T)
        return self

    def _fit(self, x):
        x = self.drop_duplicates(x)
        rank = np.argsort(np.argsort(x))
        bound = 1.0 - self.epsilon
        factor = np.max(rank) / 2.0 * bound
        scaled_rank = np.clip(rank / factor - bound, -bound, bound)
        return interp1d(
            x, scaled_rank, kind=self.interp_kind, copy=self.interp_copy, fill_value=self.fill_value)

    def transform(self, X, copy=None):
        """Scale the data with the Gauss Rank algorithm
        Parameters
        ----------
        X : array-like, shape (n_samples, n_features)
            The data used to scale along the features axis.
        copy : bool, optional (default: None)
            Copy the input X or not.
        """
        check_is_fitted(self, 'interp_func_')

        copy = copy if copy is not None else self.copy
        X = check_array(X, copy=copy, estimator=self, dtype=FLOAT_DTYPES, force_all_finite=True)

        X = np.array(Parallel(n_jobs=self.n_jobs)(delayed(self._transform)(i, x) for i, x in enumerate(X.T))).T
        return X

    def _transform(self, i, x):
        return erfinv(self.interp_func_[i](x))

    def inverse_transform(self, X, copy=None):
        """Scale back the data to the original representation
        Parameters
        ----------
        X : array-like, shape [n_samples, n_features]
            The data used to scale along the features axis.
        copy : bool, optional (default: None)
            Copy the input X or not.
        """
        check_is_fitted(self, 'interp_func_')

        copy = copy if copy is not None else self.copy
        X = check_array(X, copy=copy, estimator=self, dtype=FLOAT_DTYPES, force_all_finite=True)

        X = np.array(Parallel(n_jobs=self.n_jobs)(delayed(self._inverse_transform)(i, x) for i, x in enumerate(X.T))).T
        return X

    def _inverse_transform(self, i, x):
        inv_interp_func = interp1d(self.interp_func_[i].y, self.interp_func_[i].x, kind=self.interp_kind,
                                   copy=self.interp_copy, fill_value=self.fill_value)
        return inv_interp_func(erf(x))

    @staticmethod
    def drop_duplicates(x):
        is_unique = np.zeros_like(x, dtype=bool)
        is_unique[np.unique(x, return_index=True)[1]] = True
        return x[is_unique]
2.3 RankGaussian处理
feature_names = ['cont1', 'cont2', 'cont3', 'cont4', 'cont5', 'cont6', 'cont7','cont8', 'cont9', 'cont10', 'cont11', 'cont12', 'cont13', 'cont14']
scaler_linear    = GaussRankScaler(interp_kind='linear',) 
for c in feature_names:
    train_test[c+'_linear_grank'] = scaler_linear.fit_transform(train_test[c].values.reshape(-1,1))
    
gaussian_linear_feature_names = [c + '_linear_grank' for c in feature_names]

3. 各种模型对比

3.1 NN模型建模

from tensorflow.keras import regularizers
from sklearn.model_selection import KFold, StratifiedKFold
import tensorflow as tf
# import tensorflow_addons as tfa
import tensorflow.keras.backend as K
from tensorflow.keras.layers import *
from tensorflow.keras.models import *
from tensorflow.keras.optimizers import *
from tensorflow.keras.callbacks import *
from tensorflow.keras.layers import Input

3.2 用于训练的Loss

def RMSE_Loss(y_true, y_pred):
    return tf.sqrt(tf.reduce_mean(tf.square(y_true - y_pred)))

3.3 训练集验证集合划分

tr = train_test.iloc[:train.shape[0],:].copy()
te = train_test.iloc[train.shape[0]:,:].copy() 
kf          = KFold(n_splits=5,random_state=48,shuffle=False) 
cnt         = 0
for trn_idx, test_idx in kf.split(tr,tr['target']):
    X_tr_dnn_linear_gaussian,X_val_dnn_linear_gaussian = tr[gaussian_linear_feature_names].iloc[trn_idx],tr[gaussian_linear_feature_names].iloc[test_idx]
    y_tr,y_val = tr['target'].iloc[trn_idx],train['target'].iloc[test_idx]
    
    break

/home/inf/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_split.py:297: FutureWarning: Setting a random_state has no effect since shuffle is False. This will raise an error in 0.24. You should leave random_state to its default (None), or set shuffle=True.
  FutureWarning

4. 模型对比

4.1 自定义Generator+SwapNoise
### 自定义训练的generator

from tensorflow.python.keras.utils.data_utils import Sequence
class DataGenerator(Sequence):
    'Generates data for Keras'
    def __init__(self, IDs, features, labels,  batch_size=128, shuffle_prob=0.15, shuffle_type=1, shuffle = True):
        'Initialization' 
        self.batch_size   = batch_size 
        self.list_IDs     = IDs  
        self.shuffle      = shuffle
        self.features     = features
        self.feature_len  = features.shape[0]
        self.shuffle_prob = shuffle_prob
        self.labels       = labels 
        self.shuffle_type = shuffle_type
        self.on_epoch_end()

    def __len__(self):
        'Denotes the number of batches per epoch'
        return int(np.floor(len(self.list_IDs) / self.batch_size))

    def __getitem__(self, index):
        'Generate one batch of data'
        # Generate indexes of the batch
        indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size] 
        # Find list of IDs
        list_IDs_temp = [self.list_IDs[k] for k in indexes]

        # Generate data
        X, y = self.__data_generation(list_IDs_temp) 
        return X, y

    def on_epoch_end(self):
        'Updates indexes after each epoch'
        self.indexes = np.arange(len(self.list_IDs))
        if self.shuffle == True:
            np.random.shuffle(self.indexes)
            
         
    def __row_swap_noise(self, X1,prob = 0.15):
        batch_num_value = len(X1)
        
        num_value       = self.feature_len
        num_swap        = int(batch_num_value * prob) 
        fea_num         = X1.shape[1]
        
        for i in range(fea_num):
            swap_idx_batch = np.random.choice(batch_num_value, num_swap, replace=False)
            swap_idx_all   = np.random.choice(num_value, num_swap, replace=False)
            
            X1[swap_idx_batch,i] = self.features[swap_idx_all,i]
        return X1
    
    def __batch_row_swap_noise(self, X1,X2, prob = 0.15):
        num_value = len(X1)
        num_swap  = int(num_value * prob) 
        fea_num   = X1.shape[1]
        
        for i in range(fea_num):
            swap_idx = np.random.choice(num_value, num_swap, replace=False)
            X1[swap_idx,i] = X2[swap_idx,i]
        return X1
    
    def __data_generation(self, list_IDs_temp):
        'Generates data containing batch_size samples' # X : (n_samples, *dim, n_channels)
        # Initialization
        ID1 = list_IDs_temp
        
        # Original Features
        X1 = self.features[ID1]
        if self.shuffle_type == 1:
            
            ID2= ID1.copy()
            np.random.shuffle(ID2)
            X2 = self.features[ID2] 
            X = self.__batch_row_swap_noise(X1.copy(), X2.copy(), prob=self.shuffle_prob)
        elif self.shuffle_type == 2:
            X = self.__row_swap_noise(X1.copy(), prob=self.shuffle_prob)
            
        else:
            X = X1 
        y = self.labels[ID1]

        return X, y 
4.2. MLP(Sigmoid) + Swapnoise(Batch)：0.7101
class MLP_Model(tf.keras.Model):

    def __init__(self):
        super(MLP_Model, self).__init__() 
        self.dense1 =Dense(1024, activation='relu') 
        self.drop1  = Dropout(0.25)
        self.dense2 =Dense(512, activation='relu')  
        self.drop2  = Dropout(0.25) 
        self.dense_out =Dense(1,activation='sigmoid') 
     
    def call(self, inputs):
        min_target = 0
        max_target = 10.26757
        x1      = self.dense1(inputs)
        x1      = self.drop1(x1)
        x2      = self.dense2(x1)
        x2      = self.drop2(x2)
        outputs = self.dense_out(x2)
        outputs =  outputs * (max_target - min_target) + min_target 
        
        return outputs


training_generator   = DataGenerator(IDs = list(range(X_tr_dnn_linear_gaussian.shape[0])), features=X_tr_dnn_linear_gaussian.values,\
                                     labels = y_tr.values, shuffle_type=1, shuffle=True)
validation_generator = DataGenerator(IDs = list(range(X_val_dnn_linear_gaussian.shape[0])), features=X_val_dnn_linear_gaussian.values,\
                                     labels = y_val.values, shuffle_type=0, shuffle=False)


K.clear_session()
model_weights = f'./models/model_swapnoise.h5'
model_mlp = MLP_Model()

checkpoint = ModelCheckpoint(model_weights, monitor='val_loss', verbose=0, save_best_only=True, mode='min',
                             save_weights_only=True)

plateau        = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=10, verbose=1, min_delta=1e-4, mode='min')
early_stopping = EarlyStopping(monitor="val_loss", patience=20)

def RMSE(y_true, y_pred):
    return tf.sqrt(tf.reduce_mean(tf.square(y_true - y_pred)))

adam = tf.optimizers.Adam(lr=1e-3 * 2)
model_mlp.compile(optimizer=adam, loss=RMSE)

history = model_mlp.fit_generator(generator=training_generator,
                    validation_data=validation_generator,
                    epochs = 2000,
                    use_multiprocessing=True,
                    callbacks= [plateau, checkpoint, early_stopping], verbose=2,
                    workers=10) 

Epoch 1/2000
WARNING:tensorflow:Entity <bound method MLP_Model.call of <__main__.MLP_Model object at 0x7fb5b850dcd0>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Bad argument number for Name: 3, expecting 4
WARNING: Entity <bound method MLP_Model.call of <__main__.MLP_Model object at 0x7fb5b850dcd0>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Bad argument number for Name: 3, expecting 4
1875/1875 - 16s - loss: 0.7549 - val_loss: 0.7238
Epoch 2/2000
1875/1875 - 16s - loss: 0.7314 - val_loss: 0.7275
Epoch 3/2000
1875/1875 - 16s - loss: 0.7261 - val_loss: 0.7268
Epoch 4/2000
1875/1875 - 16s - loss: 0.7232 - val_loss: 0.7200
Epoch 5/2000
1875/1875 - 16s - loss: 0.7227 - val_loss: 0.7210
Epoch 6/2000
1875/1875 - 16s - loss: 0.7226 - val_loss: 0.7203
Epoch 7/2000
1875/1875 - 16s - loss: 0.7221 - val_loss: 0.7194
Epoch 8/2000
1875/1875 - 16s - loss: 0.7221 - val_loss: 0.7190
Epoch 9/2000
1875/1875 - 16s - loss: 0.7218 - val_loss: 0.7188
Epoch 10/2000
1875/1875 - 16s - loss: 0.7218 - val_loss: 0.7189
Epoch 11/2000
1875/1875 - 16s - loss: 0.7215 - val_loss: 0.7213
Epoch 12/2000
1875/1875 - 16s - loss: 0.7214 - val_loss: 0.7182
Epoch 13/2000
1875/1875 - 16s - loss: 0.7213 - val_loss: 0.7199
Epoch 14/2000
1875/1875 - 16s - loss: 0.7217 - val_loss: 0.7179
Epoch 15/2000
1875/1875 - 16s - loss: 0.7214 - val_loss: 0.7194
Epoch 16/2000
1875/1875 - 16s - loss: 0.7213 - val_loss: 0.7186
Epoch 17/2000
1875/1875 - 16s - loss: 0.7215 - val_loss: 0.7180
Epoch 18/2000
1875/1875 - 16s - loss: 0.7215 - val_loss: 0.7182
Epoch 19/2000
1875/1875 - 16s - loss: 0.7212 - val_loss: 0.7174
Epoch 20/2000
1875/1875 - 16s - loss: 0.7209 - val_loss: 0.7174
Epoch 21/2000
1875/1875 - 16s - loss: 0.7208 - val_loss: 0.7185
Epoch 22/2000
1875/1875 - 16s - loss: 0.7211 - val_loss: 0.7188
Epoch 23/2000
1875/1875 - 16s - loss: 0.7208 - val_loss: 0.7179
Epoch 24/2000
1875/1875 - 16s - loss: 0.7208 - val_loss: 0.7190
Epoch 25/2000
1875/1875 - 16s - loss: 0.7208 - val_loss: 0.7194
Epoch 26/2000
1875/1875 - 16s - loss: 0.7204 - val_loss: 0.7160
Epoch 27/2000
1875/1875 - 16s - loss: 0.7209 - val_loss: 0.7154
Epoch 28/2000
1875/1875 - 16s - loss: 0.7206 - val_loss: 0.7163
Epoch 29/2000
1875/1875 - 16s - loss: 0.7204 - val_loss: 0.7188
Epoch 30/2000
1875/1875 - 16s - loss: 0.7206 - val_loss: 0.7152
Epoch 31/2000
1875/1875 - 16s - loss: 0.7206 - val_loss: 0.7182
Epoch 32/2000
1875/1875 - 16s - loss: 0.7205 - val_loss: 0.7167
Epoch 33/2000
1875/1875 - 16s - loss: 0.7208 - val_loss: 0.7169
Epoch 34/2000
1875/1875 - 16s - loss: 0.7204 - val_loss: 0.7159
Epoch 35/2000
1875/1875 - 16s - loss: 0.7198 - val_loss: 0.7179
Epoch 36/2000
1875/1875 - 16s - loss: 0.7204 - val_loss: 0.7156
Epoch 37/2000
1875/1875 - 16s - loss: 0.7200 - val_loss: 0.7157
Epoch 38/2000
1875/1875 - 16s - loss: 0.7201 - val_loss: 0.7152
Epoch 39/2000
1875/1875 - 16s - loss: 0.7202 - val_loss: 0.7184
Epoch 40/2000

Epoch 00040: ReduceLROnPlateau reducing learning rate to 0.0010000000474974513.
1875/1875 - 16s - loss: 0.7204 - val_loss: 0.7159
Epoch 41/2000
1875/1875 - 16s - loss: 0.7192 - val_loss: 0.7150
Epoch 42/2000
1875/1875 - 16s - loss: 0.7184 - val_loss: 0.7153
Epoch 43/2000
1875/1875 - 16s - loss: 0.7183 - val_loss: 0.7160
Epoch 44/2000
1875/1875 - 16s - loss: 0.7184 - val_loss: 0.7140
Epoch 45/2000
1875/1875 - 16s - loss: 0.7183 - val_loss: 0.7139
Epoch 46/2000
1875/1875 - 16s - loss: 0.7182 - val_loss: 0.7146
Epoch 47/2000
1875/1875 - 16s - loss: 0.7177 - val_loss: 0.7146
Epoch 48/2000
1875/1875 - 16s - loss: 0.7179 - val_loss: 0.7158
Epoch 49/2000
1875/1875 - 16s - loss: 0.7179 - val_loss: 0.7141
Epoch 50/2000
1875/1875 - 16s - loss: 0.7180 - val_loss: 0.7136
Epoch 51/2000
1875/1875 - 16s - loss: 0.7176 - val_loss: 0.7128
Epoch 52/2000
1875/1875 - 16s - loss: 0.7180 - val_loss: 0.7135
Epoch 53/2000
1875/1875 - 16s - loss: 0.7175 - val_loss: 0.7135
Epoch 54/2000
1875/1875 - 16s - loss: 0.7177 - val_loss: 0.7152
Epoch 55/2000
1875/1875 - 16s - loss: 0.7179 - val_loss: 0.7133
Epoch 56/2000
1875/1875 - 16s - loss: 0.7178 - val_loss: 0.7136
Epoch 57/2000
1875/1875 - 16s - loss: 0.7175 - val_loss: 0.7158
Epoch 58/2000
1875/1875 - 16s - loss: 0.7177 - val_loss: 0.7127
Epoch 59/2000
1875/1875 - 16s - loss: 0.7172 - val_loss: 0.7135
Epoch 60/2000
1875/1875 - 16s - loss: 0.7176 - val_loss: 0.7141
Epoch 61/2000
1875/1875 - 16s - loss: 0.7174 - val_loss: 0.7133
Epoch 62/2000
1875/1875 - 16s - loss: 0.7173 - val_loss: 0.7122
Epoch 63/2000
1875/1875 - 16s - loss: 0.7175 - val_loss: 0.7131
Epoch 64/2000
1875/1875 - 16s - loss: 0.7173 - val_loss: 0.7138
Epoch 65/2000
1875/1875 - 16s - loss: 0.7172 - val_loss: 0.7135
Epoch 66/2000
1875/1875 - 16s - loss: 0.7174 - val_loss: 0.7132
Epoch 67/2000
1875/1875 - 16s - loss: 0.7170 - val_loss: 0.7126
Epoch 68/2000
1875/1875 - 16s - loss: 0.7170 - val_loss: 0.7128
Epoch 69/2000
1875/1875 - 16s - loss: 0.7171 - val_loss: 0.7148
Epoch 70/2000
1875/1875 - 16s - loss: 0.7171 - val_loss: 0.7150
Epoch 71/2000
1875/1875 - 16s - loss: 0.7173 - val_loss: 0.7129
Epoch 72/2000

Epoch 00072: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
1875/1875 - 16s - loss: 0.7170 - val_loss: 0.7124
Epoch 73/2000
1875/1875 - 16s - loss: 0.7162 - val_loss: 0.7126
Epoch 74/2000
1875/1875 - 16s - loss: 0.7161 - val_loss: 0.7121
Epoch 75/2000
1875/1875 - 16s - loss: 0.7159 - val_loss: 0.7123
Epoch 76/2000
1875/1875 - 16s - loss: 0.7159 - val_loss: 0.7123
Epoch 77/2000
1875/1875 - 16s - loss: 0.7157 - val_loss: 0.7122
Epoch 78/2000
1875/1875 - 16s - loss: 0.7164 - val_loss: 0.7116
Epoch 79/2000
1875/1875 - 16s - loss: 0.7161 - val_loss: 0.7117
Epoch 80/2000
1875/1875 - 16s - loss: 0.7156 - val_loss: 0.7122
Epoch 81/2000
1875/1875 - 16s - loss: 0.7153 - val_loss: 0.7119
Epoch 82/2000
1875/1875 - 16s - loss: 0.7159 - val_loss: 0.7117
Epoch 83/2000
1875/1875 - 16s - loss: 0.7155 - val_loss: 0.7119
Epoch 84/2000
1875/1875 - 16s - loss: 0.7155 - val_loss: 0.7122
Epoch 85/2000
1875/1875 - 16s - loss: 0.7157 - val_loss: 0.7121
Epoch 86/2000
1875/1875 - 16s - loss: 0.7150 - val_loss: 0.7120
Epoch 87/2000
1875/1875 - 16s - loss: 0.7154 - val_loss: 0.7115
Epoch 88/2000

Epoch 00088: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.
1875/1875 - 16s - loss: 0.7154 - val_loss: 0.7117
Epoch 89/2000
1875/1875 - 16s - loss: 0.7146 - val_loss: 0.7115
Epoch 90/2000
1875/1875 - 16s - loss: 0.7150 - val_loss: 0.7111
Epoch 91/2000
1875/1875 - 16s - loss: 0.7148 - val_loss: 0.7111
Epoch 92/2000
1875/1875 - 16s - loss: 0.7146 - val_loss: 0.7109
Epoch 93/2000
1875/1875 - 16s - loss: 0.7146 - val_loss: 0.7109
Epoch 94/2000
1875/1875 - 16s - loss: 0.7149 - val_loss: 0.7108
Epoch 95/2000
1875/1875 - 16s - loss: 0.7148 - val_loss: 0.7116
Epoch 96/2000
1875/1875 - 16s - loss: 0.7140 - val_loss: 0.7105
Epoch 97/2000
1875/1875 - 16s - loss: 0.7139 - val_loss: 0.7105
Epoch 98/2000
1875/1875 - 16s - loss: 0.7147 - val_loss: 0.7110
Epoch 99/2000
1875/1875 - 16s - loss: 0.7145 - val_loss: 0.7110
Epoch 100/2000
1875/1875 - 16s - loss: 0.7149 - val_loss: 0.7111
Epoch 101/2000
1875/1875 - 16s - loss: 0.7149 - val_loss: 0.7114
Epoch 102/2000
1875/1875 - 16s - loss: 0.7146 - val_loss: 0.7110
Epoch 103/2000
1875/1875 - 16s - loss: 0.7143 - val_loss: 0.7109
Epoch 104/2000
1875/1875 - 16s - loss: 0.7148 - val_loss: 0.7111
Epoch 105/2000
1875/1875 - 16s - loss: 0.7145 - val_loss: 0.7105
Epoch 106/2000

Epoch 00106: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.
1875/1875 - 16s - loss: 0.7145 - val_loss: 0.7109
Epoch 107/2000
1875/1875 - 16s - loss: 0.7143 - val_loss: 0.7108
Epoch 108/2000
1875/1875 - 16s - loss: 0.7145 - val_loss: 0.7109
Epoch 109/2000
1875/1875 - 16s - loss: 0.7143 - val_loss: 0.7105
Epoch 110/2000
1875/1875 - 16s - loss: 0.7140 - val_loss: 0.7107
Epoch 111/2000
1875/1875 - 16s - loss: 0.7140 - val_loss: 0.7107
Epoch 112/2000
1875/1875 - 16s - loss: 0.7137 - val_loss: 0.7104
Epoch 113/2000
1875/1875 - 16s - loss: 0.7142 - val_loss: 0.7111
Epoch 114/2000
1875/1875 - 16s - loss: 0.7140 - val_loss: 0.7104
Epoch 115/2000
1875/1875 - 16s - loss: 0.7140 - val_loss: 0.7105
Epoch 116/2000
1875/1875 - 16s - loss: 0.7144 - val_loss: 0.7106
Epoch 117/2000
1875/1875 - 16s - loss: 0.7142 - val_loss: 0.7103
Epoch 118/2000
1875/1875 - 16s - loss: 0.7142 - val_loss: 0.7108
Epoch 119/2000
1875/1875 - 16s - loss: 0.7139 - val_loss: 0.7103
Epoch 120/2000
1875/1875 - 16s - loss: 0.7140 - val_loss: 0.7105
Epoch 121/2000
1875/1875 - 16s - loss: 0.7141 - val_loss: 0.7103
Epoch 122/2000
1875/1875 - 16s - loss: 0.7134 - val_loss: 0.7104
Epoch 123/2000
1875/1875 - 16s - loss: 0.7136 - val_loss: 0.7101
Epoch 124/2000
1875/1875 - 16s - loss: 0.7143 - val_loss: 0.7105
Epoch 125/2000
1875/1875 - 16s - loss: 0.7141 - val_loss: 0.7106
Epoch 126/2000
1875/1875 - 16s - loss: 0.7141 - val_loss: 0.7109
Epoch 127/2000
1875/1875 - 16s - loss: 0.7140 - val_loss: 0.7105
Epoch 128/2000
1875/1875 - 16s - loss: 0.7138 - val_loss: 0.7109
Epoch 129/2000
1875/1875 - 16s - loss: 0.7139 - val_loss: 0.7103
Epoch 130/2000
1875/1875 - 16s - loss: 0.7140 - val_loss: 0.7104
Epoch 131/2000
1875/1875 - 16s - loss: 0.7141 - val_loss: 0.7106
Epoch 132/2000
1875/1875 - 16s - loss: 0.7137 - val_loss: 0.7109
Epoch 133/2000

Epoch 00133: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.
1875/1875 - 16s - loss: 0.7138 - val_loss: 0.7102
Epoch 134/2000
1875/1875 - 16s - loss: 0.7135 - val_loss: 0.7103
Epoch 135/2000
1875/1875 - 16s - loss: 0.7138 - val_loss: 0.7102
Epoch 136/2000
1875/1875 - 16s - loss: 0.7134 - val_loss: 0.7101
Epoch 137/2000
1875/1875 - 16s - loss: 0.7137 - val_loss: 0.7101
Epoch 138/2000
1875/1875 - 16s - loss: 0.7139 - val_loss: 0.7103
Epoch 139/2000
1875/1875 - 16s - loss: 0.7137 - val_loss: 0.7103
Epoch 140/2000
1875/1875 - 16s - loss: 0.7137 - val_loss: 0.7101
Epoch 141/2000
1875/1875 - 16s - loss: 0.7134 - val_loss: 0.7101
Epoch 142/2000
1875/1875 - 16s - loss: 0.7138 - val_loss: 0.7105
Epoch 143/2000

Epoch 00143: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05.
1875/1875 - 16s - loss: 0.7135 - val_loss: 0.7104
Epoch 144/2000
1875/1875 - 16s - loss: 0.7137 - val_loss: 0.7104
Epoch 145/2000
1875/1875 - 16s - loss: 0.7132 - val_loss: 0.7101
Epoch 151/2000
1875/1875 - 16s - loss: 0.7135 - val_loss: 0.7102
Epoch 152/2000
1875/1875 - 16s - loss: 0.7140 - val_loss: 0.7103
Epoch 153/2000
1875/1875 - 16s - loss: 0.7134 - val_loss: 0.7102
Epoch 154/2000
1875/1875 - 16s - loss: 0.7135 - val_loss: 0.7101
Epoch 155/2000
1875/1875 - 16s - loss: 0.7135 - val_loss: 0.7101
Epoch 156/2000
1875/1875 - 16s - loss: 0.7133 - val_loss: 0.7101
Epoch 157/2000

Epoch 00157: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05.
1875/1875 - 16s - loss: 0.7136 - val_loss: 0.7103
Epoch 158/2000
1875/1875 - 16s - loss: 0.7132 - val_loss: 0.7102
Epoch 159/2000
1875/1875 - 16s - loss: 0.7133 - val_loss: 0.7102
Epoch 160/2000
1875/1875 - 16s - loss: 0.7134 - val_loss: 0.7102
Epoch 161/2000
1875/1875 - 16s - loss: 0.7134 - val_loss: 0.7101
Epoch 162/2000
1875/1875 - 16s - loss: 0.7132 - val_loss: 0.7101
Epoch 163/2000
1875/1875 - 16s - loss: 0.7134 - val_loss: 0.7101
Epoch 164/2000
1875/1875 - 16s - loss: 0.7131 - val_loss: 0.7101
Epoch 165/2000
1875/1875 - 16s - loss: 0.7135 - val_loss: 0.7101
Epoch 166/2000
1875/1875 - 16s - loss: 0.7136 - val_loss: 0.7102
Epoch 167/2000

Epoch 00167: ReduceLROnPlateau reducing learning rate to 7.812500371073838e-06.
1875/1875 - 16s - loss: 0.7133 - val_loss: 0.7101
4.3 MLP(Sigmoid):0.7131
training_generator2   = DataGenerator(IDs = list(range(X_tr_dnn_linear_gaussian.shape[0])), features=X_tr_dnn_linear_gaussian.values,\
                                     labels = y_tr.values, shuffle_type=0, shuffle=False)
validation_generator2 = DataGenerator(IDs = list(range(X_val_dnn_linear_gaussian.shape[0])), features=X_val_dnn_linear_gaussian.values,\
                                     labels = y_val.values, shuffle_type=0, shuffle=False)


K.clear_session()
model_weights = f'./models/model_swap_nonoise.h5'
model_mlp_base = MLP_Model()

checkpoint = ModelCheckpoint(model_weights, monitor='val_loss', verbose=0, save_best_only=True, mode='min',
                             save_weights_only=True)

plateau        = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=10, verbose=1, min_delta=1e-4, mode='min')
early_stopping = EarlyStopping(monitor="val_loss", patience=20)

def RMSE(y_true, y_pred):
    return tf.sqrt(tf.reduce_mean(tf.square(y_true - y_pred)))

adam = tf.optimizers.Adam(lr=1e-3 * 2)
model_mlp_base.compile(optimizer=adam, loss=RMSE)

history = model_mlp_base.fit_generator(generator=training_generator2,
                    validation_data=validation_generator2,
                    epochs = 2000,
                    use_multiprocessing=True,
                    callbacks= [plateau, checkpoint, early_stopping], verbose=2,
                    workers=10) 

Epoch 1/2000
WARNING:tensorflow:Entity <bound method MLP_Model.call of <__main__.MLP_Model object at 0x7fb528743790>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Bad argument number for Name: 3, expecting 4
WARNING: Entity <bound method MLP_Model.call of <__main__.MLP_Model object at 0x7fb528743790>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Bad argument number for Name: 3, expecting 4
1875/1875 - 16s - loss: 0.7477 - val_loss: 0.7232
Epoch 2/2000
1875/1875 - 16s - loss: 0.7249 - val_loss: 0.7196
Epoch 3/2000
1875/1875 - 16s - loss: 0.7194 - val_loss: 0.7180
Epoch 4/2000
1875/1875 - 16s - loss: 0.7168 - val_loss: 0.7178
Epoch 5/2000
1875/1875 - 16s - loss: 0.7159 - val_loss: 0.7172
Epoch 6/2000
1875/1875 - 16s - loss: 0.7154 - val_loss: 0.7177
Epoch 7/2000
1875/1875 - 16s - loss: 0.7150 - val_loss: 0.7166
Epoch 8/2000
1875/1875 - 16s - loss: 0.7145 - val_loss: 0.7165
Epoch 9/2000
1875/1875 - 16s - loss: 0.7140 - val_loss: 0.7159
Epoch 10/2000
1875/1875 - 16s - loss: 0.7136 - val_loss: 0.7162
Epoch 11/2000
1875/1875 - 16s - loss: 0.7131 - val_loss: 0.7192
Epoch 12/2000
1875/1875 - 16s - loss: 0.7130 - val_loss: 0.7160
Epoch 13/2000
1875/1875 - 16s - loss: 0.7122 - val_loss: 0.7144
Epoch 14/2000
1875/1875 - 16s - loss: 0.7119 - val_loss: 0.7157
Epoch 15/2000
1875/1875 - 16s - loss: 0.7117 - val_loss: 0.7152
Epoch 16/2000
1875/1875 - 16s - loss: 0.7114 - val_loss: 0.7148
Epoch 17/2000
1875/1875 - 16s - loss: 0.7110 - val_loss: 0.7153
Epoch 18/2000
1875/1875 - 16s - loss: 0.7111 - val_loss: 0.7146
Epoch 19/2000
1875/1875 - 16s - loss: 0.7107 - val_loss: 0.7143
Epoch 20/2000
1875/1875 - 16s - loss: 0.7102 - val_loss: 0.7138
Epoch 21/2000
1875/1875 - 16s - loss: 0.7099 - val_loss: 0.7147
Epoch 22/2000
1875/1875 - 16s - loss: 0.7099 - val_loss: 0.7155
Epoch 23/2000
1875/1875 - 16s - loss: 0.7093 - val_loss: 0.7152
Epoch 24/2000
1875/1875 - 16s - loss: 0.7090 - val_loss: 0.7162
Epoch 25/2000
1875/1875 - 16s - loss: 0.7087 - val_loss: 0.7140
Epoch 26/2000
1875/1875 - 16s - loss: 0.7087 - val_loss: 0.7151
Epoch 27/2000
1875/1875 - 16s - loss: 0.7081 - val_loss: 0.7136
Epoch 28/2000
1875/1875 - 16s - loss: 0.7077 - val_loss: 0.7146
Epoch 29/2000
1875/1875 - 16s - loss: 0.7074 - val_loss: 0.7150
Epoch 30/2000
1875/1875 - 16s - loss: 0.7075 - val_loss: 0.7149
Epoch 31/2000
1875/1875 - 16s - loss: 0.7075 - val_loss: 0.7141
Epoch 32/2000
1875/1875 - 16s - loss: 0.7070 - val_loss: 0.7146
Epoch 33/2000
1875/1875 - 16s - loss: 0.7067 - val_loss: 0.7164
Epoch 34/2000
1875/1875 - 16s - loss: 0.7062 - val_loss: 0.7150
Epoch 35/2000
1875/1875 - 16s - loss: 0.7059 - val_loss: 0.7147
Epoch 36/2000
1875/1875 - 16s - loss: 0.7058 - val_loss: 0.7145
Epoch 37/2000

Epoch 00037: ReduceLROnPlateau reducing learning rate to 0.0010000000474974513.
1875/1875 - 16s - loss: 0.7060 - val_loss: 0.7163
Epoch 38/2000
1875/1875 - 16s - loss: 0.7020 - val_loss: 0.7131
Epoch 39/2000
1875/1875 - 16s - loss: 0.7000 - val_loss: 0.7136
Epoch 40/2000
1875/1875 - 16s - loss: 0.6998 - val_loss: 0.7142
Epoch 41/2000
1875/1875 - 16s - loss: 0.6991 - val_loss: 0.7140
Epoch 42/2000
1875/1875 - 16s - loss: 0.6986 - val_loss: 0.7139
Epoch 43/2000
1875/1875 - 16s - loss: 0.6985 - val_loss: 0.7142
Epoch 44/2000
1875/1875 - 16s - loss: 0.6980 - val_loss: 0.7140
Epoch 45/2000
1875/1875 - 16s - loss: 0.6974 - val_loss: 0.7145
Epoch 46/2000
1875/1875 - 16s - loss: 0.6971 - val_loss: 0.7146
Epoch 47/2000
1875/1875 - 16s - loss: 0.6964 - val_loss: 0.7142
Epoch 48/2000

Epoch 00048: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
1875/1875 - 16s - loss: 0.6965 - val_loss: 0.7142
Epoch 49/2000
1875/1875 - 16s - loss: 0.6936 - val_loss: 0.7142
Epoch 50/2000
1875/1875 - 16s - loss: 0.6930 - val_loss: 0.7146
Epoch 51/2000
1875/1875 - 16s - loss: 0.6927 - val_loss: 0.7143
Epoch 52/2000
1875/1875 - 16s - loss: 0.6918 - val_loss: 0.7147
Epoch 53/2000
1875/1875 - 16s - loss: 0.6916 - val_loss: 0.7147
Epoch 54/2000
1875/1875 - 16s - loss: 0.6907 - val_loss: 0.7147
Epoch 55/2000
1875/1875 - 16s - loss: 0.6906 - val_loss: 0.7147
Epoch 56/2000
1875/1875 - 16s - loss: 0.6907 - val_loss: 0.7151
Epoch 57/2000
1875/1875 - 16s - loss: 0.6898 - val_loss: 0.7150
Epoch 58/2000

Epoch 00058: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.
1875/1875 - 16s - loss: 0.6900 - val_loss: 0.7149

03

参考文章

1. https://www.kaggle.com/springmanndaniel/1st-place-turn-your-data-into-daeta

往期精彩回顾



适合初学者入门人工智能的路线及资料下载机器学习及深度学习笔记等资料打印机器学习在线手册深度学习笔记专辑《统计学习方法》的代码复现专辑
AI基础下载机器学习的数学基础专辑
本站qq群704220115，加入微信群请扫码：
风度78
关注
  • 1
    点赞
  • 踩
  • 3
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
训练NN技巧
NockinOnHeavensDoor的博客
05-04 428
神经网络的训练依赖于问题和数据。Designing and training a network using backprop requires making many seemingly arbitrary choices such as the number and types of nodes, layers, learning rates, training and test sets...
神经网络（NN）网络构建及模型算法介绍
stephon_100的博客
06-24 9333
人工神经网络neural network（NN）网络结构原理，模型及算法构建原理讲解
基于对比学习的NN语言模型训练方法
nlpgeek
08-03 883
语言模型、对比学习
图神经网络七日打卡营最后比赛的一些技巧
TEDDY的博客
11-29 1870
断点续训 分为两种，一种我定义为在线断点续训，即不保存模型到本地，停止运行后继续训练，一种是离线断点续训，即保存模型到本地，下次启动项目后接着训练。 这次比赛我尝试了好几种模型，目前好像只有SGC 模型支持 在线断点续训，其他都会报错，具体做法如下： 把exe.run(startup_program) 注释，再次运行，就是断点续训。 【PaddlePaddle】模型的保存与使用 https://blog.csdn.net/qq_41427568/article/details/87...
Kaggle—风控相关比赛
weixin_44267765的博客
02-04 2049
金融风控
gauss_rank_scaler：高斯秩缩放方法的Scikit-learn兼容实现
02-04
高斯等级定标器 scikit-learn样式转换器，可将数字变量缩放为正态分布。 神经网络的输入归一化非常重要。 高斯等级是一种将数字变量分布转换为法线的有效算法。 它基于等级转换。 第一步是为排序后的erfinv分配-1和1之间的间距，然后应用误差函数erfinv的逆函数使其看起来像高斯erfinv 。 通常，此方法比标准或最小最大缩放器更好。 重要连结 用法 Gauss Rank Scaler是完全兼容的sklearn转换器，可以在管道或现有脚本中使用。 支持的输入格式包括numpy数组和pandas数据框。 传递到变压器的所有列均已正确缩放。 例 from gauss_rank_sc
kaggle竞赛 https:__www.kaggle.com_c_nlp-getting-started.zip
最新发布
11-07
Kaggle是全球知名的机器学习和数据科学竞赛平台，它提供了一个环境让数据科学家和机器学习工程师解决实际问题并展示他们的技能。在这个“NLP入门”竞赛中，参赛者被要求利用文本数据，特别是推文，来预测灾害预警。 ...
Kaggle-Solution:Kaggle竞赛的解决方案代码
05-11
Kaggle是全球领先的数据科学和机器学习竞赛平台，它为数据科学家和机器学习工程师提供了实践和展示技能的机会。这个压缩包包含了作者在Kaggle竞赛中的实际代码，对于学习和提升数据分析、机器学习以及竞赛策略具有极...
kaggle-CrowdFlower：在Kaggle上CrowdFlower产品搜索结果相关性竞赛的第一名解决方案
02-03
Kaggle_CrowdFlower 第一名的解决方案 我们在比赛中获得的最佳单一模型是模型，其中公共LB得分为0.69322 ，私人LB得分为0.70768 。 我们最终的获奖作品是35个最佳公共LB作品的中位数合计。 该提交关于私人LB的得分0...
贷款违约预测竞赛训练数据-Kaggle竞赛
04-09
【Kaggle竞赛】是全球领先的数据科学和机器学习平台，它提供了各种数据集和竞赛，供数据科学家、机器学习工程师以及爱好者提升技能、展示才华。在这个"贷款违约预测"竞赛中，参赛者需要使用提供的训练数据（如train_...
kaggle-竞争：kaggle竞赛
02-16
在kaggle竞赛中，通常会提供一个数据集，参赛者需要分析数据、训练模型，并提交预测结果。这些比赛涵盖多个领域，如医疗诊断、金融风险评估、图像识别、自然语言处理等。通过参与kaggle竞赛，不仅可以提升个人的技术...
取消打印Tensorflow中的无用信息，如tensorflow:AutoGraph could not transform ＜*＞ and will run it as-is、加载CUDA信息等
地理信息系统、摄影测量与遥感、人工智能
05-25 9867
取消打印Tensorflow中的无用信息，如WARNING:tensorflow:AutoGraph could not transform bound method BatchNorm.call of model.BatchNorm object at 0x0000021764BB9F60 and will run it as-is、加载CUDA信息等
【深度学习1】神经网络NN原理介绍
lesleychu的博客
03-08 1859
前言 为什么我们要使用NN？因为任何函数都可以用NN来无限逼近，小到最基础的逻辑函数，可以通过调整权重，用一层NN实现，大到复杂的函数，可以通过多层NN，每层多个cells的深度神经网络来实现 概念介绍 1、常见的激活函数 2、最简单的一层NN 3、何谓“训练”NN？如何训练？ 定义：我们训练/学习的是NN每一层的weights 训练方法：Forward pass then backwar...
GPU训练NN
ScienceRecord
08-14 409
GPU训练NN   今天配置了GPU，用来训练NN，本以为很简单，不就弄俩驱动装上嘛，但是也搞了不短时间，主要就是，版本不匹配，比较坑……   Tensorflow-gpu-1.7 + CUDA-9.0 + cuDNN-7.0 forCUDA-9.0   前面说了，配置过程中最坑的是版本问题，tensorflow-gpu用最新的没问题，CUDA我一开始用最新的9.1，下载安装配置好后编译提示...
Cause: No module named ‘tensorflow_core.estimator‘（解决方法）
Welcome
12-11 8628
1. 问题描述：准备在VS code执行以下命令： x_input, x_output = gcn_lstm.in_out_tensors() 结果报错：Cause: No module named 'tensorflow_core.estimator' WARNING: AutoGraph could not transform <bound method FixedAdjacencyGraphConvolution.call of <stellargraph.layer.gcn_l
tensorflow （1）WARNING:tensorflow:AutoGraph could not transform；（2）自定义层 无训练的参数
Tak-Wah Blog
07-20 5117
背景：自定义了一个层，该层中引用了一个函数（该函数中有dense层）。 出现问题： （1）WARNING:tensorflow:AutoGraph could not transform <bound method DinAttentionLayer.call of <din.DinAttentionLayer object at 0x1487f00ea198>> and will run it as-is. Please report this to the TensorFl.
tensor flow Bad argument number for Name: 3, expecting 4
Learning from the mistakes
08-24 833
今天在运行t f1.14的tensorflow程序的时候出现了下面的warning，非常的多，比较烦： WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x7f36b1322c88>> could not be transformed and will be executed as-is. Please report this
column-wise 与 row-wise 之间有什么区别？
有关我的科研的足迹、算法竞赛的日子、生活记录。
05-04 788
中，这些实例以行的方式堆叠，形成了一个同样包含 5 层的神经网络。两种方式训练出来的模型都是一样的，只不过其中参数的排列方式不同。中，这 5 个实例以列的方式堆叠在一起，形成了一个包含 5 层的神经网络；在 PyTorch 中，“column-wise” 可以通过。函数来实现，“row-wise” 可以通过。中包含了 5 个相同的。
【数据竞赛】席卷Kaggle的调参神器，NN和树模型通吃！
fengdu78的博客
06-08 653
作者：杰少Optuna技术！简 介目前非常多的超参寻优算法都不可避免的有下面的一个或者多个问题：需要人为的定义搜索空间；没有剪枝操作，导致搜索耗时巨大；无法通过小的设置变化使其适用于大的和...
Kaggle Titanic竞赛：数据探索与预处理
项目链接在Kaggle竞赛页面和GitHub上提供，作者进行了数据探索和初步处理，以理解特征与生存率之间的关系。" 在这个Kaggle的Titanic问题中，主要涉及的机器学习知识点包括： 1. **数据预处理**：首先，数据预处理...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值