【数据竞赛】Kaggle秘技，用Sigmoid函数做回归问题！

最新推荐文章于 2022-02-21 21:02:05 发布
最新推荐文章于 2022-02-21 21:02:05 发布
阅读量1.2k 收藏 2
点赞数
文章标签： 人工智能 深度学习 过拟合 tensorflow 并行计算

作者:  尘沙樱落,杰少

基于Sigmoid的回归损失函数设计

背景

这是一个非常有意思的Loss设计，在你的问题是回归问题的时候，都可以考虑尝试使用一下，并不能保证所有的问题都能奏效，但是在某些特定的问题中却可以带来巨大的提升，最不济也可以作为一个用于后期stacking的方案。

该方案是设计者是：数据科学家danzel ，作者对于该设计奏效的原因描述如下，

I used a sigmoid-output and scaled its range afterwards (to look like the target). Training like this helps the model to converge faster and gives better results.

设计思路

假设对于我们的回归的问题为最小化平方损失，而且我们第 个标签为 ,

  • , 为我们的样本个数；

1. Baseline Loss

  • 一般都是Dense(1,activation = 'linear')的形式

2. 基于Sigmoid的Loss

  • 是Dense(1,activation = 'sigmoid') * (max_val - min_val) + min_val的形式;

  • ,

案例

上面说的究竟靠谱不靠谱呢？我们摘取kaggle数据进行实验，眼见为真。有兴趣的朋友可以去文末链接下载。

1.导入工具包

1.1导入使用的工具包

import pandas                as pd 
from sklearn.metrics         import mean_squared_error
from sklearn.model_selection import KFold
import xgboost               as xgb
from   tqdm                  import tqdm
import numpy                 as np
import pandas                as pd 
import tensorflow            as tf 
from lightgbm                import LGBMRegressor
from sklearn.model_selection import KFold
import numpy                 as np
import seaborn               as sns
from sklearn.metrics         import mean_squared_error

def RMSE(y_true, y_pred):
    return tf.sqrt(tf.reduce_mean(tf.square(y_true - y_pred)))

1.2 数据读取

train = pd.read_csv('./data/train.csv')
test  = pd.read_csv('./data/test.csv')
sub   = pd.read_csv('./data/sample_submission.csv')

2. 数据预处理

2.1 数据拼接

train_test = pd.concat([train,test],axis=0,ignore_index=True)

train_test.head()

idcont1cont2cont3cont4cont5cont6cont7cont8cont9cont10cont11cont12cont13cont14target
010.6703900.8113000.6439680.2917910.2841170.8559530.8907000.2855420.5582450.7794180.9218320.8667720.8787330.3054117.243043
130.3880530.6211040.6861020.5011490.6437900.4498050.5108240.5807480.4183350.4326320.4398720.4349710.3699570.3694848.203331
240.8349500.2274360.3015840.2934080.6068390.8291750.5061430.5587710.5876030.8233120.5670070.6777080.8829380.3030477.776091
350.8207080.1601550.5468870.7261040.2824440.7851080.7527580.8232670.5744660.5808430.7695940.8181430.9142810.2795286.957716
480.9352780.4212350.3038010.8802140.6656100.8301310.4871130.6041570.8746580.8634270.9835750.9004640.9359180.4357727.951046
2.2. 用于神经网络预处理的GaussianRank

  • 如果希望知道细节，可以参考之前分享的RankGaussian的部分

import numpy as np
from joblib import Parallel, delayed
from scipy.interpolate import interp1d
from scipy.special import erf, erfinv
from sklearn.preprocessing import QuantileTransformer,PowerTransformer
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.utils.validation import FLOAT_DTYPES, check_array, check_is_fitted

class GaussRankScaler(BaseEstimator, TransformerMixin):
    """Transform features by scaling each feature to a normal distribution.
    Parameters
        ----------
        epsilon : float, optional, default 1e-4
            A small amount added to the lower bound or subtracted
            from the upper bound. This value prevents infinite number
            from occurring when applying the inverse error function.
        copy : boolean, optional, default True
            If False, try to avoid a copy and do inplace scaling instead.
            This is not guaranteed to always work inplace; e.g. if the data is
            not a NumPy array, a copy may still be returned.
        n_jobs : int or None, optional, default None
            Number of jobs to run in parallel.
            ``None`` means 1 and ``-1`` means using all processors.
        interp_kind : str or int, optional, default 'linear'
           Specifies the kind of interpolation as a string
            ('linear', 'nearest', 'zero', 'slinear', 'quadratic', 'cubic',
            'previous', 'next', where 'zero', 'slinear', 'quadratic' and 'cubic'
            refer to a spline interpolation of zeroth, first, second or third
            order; 'previous' and 'next' simply return the previous or next value
            of the point) or as an integer specifying the order of the spline
            interpolator to use.
        interp_copy : bool, optional, default False
            If True, the interpolation function makes internal copies of x and y.
            If False, references to `x` and `y` are used.
        Attributes
        ----------
        interp_func_ : list
            The interpolation function for each feature in the training set.
        """

    def __init__(self, epsilon=1e-4, copy=True, n_jobs=None, interp_kind='linear', interp_copy=False):
        self.epsilon     = epsilon
        self.copy        = copy
        self.interp_kind = interp_kind
        self.interp_copy = interp_copy
        self.fill_value  = 'extrapolate'
        self.n_jobs      = n_jobs

    def fit(self, X, y=None):
        """Fit interpolation function to link rank with original data for future scaling
        Parameters
        ----------
        X : array-like, shape (n_samples, n_features)
            The data used to fit interpolation function for later scaling along the features axis.
        y
            Ignored
        """
        X = check_array(X, copy=self.copy, estimator=self, dtype=FLOAT_DTYPES, force_all_finite=True)

        self.interp_func_ = Parallel(n_jobs=self.n_jobs)(delayed(self._fit)(x) for x in X.T)
        return self

    def _fit(self, x):
        x = self.drop_duplicates(x)
        rank = np.argsort(np.argsort(x))
        bound = 1.0 - self.epsilon
        factor = np.max(rank) / 2.0 * bound
        scaled_rank = np.clip(rank / factor - bound, -bound, bound)
        return interp1d(
            x, scaled_rank, kind=self.interp_kind, copy=self.interp_copy, fill_value=self.fill_value)

    def transform(self, X, copy=None):
        """Scale the data with the Gauss Rank algorithm
        Parameters
        ----------
        X : array-like, shape (n_samples, n_features)
            The data used to scale along the features axis.
        copy : bool, optional (default: None)
            Copy the input X or not.
        """
        check_is_fitted(self, 'interp_func_')

        copy = copy if copy is not None else self.copy
        X = check_array(X, copy=copy, estimator=self, dtype=FLOAT_DTYPES, force_all_finite=True)

        X = np.array(Parallel(n_jobs=self.n_jobs)(delayed(self._transform)(i, x) for i, x in enumerate(X.T))).T
        return X

    def _transform(self, i, x):
        return erfinv(self.interp_func_[i](x))

    def inverse_transform(self, X, copy=None):
        """Scale back the data to the original representation
        Parameters
        ----------
        X : array-like, shape [n_samples, n_features]
            The data used to scale along the features axis.
        copy : bool, optional (default: None)
            Copy the input X or not.
        """
        check_is_fitted(self, 'interp_func_')

        copy = copy if copy is not None else self.copy
        X = check_array(X, copy=copy, estimator=self, dtype=FLOAT_DTYPES, force_all_finite=True)

        X = np.array(Parallel(n_jobs=self.n_jobs)(delayed(self._inverse_transform)(i, x) for i, x in enumerate(X.T))).T
        return X

    def _inverse_transform(self, i, x):
        inv_interp_func = interp1d(self.interp_func_[i].y, self.interp_func_[i].x, kind=self.interp_kind,
                                   copy=self.interp_copy, fill_value=self.fill_value)
        return inv_interp_func(erf(x))

    @staticmethod
    def drop_duplicates(x):
        is_unique = np.zeros_like(x, dtype=bool)
        is_unique[np.unique(x, return_index=True)[1]] = True
        return x[is_unique]
2.3 RankGaussian处理
feature_names = ['cont1', 'cont2', 'cont3', 'cont4', 'cont5', 'cont6', 'cont7','cont8', 'cont9', 'cont10', 'cont11', 'cont12', 'cont13', 'cont14']
scaler_linear    = GaussRankScaler(interp_kind='linear',) 
for c in feature_names:
    train_test[c+'_linear_grank'] = scaler_linear.fit_transform(train_test[c].values.reshape(-1,1))
    
gaussian_linear_feature_names = [c + '_linear_grank' for c in feature_names]

3. NN模型建模

from tensorflow.keras import regularizers
from sklearn.model_selection import KFold, StratifiedKFold
import tensorflow as tf
# import tensorflow_addons as tfa
import tensorflow.keras.backend as K
from tensorflow.keras.layers import *
from tensorflow.keras.models import *
from tensorflow.keras.optimizers import *
from tensorflow.keras.callbacks import *
from tensorflow.keras.layers import Input
import os 
3.1 训练&验证划分

  • 随机划分训练集和验证集

tr = train_test.iloc[:train.shape[0],:].copy()
te = train_test.iloc[train.shape[0]:,:].copy() 
kf          = KFold(n_splits=5,random_state=48,shuffle=False) 
cnt         = 0
for trn_idx, test_idx in kf.split(tr,tr['target']):
    if cnt == 0:
        cnt += 1
        continue
    X_tr_gbdt,X_val_gbdt = tr[feature_names].iloc[trn_idx],tr[feature_names].iloc[test_idx]
    X_tr_dnn_linear_gaussian,X_val_dnn_linear_gaussian = tr[gaussian_linear_feature_names].iloc[trn_idx],tr[gaussian_linear_feature_names].iloc[test_idx]
    y_tr,y_val = tr['target'].iloc[trn_idx],train['target'].iloc[test_idx]
    break

/home/inf/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_split.py:297: FutureWarning: Setting a random_state has no effect since shuffle is False. This will raise an error in 0.24. You should leave random_state to its default (None), or set shuffle=True.
  FutureWarning
3.2 MLP模型(sigmoid)：0.7108

  • 基于sigmoid的回归

class MLP_Model(tf.keras.Model): 
    def __init__(self):
        super(MLP_Model, self).__init__() 
        self.dense1 =Dense(1000, activation='relu')  
        self.drop1  = Dropout(0.25)
        self.dense2 =Dense(500, activation='relu')  
        self.drop2  = Dropout(0.25) 
        self.dense_out =Dense(1,activation='sigmoid') 

    def call(self, inputs):
        min_target = 0
        max_target = 10.26757
        x1      = self.dense1(inputs)
        x1      = self.drop1(x1)
        x2      = self.dense2(x1)
        x2      = self.drop2(x2)
        outputs      = self.dense_out(x2)
        outputs  =  outputs * (max_target - min_target) + min_target  
        return outputs


import time  
def RMSE(y_true, y_pred):
    return tf.sqrt(tf.reduce_mean(tf.square(y_true - y_pred)))

model = MLP_Model()
adam = tf.optimizers.Adam(lr=1e-3)
model.compile(optimizer=adam, loss=RMSE)

K.clear_session() 
model_weights = f'./models/model_gauss_mlp_mlp.h5'
checkpoint = ModelCheckpoint(model_weights, monitor='loss', verbose=0, save_best_only=True, mode='min',
                             save_weights_only=True)
plateau        = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=10, verbose=1, min_delta=1e-4, mode='min')
early_stopping = EarlyStopping(monitor="val_loss", patience=25)
history = model.fit(X_tr_dnn_linear_gaussian.values, y_tr.values,
                        validation_data=(X_val_dnn_linear_gaussian.values, y_val.values),
                    batch_size=1024, epochs=100,
                    callbacks=[plateau, checkpoint, early_stopping],
                    verbose=2
                   ) 

WARNING:tensorflow:Entity <bound method MLP_Model.call of <__main__.MLP_Model object at 0x7f4818de1e50>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Failed to parse source code of <bound method MLP_Model.call of <__main__.MLP_Model object at 0x7f4818de1e50>>, which Python reported as:
    def call(self, inputs):
        min_target = 0
        max_target = 10.26757
        x1      = self.dense1(inputs)
        x1      = self.drop1(x1)
        x2      = self.dense2(x1)
        x2      = self.drop2(x2)
        outputs      = self.dense_out(x2)
        outputs  =  outputs * (max_target - min_target) + min_target
#         outputs = self.dense_out(x3) # 1500 original  
        
        return outputs

This may be caused by multiline strings or comments not indented at the same level as the code.
WARNING: Entity <bound method MLP_Model.call of <__main__.MLP_Model object at 0x7f4818de1e50>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Failed to parse source code of <bound method MLP_Model.call of <__main__.MLP_Model object at 0x7f4818de1e50>>, which Python reported as:
    def call(self, inputs):
        min_target = 0
        max_target = 10.26757
        x1      = self.dense1(inputs)
        x1      = self.drop1(x1)
        x2      = self.dense2(x1)
        x2      = self.drop2(x2)
        outputs      = self.dense_out(x2)
        outputs  =  outputs * (max_target - min_target) + min_target
#         outputs = self.dense_out(x3) # 1500 original  
        
        return outputs

This may be caused by multiline strings or comments not indented at the same level as the code.
Train on 240000 samples, validate on 60000 samples
Epoch 1/100
WARNING:tensorflow:Entity <function Function._initialize_uninitialized_variables.<locals>.initialize_variables at 0x7f4818c32950> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Num'
WARNING: Entity <function Function._initialize_uninitialized_variables.<locals>.initialize_variables at 0x7f4818c32950> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Num'
240000/240000 - 1s - loss: 0.8020 - val_loss: 0.7203
Epoch 2/100
240000/240000 - 0s - loss: 0.7345 - val_loss: 0.7225
Epoch 3/100
240000/240000 - 0s - loss: 0.7290 - val_loss: 0.7183
Epoch 4/100
240000/240000 - 0s - loss: 0.7270 - val_loss: 0.7197
Epoch 5/100
240000/240000 - 0s - loss: 0.7247 - val_loss: 0.7170
Epoch 6/100
240000/240000 - 0s - loss: 0.7232 - val_loss: 0.7190
Epoch 7/100
240000/240000 - 0s - loss: 0.7227 - val_loss: 0.7157
Epoch 8/100
240000/240000 - 0s - loss: 0.7205 - val_loss: 0.7215
Epoch 9/100
240000/240000 - 0s - loss: 0.7199 - val_loss: 0.7144
Epoch 10/100
240000/240000 - 0s - loss: 0.7185 - val_loss: 0.7148
Epoch 11/100
240000/240000 - 0s - loss: 0.7175 - val_loss: 0.7176
Epoch 12/100
240000/240000 - 0s - loss: 0.7170 - val_loss: 0.7147
Epoch 13/100
240000/240000 - 0s - loss: 0.7165 - val_loss: 0.7142
Epoch 14/100
240000/240000 - 0s - loss: 0.7157 - val_loss: 0.7140
Epoch 15/100
240000/240000 - 0s - loss: 0.7150 - val_loss: 0.7132
Epoch 16/100
240000/240000 - 0s - loss: 0.7145 - val_loss: 0.7127
Epoch 17/100
240000/240000 - 0s - loss: 0.7136 - val_loss: 0.7127
Epoch 18/100
240000/240000 - 0s - loss: 0.7131 - val_loss: 0.7124
Epoch 19/100
240000/240000 - 0s - loss: 0.7126 - val_loss: 0.7165
Epoch 20/100
240000/240000 - 0s - loss: 0.7120 - val_loss: 0.7130
Epoch 21/100
240000/240000 - 0s - loss: 0.7116 - val_loss: 0.7119
Epoch 22/100
240000/240000 - 0s - loss: 0.7111 - val_loss: 0.7129
Epoch 23/100
240000/240000 - 0s - loss: 0.7104 - val_loss: 0.7129
Epoch 24/100
240000/240000 - 0s - loss: 0.7102 - val_loss: 0.7136
Epoch 25/100
240000/240000 - 0s - loss: 0.7097 - val_loss: 0.7120
Epoch 26/100
240000/240000 - 0s - loss: 0.7089 - val_loss: 0.7126
Epoch 27/100
240000/240000 - 0s - loss: 0.7084 - val_loss: 0.7154
Epoch 28/100
240000/240000 - 0s - loss: 0.7078 - val_loss: 0.7111
Epoch 29/100
240000/240000 - 0s - loss: 0.7075 - val_loss: 0.7132
Epoch 30/100
240000/240000 - 0s - loss: 0.7074 - val_loss: 0.7126
Epoch 31/100
240000/240000 - 0s - loss: 0.7062 - val_loss: 0.7129
Epoch 32/100
240000/240000 - 0s - loss: 0.7059 - val_loss: 0.7119
Epoch 33/100
240000/240000 - 0s - loss: 0.7054 - val_loss: 0.7135
Epoch 34/100
240000/240000 - 0s - loss: 0.7048 - val_loss: 0.7108
Epoch 35/100
240000/240000 - 0s - loss: 0.7048 - val_loss: 0.7116
Epoch 36/100
240000/240000 - 0s - loss: 0.7037 - val_loss: 0.7161
Epoch 37/100
240000/240000 - 0s - loss: 0.7034 - val_loss: 0.7131
Epoch 38/100
240000/240000 - 0s - loss: 0.7031 - val_loss: 0.7148
Epoch 39/100
240000/240000 - 0s - loss: 0.7022 - val_loss: 0.7113
Epoch 40/100
240000/240000 - 0s - loss: 0.7013 - val_loss: 0.7117
Epoch 41/100
240000/240000 - 0s - loss: 0.7012 - val_loss: 0.7124
Epoch 42/100
240000/240000 - 0s - loss: 0.7008 - val_loss: 0.7116
Epoch 43/100
240000/240000 - 0s - loss: 0.7001 - val_loss: 0.7124
Epoch 44/100

Epoch 00044: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
240000/240000 - 0s - loss: 0.6995 - val_loss: 0.7113
Epoch 45/100
240000/240000 - 0s - loss: 0.6962 - val_loss: 0.7116
Epoch 46/100
240000/240000 - 0s - loss: 0.6954 - val_loss: 0.7118
Epoch 47/100
240000/240000 - 0s - loss: 0.6940 - val_loss: 0.7116
Epoch 48/100
240000/240000 - 0s - loss: 0.6938 - val_loss: 0.7120
Epoch 49/100
240000/240000 - 0s - loss: 0.6930 - val_loss: 0.7118
Epoch 50/100
240000/240000 - 0s - loss: 0.6927 - val_loss: 0.7123
Epoch 51/100
240000/240000 - 0s - loss: 0.6920 - val_loss: 0.7123
Epoch 52/100
240000/240000 - 0s - loss: 0.6915 - val_loss: 0.7125
Epoch 53/100
240000/240000 - 0s - loss: 0.6912 - val_loss: 0.7144
Epoch 54/100

Epoch 00054: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.
240000/240000 - 0s - loss: 0.6905 - val_loss: 0.7146
Epoch 55/100
240000/240000 - 0s - loss: 0.6885 - val_loss: 0.7123
Epoch 56/100
240000/240000 - 0s - loss: 0.6874 - val_loss: 0.7135
Epoch 57/100
240000/240000 - 0s - loss: 0.6872 - val_loss: 0.7136
Epoch 58/100
240000/240000 - 0s - loss: 0.6868 - val_loss: 0.7138
Epoch 59/100
240000/240000 - 0s - loss: 0.6863 - val_loss: 0.7134
3.3 MLP模型(linear)：0.7137
class MLP_Model(tf.keras.Model):

    def __init__(self):
        super(MLP_Model, self).__init__() 
        self.dense1 =Dense(1000, activation='relu')  
        self.drop1  = Dropout(0.25)
        self.dense2 =Dense(500, activation='relu') 
        self.drop2  = Dropout(0.25) 
        self.dense_out =Dense(1)

    def call(self, inputs): 
        x1      = self.dense1(inputs)
        x1      = self.drop1(x1)
        x2      = self.dense2(x1)
        x2      = self.drop2(x2)
        outputs = self.dense_out(x2) 
        
        return outputs


model = MLP_Model()
adam = tf.optimizers.Adam(lr=1e-3) 
model.compile(optimizer=adam, loss=RMSE)

K.clear_session() 
model_weights = f'./models/model_gauss_mlp_mlp.h5'
checkpoint = ModelCheckpoint(model_weights, monitor='loss', verbose=0, save_best_only=True, mode='min',
                             save_weights_only=True)
plateau        = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=10, verbose=1, min_delta=1e-4, mode='min')
early_stopping = EarlyStopping(monitor="val_loss", patience=25)
history = model.fit(X_tr_dnn_linear_gaussian.values, y_tr.values,
                        validation_data=(X_val_dnn_linear_gaussian.values, y_val.values),
                    batch_size=1024, epochs=100,
                    callbacks=[plateau, checkpoint, early_stopping],
                    verbose=2
                   ) 

WARNING:tensorflow:Entity <bound method MLP_Model.call of <__main__.MLP_Model object at 0x7f48083f7d50>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Bad argument number for Name: 3, expecting 4
WARNING: Entity <bound method MLP_Model.call of <__main__.MLP_Model object at 0x7f48083f7d50>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Bad argument number for Name: 3, expecting 4
Train on 240000 samples, validate on 60000 samples
Epoch 1/100
WARNING:tensorflow:Entity <function Function._initialize_uninitialized_variables.<locals>.initialize_variables at 0x7f4818c487a0> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Num'
WARNING: Entity <function Function._initialize_uninitialized_variables.<locals>.initialize_variables at 0x7f4818c487a0> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Num'
240000/240000 - 1s - loss: 1.3292 - val_loss: 0.7767
Epoch 2/100
240000/240000 - 0s - loss: 0.8163 - val_loss: 0.7251
Epoch 3/100
240000/240000 - 0s - loss: 0.8072 - val_loss: 0.7251
Epoch 4/100
240000/240000 - 0s - loss: 0.8040 - val_loss: 0.7496
Epoch 5/100
240000/240000 - 0s - loss: 0.7997 - val_loss: 0.7324
Epoch 6/100
240000/240000 - 0s - loss: 0.7982 - val_loss: 0.7271
Epoch 7/100
240000/240000 - 0s - loss: 0.7936 - val_loss: 0.7202
Epoch 8/100
240000/240000 - 0s - loss: 0.7950 - val_loss: 0.7249
Epoch 9/100
240000/240000 - 0s - loss: 0.7914 - val_loss: 0.7284
Epoch 10/100
240000/240000 - 0s - loss: 0.7882 - val_loss: 0.7313
Epoch 11/100
240000/240000 - 0s - loss: 0.7886 - val_loss: 0.7303
Epoch 12/100
240000/240000 - 0s - loss: 0.7857 - val_loss: 0.7292
Epoch 13/100
240000/240000 - 0s - loss: 0.7855 - val_loss: 0.7257
Epoch 14/100
240000/240000 - 0s - loss: 0.7847 - val_loss: 0.7204
Epoch 15/100
240000/240000 - 0s - loss: 0.7825 - val_loss: 0.7224
Epoch 16/100
240000/240000 - 0s - loss: 0.7813 - val_loss: 0.7220
Epoch 17/100

Epoch 00017: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
240000/240000 - 0s - loss: 0.7808 - val_loss: 0.7208
Epoch 18/100
240000/240000 - 0s - loss: 0.7752 - val_loss: 0.7187
Epoch 19/100
240000/240000 - 0s - loss: 0.7743 - val_loss: 0.7234
Epoch 20/100
240000/240000 - 0s - loss: 0.7730 - val_loss: 0.7190
Epoch 21/100
240000/240000 - 0s - loss: 0.7750 - val_loss: 0.7196
Epoch 22/100
240000/240000 - 0s - loss: 0.7742 - val_loss: 0.7286
Epoch 23/100
240000/240000 - 0s - loss: 0.7722 - val_loss: 0.7198
Epoch 24/100
240000/240000 - 0s - loss: 0.7720 - val_loss: 0.7227
Epoch 25/100
240000/240000 - 0s - loss: 0.7724 - val_loss: 0.7176
Epoch 26/100
240000/240000 - 0s - loss: 0.7705 - val_loss: 0.7194
Epoch 27/100
240000/240000 - 0s - loss: 0.7689 - val_loss: 0.7206
Epoch 28/100
240000/240000 - 0s - loss: 0.7696 - val_loss: 0.7168
Epoch 29/100
240000/240000 - 0s - loss: 0.7695 - val_loss: 0.7171
Epoch 30/100
240000/240000 - 0s - loss: 0.7681 - val_loss: 0.7164
Epoch 31/100
240000/240000 - 0s - loss: 0.7676 - val_loss: 0.7225
Epoch 32/100
240000/240000 - 0s - loss: 0.7681 - val_loss: 0.7177
Epoch 33/100
240000/240000 - 0s - loss: 0.7660 - val_loss: 0.7198
Epoch 34/100
240000/240000 - 0s - loss: 0.7668 - val_loss: 0.7202
Epoch 35/100
240000/240000 - 0s - loss: 0.7653 - val_loss: 0.7160
Epoch 36/100
240000/240000 - 0s - loss: 0.7647 - val_loss: 0.7248
Epoch 37/100
240000/240000 - 0s - loss: 0.7638 - val_loss: 0.7173
Epoch 38/100
240000/240000 - 0s - loss: 0.7626 - val_loss: 0.7197
Epoch 39/100
240000/240000 - 0s - loss: 0.7624 - val_loss: 0.7182
Epoch 40/100
240000/240000 - 0s - loss: 0.7615 - val_loss: 0.7195
Epoch 41/100
240000/240000 - 0s - loss: 0.7621 - val_loss: 0.7195
Epoch 42/100
240000/240000 - 0s - loss: 0.7616 - val_loss: 0.7192
Epoch 43/100
240000/240000 - 0s - loss: 0.7604 - val_loss: 0.7162
Epoch 44/100
240000/240000 - 0s - loss: 0.7592 - val_loss: 0.7152
Epoch 45/100
240000/240000 - 0s - loss: 0.7600 - val_loss: 0.7193
Epoch 46/100
240000/240000 - 0s - loss: 0.7594 - val_loss: 0.7206
Epoch 47/100
240000/240000 - 0s - loss: 0.7578 - val_loss: 0.7201
Epoch 48/100
240000/240000 - 0s - loss: 0.7583 - val_loss: 0.7164
Epoch 49/100
240000/240000 - 0s - loss: 0.7581 - val_loss: 0.7163
Epoch 50/100
240000/240000 - 0s - loss: 0.7572 - val_loss: 0.7163
Epoch 51/100
240000/240000 - 0s - loss: 0.7554 - val_loss: 0.7166
Epoch 52/100
240000/240000 - 0s - loss: 0.7564 - val_loss: 0.7212
Epoch 53/100
240000/240000 - 0s - loss: 0.7560 - val_loss: 0.7156
Epoch 54/100

Epoch 00054: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.
240000/240000 - 0s - loss: 0.7547 - val_loss: 0.7180
Epoch 55/100
240000/240000 - 0s - loss: 0.7530 - val_loss: 0.7154
Epoch 56/100
240000/240000 - 0s - loss: 0.7534 - val_loss: 0.7150
Epoch 57/100
240000/240000 - 0s - loss: 0.7531 - val_loss: 0.7148
Epoch 58/100
240000/240000 - 0s - loss: 0.7530 - val_loss: 0.7156
Epoch 59/100
240000/240000 - 0s - loss: 0.7523 - val_loss: 0.7166
Epoch 60/100
240000/240000 - 0s - loss: 0.7522 - val_loss: 0.7152
Epoch 61/100
240000/240000 - 0s - loss: 0.7520 - val_loss: 0.7155
Epoch 62/100
240000/240000 - 0s - loss: 0.7514 - val_loss: 0.7148
Epoch 63/100
240000/240000 - 0s - loss: 0.7514 - val_loss: 0.7149
Epoch 64/100
240000/240000 - 0s - loss: 0.7506 - val_loss: 0.7156
Epoch 65/100
240000/240000 - 0s - loss: 0.7508 - val_loss: 0.7150
Epoch 66/100
240000/240000 - 0s - loss: 0.7516 - val_loss: 0.7154
Epoch 67/100

Epoch 00067: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.
240000/240000 - 0s - loss: 0.7507 - val_loss: 0.7153
Epoch 68/100
240000/240000 - 0s - loss: 0.7502 - val_loss: 0.7149
Epoch 69/100
240000/240000 - 0s - loss: 0.7497 - val_loss: 0.7147
Epoch 70/100
240000/240000 - 0s - loss: 0.7496 - val_loss: 0.7148
Epoch 71/100
240000/240000 - 0s - loss: 0.7502 - val_loss: 0.7142
Epoch 72/100
240000/240000 - 0s - loss: 0.7492 - val_loss: 0.7148
Epoch 73/100
240000/240000 - 0s - loss: 0.7487 - val_loss: 0.7148
Epoch 74/100
240000/240000 - 0s - loss: 0.7485 - val_loss: 0.7143
Epoch 75/100
240000/240000 - 0s - loss: 0.7496 - val_loss: 0.7154
Epoch 76/100
240000/240000 - 0s - loss: 0.7482 - val_loss: 0.7144
Epoch 77/100
240000/240000 - 0s - loss: 0.7488 - val_loss: 0.7142
Epoch 78/100
240000/240000 - 0s - loss: 0.7492 - val_loss: 0.7145
Epoch 79/100
240000/240000 - 0s - loss: 0.7483 - val_loss: 0.7143
Epoch 80/100
240000/240000 - 0s - loss: 0.7478 - val_loss: 0.7143
Epoch 81/100

Epoch 00081: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.
240000/240000 - 0s - loss: 0.7481 - val_loss: 0.7143
Epoch 82/100
240000/240000 - 0s - loss: 0.7480 - val_loss: 0.7146
Epoch 83/100
240000/240000 - 0s - loss: 0.7477 - val_loss: 0.7141
Epoch 84/100
240000/240000 - 0s - loss: 0.7471 - val_loss: 0.7139
Epoch 85/100
240000/240000 - 0s - loss: 0.7475 - val_loss: 0.7140
Epoch 86/100
240000/240000 - 0s - loss: 0.7473 - val_loss: 0.7141
Epoch 87/100
240000/240000 - 0s - loss: 0.7469 - val_loss: 0.7141
Epoch 88/100
240000/240000 - 0s - loss: 0.7474 - val_loss: 0.7148
Epoch 89/100
240000/240000 - 0s - loss: 0.7467 - val_loss: 0.7138
Epoch 90/100
240000/240000 - 0s - loss: 0.7466 - val_loss: 0.7142
Epoch 91/100
240000/240000 - 0s - loss: 0.7460 - val_loss: 0.7141
Epoch 92/100
240000/240000 - 0s - loss: 0.7465 - val_loss: 0.7138
Epoch 93/100
240000/240000 - 0s - loss: 0.7469 - val_loss: 0.7142
Epoch 94/100
240000/240000 - 0s - loss: 0.7467 - val_loss: 0.7141
Epoch 95/100
240000/240000 - 0s - loss: 0.7465 - val_loss: 0.7148
Epoch 96/100
240000/240000 - 0s - loss: 0.7465 - val_loss: 0.7138
Epoch 97/100
240000/240000 - 0s - loss: 0.7461 - val_loss: 0.7138
Epoch 98/100
240000/240000 - 0s - loss: 0.7456 - val_loss: 0.7140
Epoch 99/100

Epoch 00099: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05.
240000/240000 - 0s - loss: 0.7463 - val_loss: 0.7139
Epoch 100/100
240000/240000 - 0s - loss: 0.7461 - val_loss: 0.7137

参考文献

  1. https://www.kaggle.com/c/tabular-playground-series-jan-2021/data

  2. https://www.kaggle.com/c/tabular-playground-series-jan-2021/discussion/216037

往期精彩回顾



适合初学者入门人工智能的路线及资料下载机器学习及深度学习笔记等资料打印机器学习在线手册深度学习笔记专辑《统计学习方法》的代码复现专辑
AI基础下载机器学习的数学基础专辑
本站知识星球“黄博的机器学习圈子”（92416895）
本站qq群704220115。
加入微信群请扫码：
风度78
关注
  • 0
    点赞
  • 踩
  • 2
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
kaggle实战_4解决高维数据分类/回归问题–房价预测
12-21
Note:由房价预测例子的学到，用Stacking的思维来汲取两种或者多种模型的优点 ipython的代码和数据集在我的GitHub中，链接在下面，下面的代码是在pycharm里运行的，差别不大。 #Step 1: 检视源数据集 import numpy ...
sigmoid和tanh做有限范围内的回归问题
袁一白的博客
09-06 1798
前几天，有人质疑说sigmoid不能做回归，我翻了一下资料 某自编码机源码，采用sigmoid+BCEloss进行矩阵重建【这个矩阵数据被变换到了0-1】，论文里面确实是采用sigmoid做回归 本CSDN，采用distilbert用sigmoid做回归，完成STSB任务，（回归到1-5） 其实可以推导一下：sigmoid+BCE，大致等于直接回归+MSE ++++++++++++++++++以下实验正文++++++++++++++++++ sigmoid回归 # 环境配置 !pip install tra
linux常用端口对照
joan的专栏
10-25 1795
linux常用端口对照 著名端口 1 tcpmux TCP 端口服务多路复用 5 rje 远程作业入口 7 echo Echo 服务 9 discard 用于连接测试的空服务 11 systat 用于列举连接了的端口的系统状态 13 daytime 给请求主机发送日期和时间 17 qotd 给连接了的主机发送每日格言 18 msp 消息发送协议 19 chargen 字符生成服务；发送无止境...
收藏网址
热门推荐
u011569805的专栏
11-05 82万+
It will be read and overwritten.      DO NOT EDIT! --> Bookmarks Bookmarks     书签栏             桌面支持中心         从 IE 中导入                     桌面支持中心             入职                    
TensorFlow2使用AutoGraph时出现关于gast的一个warning
adanox的博客
11-16 4519
在用tf.keras的model训练数据使，貌似用到了TensorFlow的AutoGraph，出现了warning WARNING:tensorflow:Entity <function Function._initialize_uninitialized_variables.<locals>.initialize_variables at 0x00000132B7...
azchn.com+app.html,Analysis.ipynb
weixin_34518190的博客
07-02 41万+
{"cells": [{"cell_type": "code","execution_count": 1,"metadata": {},"outputs": [],"source": ["import torch\n","import matplotlib.pyplot as plt\n","import cv2\n","import pandas as pd"]},{"cell_type": "...
贷款违约预测竞赛数据【Kaggle竞赛】.zip
06-29
在数据分析和机器学习领域，Kaggle竞赛是全球知名的平台，它为专业人士提供了一系列的数据挑战，包括本例中的“贷款违约预测”比赛。这个压缩包文件包含的数据集是为了帮助参赛者构建模型，预测哪些贷款申请可能会...
Kaggle 各项竞赛情况数据数据集
07-07
本数据集名为“Kaggle各项竞赛情况数据数据集”，旨在提供有关Kaggle平台上竞赛的全面信息，帮助参赛者、组织者以及研究者更好地理解和参与数据竞赛。 1. **竞赛信息**：数据集中包含了每个竞赛的详细信息，如竞赛...
猫和狗图像分类数据【Kaggle竞赛】.zip
07-04
《猫和狗图像分类数据——Kaggle竞赛深度解析》 在计算机视觉领域，图像分类是一项基础且重要的任务，它涉及到让机器理解并识别图片中的物体。Kaggle作为全球领先的机器学习竞赛平台，曾举办了一场名为“猫和狗图像...
机器学习之逻辑回归算法详解（Sigmoid函数、逻辑回归的损失函数、梯度下降 、逻辑回归的正则化、逻辑回归API中的超参数介绍）
weixin_40247876的博客
06-07 5777
机器学习19_逻辑回归算法详解（2021.06.07 ~） 一. 究极总结 逻辑回归：一个打着回归旗号，却在做分类任务的分类器。 二. 必备知识复习 逻辑回归在本质上其实是由线性回归衍变而来的，因此想要更好地理解逻辑回归，先让我们复习一下线性回归。线性回归的原理其实就是用一个线性回归方程来描述一个线性回归的问题。而线性回归的方程可以写作一个几乎大家都熟悉的方程： 在这个方程中，w0被称作截距，w1 ~ wn 被称作系数，这个线性回归方程也可以用矩阵来表示： 通过求出函数z，线性回归便完成了使用输入的特征
gauss_rank_scaler：高斯秩缩放方法的Scikit-learn兼容实现
02-04
高斯等级定标器 scikit-learn样式转换器，可将数字变量缩放为正态分布。 神经网络的输入归一化非常重要。 高斯等级是一种将数字变量分布转换为法线的有效算法。 它基于等级转换。 第一步是为排序后的erfinv分配-1和1之间的间距，然后应用误差函数erfinv的逆函数使其看起来像高斯erfinv 。 通常，此方法比标准或最小最大缩放器更好。 重要连结 用法 Gauss Rank Scaler是完全兼容的sklearn转换器，可以在管道或现有脚本中使用。 支持的输入格式包括numpy数组和pandas数据框。 传递到变压器的所有列均已正确缩放。 例 from gauss_rank_sc
scaler算法
03-02
最好的SCALER的算法，和在视频处理的应用；
逻辑 回归
weixin_46528566的博客
02-21 860
逻辑回归 我们之前讲述了线性回归，那么这里面引入逻辑回归，逻辑回归更侧重于数据的分类。 我们在线性回归中所说的无论是一元线性回归，多元线性回归或者是多项式回归，其函数构造成代价函数之后可能是非凸函数，所以使用 梯度下降法可能会造成求出的是局部最小值。 所以我们在参数回归中所用到的函数是 sigmoid函数，而且是用参数矩阵和sigmoid函数所结合： 在图像中我们用0.5作为边界，大于0.5的都可以归到1里面，小于0.5的都可以归于0中，便于分类。 我们想要想到决策边界，就是就数据分成不同类型的边界，
http://mail.163.com/help/help_spam_16.htm?ip=118.186.207.7&hostid=smtp5&time=1358341921
云计算?
01-16 66万+
0INRMumLsiQwT0xVgvYVmNCBWS7mV8LzSeLOZGHzflL3ziBSx+iej3G1syAeYvPZxqagQ0P7mgdX /qgnEWWuIcv4cTR6ZI5QNmqULAGtRkCtCNsphAD7cLBiV7UpV7yvURpKw6H8jHyDDZc8zP5P8QF9 abbRoeoTcPcDs/Ij0+JSX9fkdkqCvmUFYzy/GBb+hMWJ...
volley+viewpagaer+fragment实现左右滑动加载图片
bruceyang的博客
12-23 42万+
首先、主布局是一个viewpager <FrameLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="match_parent" android:layout_height="match_parent"> <android.support.v4.
tensorflow2.0学习记录-三种建模方式Three modeling methods
batmanchen的博客
06-02 563
本篇blog介绍了tensorflow2.0的三种建模方式。 初学者建议前两种，学到后期强烈建议后一种，可以高度自定义。 顺序模型Sequential model tf.keras.Sequential 就像搭积木一样，一层一层往上搭建。 第一种搭积木方式： from tensorflow.keras import layers import tensorflow as tf model = tf.keras.Sequential() model.add(layers.Dense(64, activati
python3爬取图片
onj123的博客
07-28 5470
对http://code.py40.com/42.html的爬虫代码进行改造，添加多线程，多进程下载，图片完整性判断，爬取百度图片分析 import multiprocessing import re import os import requests import time from multiprocessing import Process, Queue, cpu_count #使用P...
Windows 服务器系统的服务概述和网络端口要求
weixin_33976072的博客
07-25 240
Windows 服务器系统的服务概述和网络端口要求文章编号 : 832017 最后修改 : 2006年7月20日 修订 : 21.0 概述系统服务端口Active Directory（本地安全机构）应用程序层网关服务ASP.NET 状态服务证书服务群集服务计算机浏览器DHCP 服务器分布式文件系统分布式文件系统复制分布式链接跟踪服务器分布式事务处理协调器端口和协议Active Directory ...
TCP Ports list (3498 ports in list)
zyb378747350的专栏
01-11 40万+
参考地址:https://www.gasmi.net/tcp.php TCP Ports list (3498 ports in list) 1 tcpmux TCP Port Service Multiplexer 2 compressnet Management Utility 3 compressnet Compression Process
逻辑回归数据集在kaggle里可以用
最新发布
01-08
逻辑回归是一种常用的分类算法，在Kaggle上有很多数据集可以用来进行逻辑回归模型的训练和测试。这些数据集涵盖了各种领域和主题，比如金融、医疗、零售等等，可以满足不同研究和应用的需求。 在Kaggle上，用户可以通过搜索功能找到各种适合逻辑回归的数据集，然后下载到本地进行数据清洗和预处理。接着可以使用Python或R等编程语言的数据分析和机器学习库，比如pandas、scikit-learn等，来建立逻辑回归模型并进行训练和评估。 Kaggle还提供了丰富的教程和竞赛项目，用户可以参与这些项目来练习和提升逻辑回归模型的应用能力。同时，用户也可以提交自己的逻辑回归模型在Kaggle上进行排名和比赛，从而与全球的数据科学家和机器学习爱好者交流和竞争。 总之，在Kaggle上可以找到丰富的逻辑回归数据集和相关资源，对于想要学习和应用逻辑回归的用户来说是一个非常理想的平台。通过实际的数据集和项目实践，可以更好地理解和掌握逻辑回归算法的原理和实际应用技巧。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值