Lzj000lzj的博客

缺失值填充SimpleImputer(missing_values=nan, strategy=’mean’, fill_value=None, verbose=0, copy=True)1. missing_values:number，string,nan2. strategy:mean,median,most_frequent,constant3. fill_value:constan...

数据决定上限，模型逼近上限类别特征类别特征的特点：当类别基数较大时在处理后会产生非常稀疏的特征，难以处理缺失值。onehot encodingonehot之前要去掉NAN值label encoding对非线性树算法有用不增加维度count encoding将类别名替换为这一类别出现的次数对离群点很敏感可以尝试加入 log平滑可能会引入共线性LabelCoun...

cleaning大小写同一解码去特殊符号修剪单词内部的错误符号tokenizingtokenizeN-Gramsskip-gramschar-gramsremove清除停用词，稀有单词roots修剪单词的前几个字符修正单词回词根...

features = {key:np.array(value) for key,value in dict(features).items()}

数据加载import numpy as npimport pandas as pdfrom sklearn.model_selection import train_test_splitfrom sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, confusion_matrixf...

数据加载import numpy as np # linear algebraimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)import os# print(os.listdir("../input"))# print(os.listdir("../input/crawl300d2m"))...

离群点单变量分析首先用标准化（标准化不会改变数据相对分布的特性）把数据转变成正态分布，分别查看最大和最小的十个值saleprice_scaled = StandardScaler().fit_transform(df_train['SalePrice'][:,np.newaxis]);low_range = saleprice_scaled[saleprice_scaled[:,0].ar...

当某个特征只有很少类型的取值，而且99%都是其中一类的取值时，这种特征可以删除