qq_22667507的博客

一、学习知识点概括模型融合可以集合各种模型，从而提高模型的准确率二、学习内容1.投票融合：Voting即投票机制，分为软投票和硬投票两种，其原理采用少数服从多数的思想。hard_vote = VotingClassifier(estimators=[('xgb', clf1), ('rf', clf2), ('svc', clf3)], voting='hard')eclf = VotingClassifier(estimators=[('xgb', clf1), ('rf', clf2), (

02|CART的生成：决策树的生成就是递归地构建二叉决策树的过程。对回归树用平方差最小化准则，对分类树用基尼指数最小化准则，进行特征选择，生成二叉树。分类树与回归树的一个区别是：如果目标变量是离散型变量则用分类树，如果目标变量是连续型变量则用回归树。2.1回归树的生成回归树是用于目标变量是连续型变量的情况下，假设X与Y分别为输入和输出变量，并且Y是连续型变量，给定数据即D={(x1,y1),(x2,y2),…(xn,yn)}，根据训练数据集D生成决策树。前面说过，回归树的生成准则是平方差(总离差平

模型正则化之岭回归：实际上就是在损失函数后面加一个惩罚函数，求J(theta)的最小值

对数据先升维+再归一化+再线性回归 = 多项式回归pipeline将三个步骤封装到一起：

import numpy as npimport matplotlib.pyplot as plt### scikit-learn中的PCAfrom sklearn.decomposition import PCAx = np.empty((100,2))x[:,0] = np.random.uniform(0.,100,size = 100)x[:,1] = 0.75 * x[:,0] + 3. + np.random.normal(0,10,size = 100)pca = PC

手写一个简单的求前n个主成分的pca算法：x = np.empty((100,2))x[:,0] = np.random.uniform(0.,100,size = 100)x[:,1] = 0.75 * x[:,0] + 3. + np.random.normal(0,10,size = 100)#目标函数：找w使得返回值最大def f(w,x): return np.sum((x.dot(w)**2)) / len(x)#梯度的求解方法：def df_math(w,x):

#简易的随机梯度下降：def dj_sgd(theta,x_b_i,y_i): return x_b_i.T.dot(x_b_i.dot(theta) - y_i) * 2def sgd(x_b,y,initial_theta,n_iters): t0 = 5 t1 = 50 def learing_rate(t): return t0/(t+t1) theta = initial_theta for cur_iters in

class LinearRegression: def __init__(self): self.interception = None self.coef = None self._theta = None def fit(self,xtrain,ytrain): assert xtrain.shape[0] == ytrain.shape[0],'error' x_b = np.hstack([np

from sklearn import datasetsfrom sklearn.model_selection import train_test_splitfrom sklearn.preprocessing import StandardScalerdata = datasets.load_iris()data.keys()data_x = data.datadata_y = data.targetxtrain,xtest,ytrain,ytest = train_test_spli

from sklearn.model_selection import GridSearchCVparam_grid = [{'weights':['uniform'], 'n_neighbors':[i for i in range(1,11)]}, {'weights':['distance'], 'n_neighbors':[i for i in range(1,11)], 'p':[i f

原理：返回欧式距离中最近的k个点中 分类数目最多的那个label#定义一个knn函数：class Knnclassify: def __init__(self,k): assert k>0,'error' self.k = k self._xtrain = None self._ytrain = None def fit(self,xdata,ydata): assert xdata.

sklearn中的train_test_split：from sklearn.model_selection import train_test_splitdata_train,data_test,label_train,label_test = train_test_split(x,y,test_size=0.3,random_state=123)test = KNeighborsClassifier(n_neighbors=4)test.fit(data_train,label_train)