多因子数据的处理:主要包括三种方式-----------去极值;标准化;中性化
1.去极值
去极值不是删除而是拉回正常值
(1)分位数去极值
(2)3倍中位数去极值(3mad)
(3)正太分布去极值(3sigma)
首先安装以下需要的工具包
from scipy.stats.mstats import winsorize
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression
import numpy as np
import pandas as pd
(1)分位数去极值
获取数据,选择revenue和cost_of_goods_sold两个特征
q = query(fundamentals.income_statement.revenue,
fundamentals.income_statement.cost_of_goods_sold
)
fund = get_fundamentals(q, entry_date="2017-01-03")
#换成截面数据
fund[:,"2017-01-03",:]
#对pe_ratio去极值
#分位数去极值
fund = pe_ratio = get_fundamentals(query(fundamentals.eod_derivative_indicator.pe_ratio), entry_date="2018-01-03")[:,"2018-01-03",:]
fund['pe_ratio_winsorize'] = winsorize(fund['pe_ratio'], limits=0.025)
选取500个数据查看情况
fund['pe_ratio'][:500].plot()
fund['pe_ratio_winsorize'][