1. 单变量特征衍生
1.1 数据重编码
连续变量 标准化:0-1标准化、Z-Score标准化 离散化:等距分箱、等频分箱、聚类分箱 离散变量 非数值->数值:自然数编码、字典编码 列->新列:独热编码、哑变量变换
1.2 高阶多项式
原理 X >>> X2 、X3 、X4 … … 代码实现 可以手动实现,也可利用sklearn中的PolynomialFeature评估器实现。
from sklearn. preprocessing import PolynomialFeatures
import numpy as np
x1 = np. array( [ 1 , 2 , 3 ] )
x1. reshape( - 1 , 1 )
'''
array([[1],
[2],
[3]])
'''
PolynomialFeatures( degree= 5 ) . fit_transform( x1. reshape( - 1 , 1 ) )
'''
array([[ 1., 1., 1., 1., 1., 1.],
[ 1., 2., 4., 8., 16., 32.],
[ 1., 3., 9., 27., 81., 243.]])
'''
2. 双变量特征衍生
2.1 四则运算
X1, X2 >>> X1+X2, X1-X2, X1*X2, X1/X2
2.2 多项式衍生
2.2.1 导包 & 数据
from sklearn. preprocessing import PolynomialFeatures
import pandas as pd
df = pd. DataFrame( { 'X1' : [ 1 , 2 , 3 ] , 'X2' : [ 2 , 3 , 4 ] } )
df
'''
X1 X2
0 1 2
1 2 3
2 3 4
'''
2.2.2 二阶衍生
'''
多项式衍生
X1,X2 >>> X1,X2 | X1^2,X1*X2,X2^2
X1,X2,X3 >>> X1,X2,X3 | X1^2,X1*X2,X1*X3,X2^2,X2*X3,X3^2
include_bias 默认True,包含特征的0次方
interaction_only 默认False,True则只创建交叉项
'''
PolynomialFeatures( degree= 2 , include_bias= False ) . fit_transform( df)
'''
array([[ 1., 2., 1., 2., 4.],
[ 2., 3., 4., 6., 9.],
[ 3., 4., 9., 12., 16.]])
'''
2.2.3 三阶衍生
'''
X1,X2 >>> X1,X2 | X1^2,X1*X2,X2^2 | X1^3,X1^2*X2,X1*X2^2,X2^3
'''
PolynomialFeatures( degree= 3 , include_bias= False ) . fit_transform( df)
'''
array([[ 1., 2., 1., 2., 4., 1., 2., 4., 8.],
[ 2., 3., 4., 6., 9., 8., 12., 18., 27.],
[ 3., 4., 9., 12., 16., 27., 36., 48., 64.]])
'''
创建特征的同时创建列名称
df = pd. DataFrame( { 'X1' : [ 1 , 2 , 3 ]
, 'X2' : [ 2 , 3 , 4 ]
, 'X3' : [ 1 , 0 , 0 ] } )
df
'''
X1 X2 X3
0 1 2 1
1 2 3 0
2 3 4 0
'''
colNames = [ 'X1' , 'X2' ]
degree = 3
colNames_new = [ ]
for deg in range ( 2 , degree+ 1 ) :
for i in range ( deg+ 1 ) :
col_temp = colNames[ 0 ] + '^' + str ( deg- i) + '*' + colNames[ 1 ] + '^' + str ( i)
colNames_new. append( col_temp)
colNames_new
'''
['X1^2*X2^0',
'X1^1*X2^1',
'X1^0*X2^2',
'X1^3*X2^0',
'X1^2*X2^1',
'X1^1*X2^2',
'X1^0*X2^3']
'''
3. 交叉组合特征衍生
3.1 导包 & 数据
df = pd. DataFrame( { 'SeniorCitizen' : [ 0 , 0 , 0 , 0 , 0 ]
, 'Partner' : [ 'Yes' , 'No' , 'No' , 'No' , 'No' ]
, 'Dependents' : [ 'No' , 'No' , 'No' , 'No' , 'No' ] } )
df
'''
SeniorCitizen Partner Dependents
0 0 Yes No
1 0 No No
2 0 No No
3 0 No No
4 0 No No
'''
3.2 生成衍生列和名称
colNames = [ 'SeniorCitizen' , 'Partner' , 'Dependents' ]
colNames_new_l = [ ]
features_new_l = [ ]
for col_index, col_name in enumerate ( colNames) :
print ( col_index, col_name)
'''
0 SeniorCitizen
1 Partner
2 Dependents
'''
for col_index, col_name in enumerate ( colNames) :
for col_sub_index in range ( col_index+ 1 , len ( colNames) ) :
newNames = col_name + ' & ' + colNames[ col_sub_index]
print ( newNames)
'''
SeniorCitizen & Partner
SeniorCitizen & Dependents
Partner & Dependents
'''
for col_index, col_name in enumerate ( colNames) :
for col_sub_index in range ( col_index+ 1 , len ( colNames) ) :
newNames = col_name + '&' + colNames[ col_sub_index]
colNames_new_l. append( newNames)
newDF = pd. Series( df[ col_name] . astype( 'str' )
+ '&'
+ df[ colNames[ col_sub_index] ] . astype( 'str' )
, name= newNames)
features_new_l. append( newDF)
features_new = pd. concat( features_new_l, axis= 1 )
features_new
'''
SeniorCitizen&Partner SeniorCitizen&Dependents Partner&Dependents
0 0&Yes 0&No Yes&No
1 0&No 0&No No&No
2 0&No 0&No No&No
3 0&No 0&No No&No
4 0&No 0&No No&No
'''
4. 分组统计特征衍生
4.1 分组统计原理
'''
B特征根据A特征的不同取值进行分组统计,
统计量可以是均值、方差等(针对连续变量)或众数、分位数等(针对离散变量).
如: Monthly Charges根据tenure取值进行分组统计
ID tenure Monthly_Charges | mean min max
1 1 3 | 5 3 7
2 2 10 | 11 10 12
3 3 15 | 15 15 15
4 2 12 | 11 10 12
5 1 7 | 5 3 7
'''
'''
1. B特征可离散也可连续,A特征必须离散.
A最好是取值较多的离散变量(固定取值的连续变量),否则重复行多.
2. 计算B特征分组统计量时,不局限于连续特征只用连续变量统计量、离散特征只用离散变量统计量.
离散也可用均值、方差、偏度、峰度等,连续也可用众数、分位数等.
3. 分组统计可用于多表连接.
ID 消费金额 消费品类 ID 消费金额_均值 消费金额_总额 消费品类_众数
1 11 A 1 10.6667 32 C
2 5 B >>> 2 7.33333 22 A
3 1 A 3 1 2 A
4. 在分组统计的基础上进一步四则运算特征衍生
ID tenure Monthly_Charges | mean | Monthly_Charges-mean Monthly_Charges/mean
1 1 3 | 5 | -2 0.6
2 2 10 | 11 | -1 0.909
3 3 15 | 15 | 0 1
'''
4.2 过程
4.2.1 数据准备
data = pd. DataFrame( { 'tenure' : [ 1 , 3 , 2 , 4 , 2 , 3 , 4 ]
, 'SeniorCitizen' : [ 0 , 0 , 2 , 0 , 0 , 1 , 3 ]
, 'MonthlyCharges' : [ 29.85 , 56.95 , 53.85 , 42.30 , 70.70 , 73.12 , 76.37 ]
, 'gender' : [ 'F' , 'M' , 'M' , 'F' , 'M' , 'M' , 'F' ] } )
colNames = [ 'tenure' , 'SeniorCitizen' , 'MonthlyCharges' ]
features_temp = data[ colNames]
features_temp
tenure SeniorCitizen MonthlyCharges 0 1 0 29.85 1 3 0 56.95 2 2 2 53.85 3 4 0 42.30 4 2 0 70.70 5 3 1 73.12 6 4 3 76.37
4.2.2 单统计变量衍生
features_temp. groupby( 'tenure' ) . mean( )
SeniorCitizen MonthlyCharges tenure 1 0.0 29.850 2 1.0 62.275 3 0.5 65.035 4 1.5 59.335
4.2.3 多统计变量衍生(新数据+新列名)
colNames = [ 'tenure' , 'SeniorCitizen' , 'MonthlyCharges' ]
colNames_sub = [ 'SeniorCitizen' , 'MonthlyCharges' ]
aggs = { }
for col in colNames_sub:
aggs[ col] = [ 'mean' , 'min' , 'max' ]
aggs
'''
{'SeniorCitizen': ['mean', 'min', 'max'],
'MonthlyCharges': ['mean', 'min', 'max']}
'''
cols = [ 'tenure' ]
for key in aggs. keys( ) :
cols. extend( [ key+ '_' + 'tenure' + '_' + stat for stat in aggs[ key] ] )
cols
'''
['tenure',
'SeniorCitizen_tenure_mean',
'SeniorCitizen_tenure_min',
'SeniorCitizen_tenure_max',
'MonthlyCharges_tenure_mean',
'MonthlyCharges_tenure_min',
'MonthlyCharges_tenure_max']
'''
'''
df.agg('mean') 计算列均值
df.agg('mean', axis=1) 计算行均值
df.agg({'行名/列名1': ['mean', 'min', 'max']
,'行名/列名2':['函数名1', '函数名2', '函数名3']})
df.reset_index() 更新索引,'tenure'成为特征列
'''
features_new = features_temp. groupby( 'tenure' ) . agg( aggs) . reset_index( )
features_new
tenure SeniorCitizen MonthlyCharges mean min max mean min max 0 1 0.0 0 0 29.850 29.85 29.85 1 2 1.0 0 2 62.275 53.85 70.70 2 3 0.5 0 1 65.035 56.95 73.12 3 4 1.5 0 3 59.335 42.30 76.37
features_new. columns = cols
features_new
tenure SeniorCitizen_tenure_mean SeniorCitizen_tenure_min SeniorCitizen_tenure_max MonthlyCharges_tenure_mean MonthlyCharges_tenure_min MonthlyCharges_tenure_max 0 1 0.0 0 0 29.850 29.85 29.85 1 2 1.0 0 2 62.275 53.85 70.70 2 3 0.5 0 1 65.035 56.95 73.12 3 4 1.5 0 3 59.335 42.30 76.37
data_new = pd. merge( data, features_new, how= 'left' , on= 'tenure' )
data_new
tenure SeniorCitizen MonthlyCharges gender SeniorCitizen_tenure_mean SeniorCitizen_tenure_min SeniorCitizen_tenure_max MonthlyCharges_tenure_mean MonthlyCharges_tenure_min MonthlyCharges_tenure_max 0 1 0 29.85 F 0.0 0 0 29.850 29.85 29.85 1 3 0 56.95 M 0.5 0 1 65.035 56.95 73.12 2 2 2 53.85 M 1.0 0 2 62.275 53.85 70.70 3 4 0 42.30 F 1.5 0 3 59.335 42.30 76.37 4 2 0 70.70 M 1.0 0 2 62.275 53.85 70.70 5 3 1 73.12 M 0.5 0 1 65.035 56.95 73.12 6 4 3 76.37 F 1.5 0 3 59.335 42.30 76.37
4.2.4 常用统计量整理
'''
- mean/var: 均值、方差
- max/min: 最大值、最小值
- skew: 数据分布偏度,小于0时左偏,大于0时右偏
- median: 中位数
- count: 个数
- nunique: 类别数
- quantile: 分位数(不能.agg(),需要自定义函数完成计算)
'''
a = np. array( [ [ 1 , 2 , 3 , 2 , 4 , 1 ]
, [ 0 , 0 , 0 , 1 , 1 , 1 ] ] )
df = pd. DataFrame( a. T, columns= [ 'x1' , 'x2' ] )
df
aggs = { 'x1' : [ 'mean' , 'var' , 'max' , 'min' , 'skew' , 'median' , 'count' , 'nunique' ] }
df. groupby( 'x2' ) . agg( aggs) . reset_index( )
x2 x1 mean var max min skew median count nunique 0 0 2.000000 1.000000 3 1 0.00000 2.0 3 3 1 1 2.333333 2.333333 4 1 0.93522 2.0 3 3
4.2.5 分位数统计量衍生
def q1 ( x) :
'''
下四分位数
'''
return x. quantile( 0.25 )
def q2 ( x) :
'''
上四分位数
'''
return x. quantile( 0.75 )
'''
aggs = {'x2': ['q1', 'q2']}
第一次用于创建列名称
aggs = {'x2': [q1, q2]}
第二次用于作为函数名带入.agg()
'''
colNames = [ 'x2' ]
keyCol = 'x1'
aggs = { }
for col in colNames:
aggs[ col] = [ 'q1' , 'q2' ]
aggs
'''
{'x2': ['q1', 'q2']}
'''
cols = [ keyCol]
for key in aggs. keys( ) :
cols. extend( [ key+ '_' + keyCol+ '_' + stat for stat in aggs[ key] ] )
cols
'''
['x1', 'x2_x1_q1', 'x2_x1_q2']
'''
aggs = { }
for col in colNames:
aggs[ col] = [ q1, q2]
aggs
'''
{'x2': [<function __main__.q1(x)>, <function __main__.q2(x)>]}
'''
d2 = df. groupby( keyCol) . agg( aggs) . reset_index( )
d2
x1 x2 q1 q2 0 1 0.25 0.75 1 2 0.25 0.75 2 3 0.00 0.00 3 4 1.00 1.00
d2. columns = cols
d2
x1 x2_x1_q1 x2_x1_q2 0 1 0.25 0.75 1 2 0.25 0.75 2 3 0.00 0.00 3 4 1.00 1.00
4.3 分组统计函数封装
def Binary_Group_Statistics ( keyCol
, features
, col_num= None
, col_cat= None
, num_stat= [ 'mean' , 'var' , 'max' , 'min' , 'skew' , 'median' ]
, cat_stat= [ 'mean' , 'var' , 'max' , 'min' , 'median' , 'count' , 'nunique' ]
, quant= True ) :
"""
双变量分组统计特征衍生函数
:param keyCol: 分组参考的关键变量
:param features: 原始数据集
:param col_num: 参与衍生的连续型变量
:param col_cat: 参与衍生的离散型变量
:param num_stat: 连续变量分组统计量
:param cat_num: 离散变量分组统计量
:param quant: 是否计算分位数
:return: 交叉衍生后的新特征和新特征名称
"""
if col_num != None :
aggs_num = { }
colNames = col_num
for col in col_num:
aggs_num[ col] = num_stat
cols_num = [ keyCol]
for key in aggs_num. keys( ) :
cols_num. extend( [ key+ '_' + keyCol+ '_' + stat for stat in aggs_num[ key] ] )
features_num_new = features[ col_num+ [ keyCol] ] . groupby( keyCol) . agg( aggs_num) . reset_index( )
features_num_new. columns = cols_num
if col_cat != None :
aggs_cat = { }
colName = col_num + col_cat
for col in col_cat:
aggs_cat[ col] = cat_stat
cols_cat = [ keyCol]
for key in aggs_cat. keys( ) :
cols_cat. extend( [ key+ '_' + keyCol+ '_' + stat for stat in aggs_cat[ key] ] )
features_cat_new = features[ col_cat+ [ keyCol] ] . groupby( keyCol) . agg( aggs_cat) . reset_index( )
features_cat_new. columns = cols_cat
df_temp = pd. merge( features_num_new, features_cat_new, how= 'left' , on= keyCol)
features_new = pd. merge( features[ keyCol] , df_temp, how= 'left' , on= keyCol)
features_new. loc[ : , ~ features_new. columns. duplicated( ) ]
colNames_new = cols_num + cols_cat
colNames_new. remove( keyCol)
colNames_new. remove( keyCol)
else :
features_new = pd. merge( features[ keyCol] , features_num_new, how= 'left' , on= keyCol)
features_new. loc[ : , ~ features_new. columns. duplicated( ) ]
colNames_new = cols_num
colNames_new. remove( keyCol)
else :
if col_cat != None :
aggs_cat = { }
colNames = col_cat
for col in col_cat:
aggs_cat[ col] = cat_stat
cols_cat = [ keyCol]
for key in aggs_cat. keys( ) :
cols_cat. extend( [ key+ '_' + keyCol+ '_' + stat for stat in aggs_cat[ key] ] )
features_cat_new = features[ col_cat+ [ keyCol] ] . groupby( keyCol) . agg( aggs_cat) . reset_index( )
features_cat_new. columns = cols_cat
features_new = pd. merge( features[ keyCol] , features_cat_new, how= 'left' , on= keyCol)
features_new. loc[ : , ~ features_new. columns. duplicated( ) ]
colNames_new = cols_cat
colNames_new. remove( keyCol)
if quant:
def q1 ( x) :
return x. quantile( 0.25 )
def q2 ( x) :
return x. quantile( 0.75 )
aggs = { }
for col in colNames:
aggs[ col] = [ 'q1' , 'q2' ]
cols = [ keyCol]
for key in aggs. keys( ) :
cols. extend( [ key+ '_' + keyCol+ '_' + stat for stat in aggs[ key] ] )
aggs = { }
for col in colNames:
aggs[ col] = [ q1, q2]
features_temp = features[ colNames+ [ keyCol] ] . groupby( keyCol) . agg( aggs) . reset_index( )
features_temp. columns = cols
features_new = pd. merge( features_new, features_temp, how= 'left' , on= keyCol)
features_new. loc[ : , ~ features_new. columns. duplicated( ) ]
colNames_new = colNames_new + cols
colNames_new. remove( keyCol)
features_new. drop( [ keyCol] , axis= 1 , inplace= True )
return features_new, colNames_new
features = pd. DataFrame( { 'tenure' : [ 1 , 3 , 2 , 4 , 2 , 3 , 4 ]
, 'SeniorCitizen' : [ 0 , 0 , 2 , 0 , 0 , 1 , 3 ]
, 'MonthlyCharges' : [ 29.85 , 56.95 , 53.85 , 42.30 , 70.70 , 73.12 , 76.37 ]
, 'gender' : [ 'F' , 'M' , 'M' , 'F' , 'M' , 'M' , 'F' ] } )
col_num = [ 'MonthlyCharges' ]
col_cat = [ 'SeniorCitizen' ]
keyCol = 'tenure'
df, col = Binary_Group_Statistics( keyCol, features, col_num, col_cat)
df
MonthlyCharges_tenure_mean MonthlyCharges_tenure_var MonthlyCharges_tenure_max MonthlyCharges_tenure_min MonthlyCharges_tenure_skew MonthlyCharges_tenure_median SeniorCitizen_tenure_mean SeniorCitizen_tenure_var SeniorCitizen_tenure_max SeniorCitizen_tenure_min SeniorCitizen_tenure_median SeniorCitizen_tenure_count SeniorCitizen_tenure_nunique MonthlyCharges_tenure_q1 MonthlyCharges_tenure_q2 0 29.850 NaN 29.85 29.85 NaN 29.850 0.0 NaN 0 0 0.0 1 1 29.8500 29.8500 1 65.035 130.73445 73.12 56.95 NaN 65.035 0.5 0.5 1 0 0.5 2 2 60.9925 69.0775 2 62.275 141.96125 70.70 53.85 NaN 62.275 1.0 2.0 2 0 1.0 2 2 58.0625 66.4875 3 59.335 580.38245 76.37 42.30 NaN 59.335 1.5 4.5 3 0 1.5 2 2 50.8175 67.8525 4 62.275 141.96125 70.70 53.85 NaN 62.275 1.0 2.0 2 0 1.0 2 2 58.0625 66.4875 5 65.035 130.73445 73.12 56.95 NaN 65.035 0.5 0.5 1 0 0.5 2 2 60.9925 69.0775 6 59.335 580.38245 76.37 42.30 NaN 59.335 1.5 4.5 3 0 1.5 2 2 50.8175 67.8525
col
'''
['MonthlyCharges_tenure_mean',
'MonthlyCharges_tenure_var',
'MonthlyCharges_tenure_max',
'MonthlyCharges_tenure_min',
'MonthlyCharges_tenure_skew',
'MonthlyCharges_tenure_median',
'SeniorCitizen_tenure_mean',
'SeniorCitizen_tenure_var',
'SeniorCitizen_tenure_max',
'SeniorCitizen_tenure_min',
'SeniorCitizen_tenure_median',
'SeniorCitizen_tenure_count',
'SeniorCitizen_tenure_nunique',
'MonthlyCharges_tenure_q1',
'MonthlyCharges_tenure_q2']
'''
5. 时序字段特征衍生
本质是增加分组 发现规律,如某一季度用户流失 根据自然周期和业务周期衍生,如季节和旅游旺淡季 根据关键时间点的时间差值衍生
5.1 时间格式转换 pd.to_datetime()
t = pd. DataFrame( )
t[ 'time' ] = [ '2022-01-03;02:31:52'
, '2022-07-01;14:22:01'
, '2022-08-22;08:02:31'
, '2022-04-30;11:41:31'
, '2022-05-02;22:01:27' ]
'''
或传入字典
t = pd.DataFrame({'time': ['2022-01-03;02:31:52'
,'2022-07-01;14:22:01'
,'2022-08-22;08:02:31'
,'2022-04-30;11:41:31'
,'2022-05-02;22:01:27']})
'''
t[ 'time' ] = pd. to_datetime( t[ 'time' ] )
t[ 'time' ] , t[ 'time' ] [ 0 ]
'''
(0 2022-01-03 02:31:52
1 2022-07-01 14:22:01
2 2022-08-22 08:02:31
3 2022-04-30 11:41:31
4 2022-05-02 22:01:27
Name: time, dtype: datetime64[ns],
Timestamp('2022-01-03 02:31:52'))
'''
t[ 'time' ] . values, t[ 'time' ] . values. astype( 'datetime64[D]' )
'''
(array(['2022-01-03T02:31:52.000000000', '2022-07-01T14:22:01.000000000',
'2022-08-22T08:02:31.000000000', '2022-04-30T11:41:31.000000000',
'2022-05-02T22:01:27.000000000'], dtype='datetime64[ns]'),
array(['2022-01-03', '2022-07-01', '2022-08-22', '2022-04-30',
'2022-05-02'], dtype='datetime64[D]'))
'''
t[ 'time-D' ] = t[ 'time' ] . values. astype( 'datetime64[D]' )
t
time time-D 0 2022-01-03 02:31:52 2022-01-03 1 2022-07-01 14:22:01 2022-07-01 2 2022-08-22 08:02:31 2022-08-22 3 2022-04-30 11:41:31 2022-04-30 4 2022-05-02 22:01:27 2022-05-02
5.2 时序字段信息提取 dt.year/month/day…
'''
- 年月日时分秒
dt.year dt.month dt.day dt.hour dt.minute dt.second
- 提取季度、一年第几周、星期几
dt.quarter dt.weekofyear dt.dayofweek/dt.weekday
'''
t[ 'time' ] . dt. day
'''
0 3
1 1
2 22
3 30
4 2
Name: time, dtype: int64
'''
t[ 'year' ] = t[ 'time' ] . dt. year
t[ 'month' ] = t[ 'time' ] . dt. month
t[ 'day' ] = t[ 'time' ] . dt. day
t[ 'hour' ] = t[ 'time' ] . dt. hour
t[ 'minute' ] = t[ 'time' ] . dt. minute
t[ 'second' ] = t[ 'time' ] . dt. second
t[ 'dayofweek' ] = t[ 'time' ] . dt. dayofweek + 1
t[ 'quarter' ] = t[ 'time' ] . dt. quarter
t[ 'weekofyear' ] = t[ 'time' ] . dt. weekofyear
t
time time-D year month day hour minute second dayofweek quarter weekofyear 0 2022-01-03 02:31:52 2022-01-03 2022 1 3 2 31 52 1 1 1 1 2022-07-01 14:22:01 2022-07-01 2022 7 1 14 22 1 5 3 26 2 2022-08-22 08:02:31 2022-08-22 2022 8 22 8 2 31 1 3 34 3 2022-04-30 11:41:31 2022-04-30 2022 4 30 11 41 31 6 2 17 4 2022-05-02 22:01:27 2022-05-02 2022 5 2 22 1 27 1 2 18
t[ 'weekend' ] = ( t[ 'dayofweek' ] > 5 ) . astype( int )
t[ 'hour_section' ] = ( t[ 'hour' ] // 6 ) . astype( int )
t
time time-D year month day hour minute second dayofweek quarter weekofyear weekend hour_section 0 2022-01-03 02:31:52 2022-01-03 2022 1 3 2 31 52 1 1 1 0 0 1 2022-07-01 14:22:01 2022-07-01 2022 7 1 14 22 1 5 3 26 0 2 2 2022-08-22 08:02:31 2022-08-22 2022 8 22 8 2 31 1 3 34 0 1 3 2022-04-30 11:41:31 2022-04-30 2022 4 30 11 41 31 6 2 17 1 1 4 2022-05-02 22:01:27 2022-05-02 2022 5 2 22 1 27 1 2 18 0 3
5.3 时间差值衍生
5.3.1 列与列的运算
t[ 'time' ] - t[ 'time' ]
'''
0 0 days
1 0 days
2 0 days
3 0 days
4 0 days
Name: time, dtype: timedelta64[ns]
'''
5.3.2 列与时间戳的运算
p1 = '2022-01-03;02:31:52'
pd. Timestamp( p1) , pd. Timestamp( p1) . year
'''
(Timestamp('2022-01-03 02:31:52'), 2022)
'''
t[ 'time_diff' ] = t[ 'time' ] - pd. Timestamp( p1)
t[ 'time_diff' ]
'''
0 0 days 00:00:00
1 179 days 11:50:09
2 231 days 05:30:39
3 117 days 09:09:39
4 119 days 19:29:35
Name: time_diff, dtype: timedelta64[ns]
'''
'''
- timedelta类型提取信息
月 天 秒
dt.days/30 dt.days dt.seconds
得到的相差的seconds是忽略天数计算的结果,
相差的days是忽略时分秒计算的结果
'''
t[ 'time_diff' ] . dt. seconds
'''
0 0
1 42609
2 19839
3 32979
4 70175
Name: time_diff, dtype: int64
'''
t[ 'time_diff_s' ] = t[ 'time_diff' ] . values. astype( 'timedelta64[s]' ) . astype( 'int' )
t[ 'time_diff_s' ]
'''
0 0
1 15508209
2 19978239
3 10141779
4 10351775
Name: time_diff_s, dtype: int64
'''
t
time time-D year month day hour minute second dayofweek quarter weekofyear weekend hour_section time_diff time_diff_s 0 2022-01-03 02:31:52 2022-01-03 2022 1 3 2 31 52 1 1 1 0 0 0 days 00:00:00 0 1 2022-07-01 14:22:01 2022-07-01 2022 7 1 14 22 1 5 3 26 0 2 179 days 11:50:09 15508209 2 2022-08-22 08:02:31 2022-08-22 2022 8 22 8 2 31 1 3 34 0 1 231 days 05:30:39 19978239 3 2022-04-30 11:41:31 2022-04-30 2022 4 30 11 41 31 6 2 17 1 1 117 days 09:09:39 10141779 4 2022-05-02 22:01:27 2022-05-02 2022 5 2 22 1 27 1 2 18 0 3 119 days 19:29:35 10351775
5.3.2 选择时间戳
t[ 'time' ] . max ( ) , t[ 'time' ] . min ( )
'''
(Timestamp('2022-08-22 08:02:31'), Timestamp('2022-01-03 02:31:52'))
'''
import datetime
print ( datetime. datetime. now( ) )
print ( '-' )
print ( datetime. datetime. now( ) . strftime( '%Y-%m-%d' ) )
print ( pd. Timestamp( datetime. datetime. now( ) . strftime( '%Y-%m-%d' ) ) )
print ( '-' )
print ( datetime. datetime. now( ) . strftime( '%Y-%m-%d %H:%M:%S' ) )
print ( pd. Timestamp( datetime. datetime. now( ) . strftime( '%Y-%m-%d %H:%M:%S' ) ) )
'''
2023-02-21 10:35:29.455932
-
2023-02-21
2023-02-21 00:00:00
-
2023-02-21 10:35:29
2023-02-21 10:35:29
'''