Pyspark回归--AFTSurvivalRegression

AFTSurvivalRegression

class pyspark.ml.regression.AFTSurvivalRegression(featuresCol=‘features’, labelCol=‘label’, predictionCol=‘prediction’, fitIntercept=True, maxIter=100, tol=1e-06, censorCol=‘censor’, quantileProbabilities=[0.01, 0.05, 0.1, 0.25, 0.5, 0.75, 0.9, 0.95, 0.99], quantilesCol=None, aggregationDepth=2)

加速失效时间 (AFT) 模型生存回归 根据生存时间的 Weibull 分布拟合参数 AFT 生存回归模型。

aggregationDepth = Param(parent=‘undefined’, name=‘aggregationDepth’, doc=‘树聚合 (>= 2) 的建议深度。’

censorCol = Param(parent=‘undefined’, name=‘censorCol’, doc=‘censor 列名。该列的值可以是0或1。如果值为1，则表示事件已经发生，即未经审查；否则审查。’）

fitIntercept = Param(parent=‘undefined’, name=‘fitIntercept’, doc=‘是否适合截取项。’)

quantileProbabilities = Param(parent=‘undefined’, name=‘quantileProbabilities’, doc=‘quantile probabilities array。分位数概率数组的值应该在 (0, 1) 范围内，并且数组应该是非空的。’)

quantilesCol = Param(parent=‘undefined’, name=‘quantilesCol’, doc=‘quantiles 列名。如果设置了，该列将输出对应的 quantileProbabilities 的分位数。’)

tol = Param(parent=‘undefined’, name=‘tol’, doc=‘迭代算法的收敛容差 (>= 0).’)

model.coefficients：模型系数

model.predictQuantiles(features)：预测分位数

model.scale：模型比例参数

01.创建数据集

from pyspark.sql import SparkSession
spark = SparkSession.builder.config("spark.driver.host","192.168.1.10")\
    .config("spark.ui.showConsoleProgress","false")\
    .appName("AFTSurvivalRegression").master("local[*]").getOrCreate()
from pyspark.ml.linalg import Vectors
df = spark.createDataFrame([
    (1.0, Vectors.dense(1.0), 1.0),
    (1e-40, Vectors.sparse(1, [], []), 0.0)], ["label", "features", "censor"])
df.show()
df.printSchema()

​ 输出结果：

+-------+---------+------+
|  label| features|censor|
+-------+---------+------+
|    1.0|    [1.0]|   1.0|
|1.0E-40|(1,[],[])|   0.0|
+-------+---------+------+

root
 |-- label: double (nullable = true)
 |-- features: vector (nullable = true)
 |-- censor: double (nullable = true)

02.构建模型转换原始数据并进行查看

from pyspark.ml.regression import AFTSurvivalRegression
aftsr = AFTSurvivalRegression()
model = aftsr.fit(df)
model.transform(df).show()

​ 输出结果：

+-------+---------+------+----------+
|  label| features|censor|prediction|
+-------+---------+------+----------+
|    1.0|    [1.0]|   1.0|       1.0|
|1.0E-40|(1,[],[])|   0.0|       1.0|
+-------+---------+------+----------+

03.使用模型对数据进行预测

print(model.predict(Vectors.dense(6.3)))

​ 输出结果：

1.0

04.查看预测分位数

print(model.predictQuantiles(Vectors.dense(6.3)))

​ 输出结果：

[0.010050335853501452,
0.05129329438755058,
0.1053605156578263,
0.2876820724517809,
0.6931471805599453,
1.3862943611198906,
2.302585092994046,
2.99573227355399,
4.605170185988092]