TransmogrifAI架构的AotoML方法中获取最佳模型信息和最优设置参数和模型特征等日志的方法Scala

TransmogrifAI的官方文档没有这方面的注释,导致我在实现时用了一些功夫,在读了一部分TransmogrifAI的源码后,我发现了一些获取这些日志的方法。
可以参考TransmogrifAI源码的获取方法,详细请查看modelInsights.selectedModelInfo方法源码
直接放代码:

获取模型最佳设置参数

	val selectedModelInfo: Option[ModelSelectorSummary] = modelInsights.selectedModelInfo
    val excludedParams = Set(
      SparkWrapperParams.SparkStageParamName,
      ModelSelectorNames.outputParamName, ModelSelectorNames.inputParam1Name,
      ModelSelectorNames.inputParam2Name, ModelSelectorNames.outputParamName,
      OpPipelineStageParamsNames.InputFeatures, OpPipelineStageParamsNames.InputSchema,
      OpPipelineStageParamsNames.OutputMetadata,
      "labelCol", "predictionCol", "predictionValueCol", "rawPredictionCol", "probabilityCol"
    )
    val name = selectedModelInfo.map(sm => s"Selected Model - ${sm.bestModelType}").getOrElse("")
    val validationResults = (for {
      sm <- selectedModelInfo.toSeq
      e <- sm.validationResults.filter(v =>
        v.modelUID == sm.bestModelUID && v.modelName == sm.bestModelName && v.modelType == sm.bestModelType
      )
    } yield {
      val params = e.modelParameters.filterKeys(!excludedParams.contains(_))
      Seq("name" -> e.modelName, "uid" -> e.modelUID, "modelType" -> e.modelType) ++ params
    }).flatten.sortBy(_._1)
    if (validationResults.nonEmpty) {
      val table = Table(name = name, columns = Seq("Model Param", "Value"), rows = validationResults)
      Seq(table.prettyString())
    } else Seq.empty
    validationResults

获取最佳模型信息

	val selectedModelInfo: Option[ModelSelectorSummary] = modelInsights.selectedModelInfo
    val opModelName: String = selectedModelInfo.map(sm => s"${sm.bestModelType}").getOrElse("")
    opModelName.slice(2, opModelName.size);

获取模型特征

    // 获取模型特征
    val modelFeatures: Seq[Insights] = modelInsights.features.flatMap(feature => feature.derivedFeatures)
    val featureContributions: Seq[(String, Double)] = modelFeatures.map(feature => (feature.derivedFeatureName,
      feature.contribution.map(contribution => math.abs(contribution))
        .foldLeft(0.0) { (max, contribution) => math.max(max, contribution) }))
    val sortedContributions: Seq[(String, Double)] = featureContributions.sortBy(contribution => -contribution._2)
    val topNum: Int = math.min(20, sortedContributions.size)
    //    println(s"Top $topNum 特征贡献:")
    //    sortedContributions.take(topNum).foreach(featureInfo => println(s"${featureInfo._1}: ${featureInfo._2}"))
    sortedContributions.take(topNum)
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值