关于KAN（Kolmogorov-Arnold Networks）的小小调研

前言

KAN是MIT2024年5月提出，与传统MLP（多层感知机）并列的全新深度学习架构。截至2024.9.13已有150+次引用。笔者是该方向的入门新手，开此贴记录相关方向的论文调研笔记~（欢迎指正和补充）

KAN 1.0 原文链接
KAN 2.0 原文链接

相关研究

basis function改进类

（KAN使用B样条拟合每个node的一元非线性函数）

《Kolmogorov-arnold networks are radial basis function networks》

• replaces the B-spline bases calculations with Gaussian RBF（径向基函数）

• 开源代码：Code

与其他DL场景结合类

联邦学习 《F-KANs: Federated Kolmogorov-Arnold Networks》

• 每个client训练局部KAN，中心server集成模型
• 实验（代码link：Code）
  • 2 clients
  • dataset: Iris 数据集链接
  • summary: KAN model is not only accurate but also consistent in
    its predictions, capturing all true positives (300s vs 2s/ KAN100% accuracy)
  • The spline-based univariate features enable KAN to capture
    complex patterns quickly and accurately, resulting in stable
    and high performance with fewer training rounds.

时间序列1 《KAN for Time Series Analysis》

• KANs outperforms conventional MLP in a real-world satellite traffic forecasting task, providing more accurate results with considerably fewer number of parameters.

• aim to evaluate the practicality of KANs in realworld scenarios, analyzing their efficiency in terms of the
  number of trainable parameters and discussing how the additional degrees of freedom might affect forecasting performance.

• using realworld satellite traffic data.

• 单变量时间序列预测

• 两层KAN，输入层和输出层的节点数分别对应total amount of time steps.

• 基于卫星数据预测交通状况

• 实验

  • 对比KAN与MLP架构的performance(6 beam area)
  • 用一周数据预测一天
  • KAN shows a rapid adjustment <> MLP exhibits a lag
  • KAN matched the rapid volume <> MLP moderately over/under-predicted
  • the robustness of KAN despite the complexity and higher volume <>
  • 参数方面：This reduced complexity suggests that KANs can achieve higher or comparable forecasting accuracy with simpler and potentially faster models.（用于资源有限和快速部署的场景）
    

时间序列2 《Kolmogorov-Arnold Networks (KANs) for Time Series Analysis》

• T-KAN:detect conecpt drift within time series(univariate)
  • core：use sliding window (two historical time steps to predict the next time step in the example, different KAN structures & activation functions represent different concepts) to observe the variations in KAN and identify concept drift.
    
• MT-KAN: imrove predictive performance (multivariate time series)
  

时间序列3 KAN4TSF: Are KAN and KAN-based models Effective for Time Series Forecasting?

• 代码：code

【持续更新中····】