机器学习高阶认识(一): 机器学习假设与迁移学习

目录

 

(1) 传统机器学习的主要假设之一

(2) 迁移学习的目的

应用

(1) 脑电图分类

(2) 事件抽取-transfer learning and shared semantic space

---

(1) 传统机器学习的主要假设之一

是用于训练分类器的训练数据和用于评估分类器的测试数据属于相同的特征空间，并且遵循相同的概率分布。但是，由于人的可变性，在许多应用中经常违反这一假设[55]。

reference: 

1. A. M. Azab, J. Toth, L. S. Mihaylova, and M. Arvaneh, “A review on transfer learning approaches in brain–computer interface,” inSignal Processing and Machine Learning for Brain-Machine Interfaces. Institution of Engineering and Technology, 2018, pp. 81–101.

例如，脑电图EEG(Electroencephalography)数据分布中的变化通常发生在从不同主题或跨主题内的会话和时间中获取数据时。而且，由于EEG信号是变化的而不是静态的(EEG has a high temporal resolution)，因此广泛的BCI会话表现出独特的一致性一致性分类问题[56]。

➡️while fMRI(functional magnetic resonance imaging)功能磁共振成像的一项基本挑战是将观察到的反应映射到皮层下结构。

reference: mapping of observed responses to subcortical structures is a fundamental challenge in contrast to other neuroimaging approaches such as functional magnetic resonance imaging (fMRI), cf. Glover, 2011)

(2) 迁移学习的目的

➡️因此，迁移学习的目的是通过利用在学习给定任务时获得的知识来解决不同但相关的任务来应对违反该假设的数据。换句话说，迁移学习考虑用基于在学习另一个任务的同时获得的信息来增强在一个任务（也扩展到一个会话或主题）上学习的分类器的性能的一组方法。迁移学习的进步可以放宽BCI的局限性，因为它不需要从开始就进行校准，迁移信息的噪音更少，并且无需依赖先前可用的数据来增加数据集的大小。

应用

(1) 脑电图分类

 当前大多数机器学习研究都集中在静态数据上，这不是对快速变化的大脑信号进行准确分类的最佳方法[34]

reference:

X. Zhang, L. Yao, X. Wang, J. Monaghan, and D. Mcalpine, “A survey on deep learning based brain computer interface: Recent advances and new frontiers,” arXiv preprint arXiv:1905.04149, 2019.

我们将介绍具有CNN架构的自发EEG应用程序，近期研究中GAN的利用，RNN的过程和应用程序，尤其是长短期记忆（LSTM）。我们还说明了从深度学习算法和转移学习方法扩展而来的深度转移学习，然后以对抗攻击为例，对用于系统测试的深度学习模型进行了举例说明.

(2) 事件抽取-transfer learning and shared semantic space

我们提出了一种可转移的神经体系结构，该体系结构利用现有的人为构造的事件模式和少量手动注释的可见类型集，并将现有类型的知识转移到未见类型的提取中，以提高事件提取的可扩展性，并节省人力。 在没有任何注释的情况下，我们的方法可以达到从大量标记数据中训练而来的最新监督模型可比的性能。

将来，我们将通过合并事件定义和参数描述来扩展我们的框架，以提高事件提取性能。

We propose a transferable neural architecture, which leverages existing human constructed event schemas and manual annotations for a small set of seen types, and transfers the knowledge from the existing types to the extraction of unseen types, to improve the scalability of event extraction as well as save human effort. Without any annotation, our approach can achieve comparable performance with state-of-the-art supervised models trained from a large amount of labeled data. In the future, we will extend our framework by incorporating event definitions and argument descriptions to improve the event extraction performance。

This transferable neural architecture jointly learns and maps the structural representations of both event metions and types into a shared semantic space by minimizing the distance between each event mention and its corresponding type. For event mentions with unseen types, their structures will be projected into the same semantic space using the same framework and assigned types with top-ranked similarity values.

• candidate trigger
• construct event mention structute <--based on--candidate arguments, AMR parsing ---->每个事件mention元组数据：2AMR concepts, 1 AMP relation

        event type structure, each type y <---- incorporating its predefined roles and take the type as the root ---->事件类型元组数据：1 type name, 1 argument role name

• we apply a weight-sharing CNN to each input structure to jointly learn event mention and type structual representations, which will be later used to learn the ranking function for zero-shot event extraction.

        1) input layer, dimensionality d x 2h* and d x 2p* respectively represent mention structure and type structure.

        2) convolution layer, input matrix of St is a feature map of dimensionality d x 2h*

        3) max-pooling, 所有的元组表示 all tuple representations Ci --max-pooling-->input sequence

        4) learning

        5) the linge loss, design a new loss function L1(t, y)

        6) By minimizing L1, we can learn the optimized model ----> can compose structure representations

                                                                                                      ----> map both event mention and types into a shared semantic space, where the positive type ranks the highest for each mention.

Reference:

Zero-Shot Transfer Learning for Event Extraction, Lifu Huang and Huai-zhong Ji and Kyunghyun Cho and Clare R. Voss, ACL, year= 2018