今天15:00|ICML-3，7位嘉宾的豪华阵容！

点击蓝字

关注我们

AI TIME欢迎每一位AI爱好者的加入！

9月14日 15:00~21:00

AI TIME特别邀请了多位PhD，带来ICML-3！

哔哩哔哩直播通道

扫码关注AITIME哔哩哔哩官方账号

观看直播

链接：https://live.bilibili.com/21813994

15:00-17:00

★ 嘉宾介绍 ★

闵明：

加州大学圣巴巴拉分校三年级PhD，在Tomoyuki Ichiba和Ruimeng Hu的指导下做Rough Path，Stochastic Analysis，Mean Field Game等方向研究。

报告题目：

SigDFP算法-具有共同噪声的平均场博弈

内容简介：

Existing deep learning methods for solving mean-field games (MFGs) with common noise fix the sampling common noise paths and then solve the corresponding MFGs. This leads to a nested loop structure with millions of simulations of common noise paths in order to produce accurate solutions, which results in prohibitive computational cost and limits the applications to a large extent. In this paper, based on the rough path theory, we propose a novel single-loop algorithm, named signatured deep fictitious play (Sig-DFP), by which we can work with the unfixed common noise setup to avoid the nested loop structure and reduce the computational complexity significantly. The proposed algorithm can accurately capture the effect of common uncertainty changes on mean-field equilibria without further training of neural networks, as previously needed in the existing machine learning algorithms. The efficiency is supported by three applications, including linear-quadratic MFGs, mean-field portfolio game, and mean-field game of optimal consumption and investment. Overall, we provide a new point of view from the rough path theory to solve MFGs with common noise with significantly improved efficiency and an extensive range of applications. In addition, we report the first deep learning work to deal with extended MFGs (a mean-field interaction via both the states and controls) with common noise.

朱兆伟：

加州大学圣克鲁兹分校博士二年级学生。研究兴趣集中在弱监督学习、联邦学习等领域相关的理论及应用，比如，训练标签带有人为标注噪声时如何设计损失函数去抵消噪声影响，如何处理联邦学习中低质量、有系统误差的本地训练集。目前在ICML,ICLR,ACM Sigmetrics,CVPR等会议，IEEE TWC，IEEE TPDS等期刊上发表多篇一作论文。

报告题目：

基于表征的噪声转移矩阵估计方法

内容简介：

当训练数据集有标签噪声时，我们通常用噪声转移矩阵来刻画一个训练数据点被错误标注的概率。准确地估计该矩阵对于带噪学习具有重要意义。传统的估计方法大多依赖于模型的预测值，从而找到一些模型能够以高置信度预测的样本，进而估计噪声转移矩阵。然而，寻找足够多的符合条件的样本是一件困难的事情。为了摆脱对模型预测的依赖，我们从表征（representation）的角度出发，提出了一种基于表征的噪声转移矩阵估计方法。该方法的基本思想是：具有相似表征的数据点应该属于同一类别，即相同真实标签。标签噪声的存在使得我们观测到的相似表征的噪声标签可能不一致，而标签的一致性蕴含了噪声转移矩阵的信息。我们证明：仅比较至多三个相似表征的标签一致性就可以得到噪声转移矩阵的唯一真实解。该方法为噪声转移矩阵的估计提供了一个全新的视角，并有潜力与自监督等表征学习方法相结合。

游凯超：

清华大学软件学院机器学习组在读博士，师从龙明盛副教授。他的研究兴趣包括迁移学习、机器学习、人工智能等方向。曾获清华大学特等奖学金、北京市优秀毕业生等荣誉称号。

报告题目：

LogME：如何快速准确地选择下游任务

适用的预训练模型

内容简介：

在深度学习时代，神经网络的参数量越来越大，从头开始训练(train from scratch)的成本也越来越大。虽然人们能够采用迁移学习的预训练-微调范式来有效降低训练成本，但是在面对预训练模型库中成百上千的预训练模型时，人们却还没有行之有效的办法来选择合适的预训练模型。如果想要准确地选择最好的预训练模型，决定一个预训练模型的迁移效果就需要将近50个小时！本次报告，我将介绍我们最近在预训练模型选择方面的研究成果——名为LogME的预训练模型选择指标。它能极大地加速预训练模型选择的过程，将衡量单个预训练模型的时间从50个小时减少到一分钟，带来三千倍加速，从而使得快速准确地选择下游任务适用的预训练模型成为可能。

王希梅：

清华大学软件学院博士生，师从王建民教授和龙明盛副教授。他的研究兴趣包括领域适应、迁移学习和深度学习。他有多篇论文发表在NeurIPS、ICML等机器学习顶级国际会议上，同时担任NeurIPS (Top 10% Reviewer)、ICML (Expert Reviewer)、ICLR（Outstanding Reviewer）、CVPR、ICCV和TPAMI、IJCV、TIP等国际顶级期刊会议审稿人。

报告题目：

Self-Tuning: 如何减少对标记数据的需求？

内容简介：

大规模标记数据集推动深度学习获得了广泛应用，然而，在现实场景中收集足量的标记数据往往耗时耗力。为了减少对标记数据的需求，半监督学习和迁移学习的研究者们从两个不同的视角给出了自己的思考：半监督学习(Semi-supervised Learning, SSL)侧重于同时探索标记数据和无标记数据，通过挖掘无标记数据的内在结构增强模型的泛化能力，而迁移学习(Transfer Learning, TL)旨在将预训练模型微调到目标数据中，也就是我们耳熟能详的预训练-微调范式。然而，迁移学习面临模型漂移的问题，而半监督学习往往存在确认偏差的困境。为了实现数据高效深度学习(data-efficient deep learning)，我们提出一种新方法Self-Tuning，将标记数据和无标记数据的探索与预训练模型的迁移融为一体。在多个标准数据集的实验表明，Self-Tuning远远优于半监督学习和迁移学习的同类方法。例如，在标签比例为15%的Stanford-Cars数据集上，Self-Tuning的测试精度比fine-tuning几乎提高了一倍。

19:30-21:00

陈云路：

阿姆斯特丹大学在读博士生，研究方向是3D深度学习。

报告题目：

用特征匹配理解神经隐式3D表征

内容简介：

Recently, neural implicit functions have achieved impressive results for encoding 3D shapes. Conditioning on low-dimensional latent codes generalises a single implicit function to learn shared representation space for a variety of shapes, with the advantage of smooth interpolation. While the benefits from the global latent space do not correspond to explicit points at local level, we propose to track the continuous point trajectory by matching implicit features with the latent code interpolating between shapes, from which we corroborate the hierarchical functionality of the deep implicit functions, where early layers map the latent code to fitting the coarse shape structure, and deeper layers further refine the shape details. Furthermore, the structured representation space of implicit functions enables to apply feature matching for shape deformation, with the benefits to handle topology and semantics inconsistency, such as from an armchair to a chair with no arms, without explicit flow functions or manual annotations.

刘泽春：

香港科技大学博士生，卡内基梅隆大学访问学者

报告题目：

理解二值化网络训练以及优化器

和训练策略选取

内容简介：

The best performing Binary Neural Networks (BNNs) are usually attained using Adam optimization and its multi-step training variants. However, to the best of our knowledge, few studies explore the fundamental reasons why Adam is superior to other optimizers like SGD for BNN optimization or provide analytical explanations that support specific training strategies. To address this, in this paper we first investigate the trajectories of gradients and weights in BNNs during the training process. We show the regularization effect of second-order momentum in Adam is crucial to revitalize the weights that are dead due to the activation saturation in BNNs. We find that Adam, through its adaptive learning rate strategy, is better equipped to handle the rugged loss surface of BNNs and reaches a better optimum with higher generalization ability. Furthermore, we inspect the intriguing role of the real-valued weights in binary networks, and reveal the effect of weight decay on the stability and sluggishness of BNN optimization. Through extensive experiments and analysis, we derive a simple training scheme, building on existing Adam-based optimization, which achieves 70.5% top-1 accuracy on the ImageNet dataset using the same architecture as the state-of-the-art ReActNet while achieving 1.1% higher accuracy. Code and models are available at https://github.com/liuzechun/AdamBNN.

谢桑：

斯坦福大学计算机科学博士生，导师是Percy Liang和Tengyu Ma，由NDSEG奖学金支持。研究兴趣是对未见过的输入和任务的泛化（对分布转移的鲁棒性），用未标记的数据和有限的监督学习（迁移学习，半监督学习，无监督学习），以及来自现实世界鲁棒性问题和自然语言处理的应用。于2017年在斯坦福大学获得计算机科学专业学士学位和硕士学位，与Stefano Ermon一起研究可持续发展的机器学习方法，特别是在使用卫星图像的贫困分布图方面。

报告题目：

通过冻结参数来改进预训练模型的微调，

以简化学习问题

内容简介：

We focus on prediction problems with structured outputs that are subject to output validity constraints, e.g. pseudocode-to-code translation where the code must compile. While labeled input-output pairs are expensive to obtain, "unlabeled" outputs, i.e. outputs without corresponding inputs, are freely available (e.g. code on GitHub) and provide information about output validity. Pre-training captures this structure by training a denoiser to denoise corrupted versions of unlabeled outputs. We first show that standard fine-tuning after pre-training destroys some of this structure. We then propose composed fine-tuning, which trains a predictor composed with the pre-trained denoiser. Importantly, the denoiser is fixed to preserve output structure. Like standard fine-tuning, the predictor is also initialized with the pre-trained denoiser. We prove for two-layer ReLU networks that composed fine-tuning significantly reduces the complexity of the predictor, thus improving generalization. Empirically, we show that composed fine-tuning improves over standard fine-tuning on two pseudocode-to-code translation datasets (3% and 6% relative). The improvement is magnified on out-of-distribution (OOD) examples (4% and 25% relative), suggesting that reducing predictor complexity improves OOD extrapolation.

# 今日视频推荐 #

直播结束后我们会邀请讲者在微信群中与大家答疑交流，请添加“AI TIME小助手（微信号：AITIME_HY）”，回复“icml”，将拉您进“AI TIME ICML 会议交流群”！

AI TIME微信小助手

主       办：AI TIME

合作媒体：学术头条、AI 数据派

合作伙伴：智谱·AI、中国工程院知领直播、学堂在线、学术头条、biendata、 Ever链动

AI TIME欢迎AI领域学者投稿，期待大家剖析学科历史发展和前沿技术。针对热门话题，我们将邀请专家一起论道。同时，我们也长期招募优质的撰稿人，顶级的平台需要顶级的你，

请将简历等信息发至yun.he@aminer.cn！

微信联系:AITIME_HY

AI TIME是清华大学计算机系一群关注人工智能发展，并有思想情怀的青年学者们创办的圈子,旨在发扬科学思辨精神，邀请各界人士对人工智能理论、算法、场景、应用的本质问题进行探索，加强思想碰撞，打造一个知识分享的聚集地。

更多资讯请扫码关注

我知道你“在看”哟～

点击 阅读原文 了解更多精彩