Hypernet

Hypernet

Hypernet起源于2017年 iclr的一篇文章 hypernetworks

In this work, we consider an approach of using a small network (called a “hypernetwork") to generate the weights for a larger network (called a main network). The behavior of the main network is the same as with any usual neural network: it learns to map some raw inputs to their desired targets; whereas the hypernetwork takes a set of inputs that contain information about the structure of the weights and generates the weights for that layer.

Hypernet 在 Module Parameter Generation中起到了重要的作用

需要去阅读的文章：

1. https://aclanthology.org/2021.tacl-1.25/
2. https://proceedings.mlr.press/v162/he22f/he22f.pdf
3. https://aclanthology.org/2021.acl-long.47/
4. https://arxiv.org/abs/2205.12148