DS Wannabe之5-AM Project: DS 30day int prep day20

Q1. Do you have any idea about Event2Mind in NLP?

Yes, it is based on NLP research paper to understand the common-sense inference from sentences.

Event2Mind: Common-sense Inference on Events, Intents, and Reactions

The study of “Commonsense Reasoning” in NLP deals with teaching computers how to gain and employ common sense knowledge. NLP systems require common sense to adapt quickly and understand humans as we talk to each other in a natural environment.

This paper proposes a new task to teach systems commonsense reasoning: given an event described in a short “event phrase” (e.g. “PersonX drinks coffee in the morning”), the researchers teach a system to reason about the likely intents (“PersonX wants to stay awake”) and reactions (“PersonX feels alert”) of the event’s participants.

Understanding a narrative requires common-sense reasoning about the mental states of people in relation to events. For example, if “Robert is dragging his feet at work,” pragmatic implications about Robert’s intent are that “Robert wants to avoid doing things” (Above Fig). You can also infer that Robert’s emotional reaction might be feeling “bored” or “lazy.” Furthermore, while not explicitly mentioned, you can assume that people other than Robert are affected by the situation, and these people are likely to feel “impatient” or “frustrated.”

This type of pragmatic inference can likely be useful for a wide range of NLP applications that require accurate anticipation of people’s intents and emotional reactions, even when they are not expressly mentioned. For example, an ideal dialogue system should react in empathetic ways by reasoning about the human user’s mental state based on the events the user has experienced, without the user explicitly stating how they are feeling. Furthermore, advertisement systems on social media should be able to reason about the emotional reactions of people after events such as mass shootings and remove ads for guns, which might increase social distress. Also, the pragmatic inference is a necessary step toward automatic narrative understanding and generation. However, this type of commonsense social reasoning goes far beyond the widely studied entailment tasks and thus falls outside the scope of existing benchmarks.

Event2Mind是一个自然语言处理模型，它旨在从文本中推断事件所暗示的参与者的意图和反应。具体来说，它试图预测事件背后的人们的心理状态，如他们的欲望、情感或动机。例如，给定一个事件描述“某人给了一个惊喜派对”，Event2Mind模型可能会推断出“某人”想要“使朋友开心”，而朋友可能会感到“震惊”和“高兴”。Event2Mind侧重于推断事件的直接和间接参与者的心理状态，并为理解复杂的人类行为和关系提供深入见解。

Q2. SWAG in NLP

SWAG（Situations With Adversarial Generations）是一种用于评估常识推理能力的NLP数据集。它包含了大量的多项选择问题，这些问题旨在测试模型对日常情境的理解能力。每个问题都涉及一个给定的情境，并提供了四个可能的后续情况，其中只有一个是合理的。任务是选择最合适的选项。例如，一个情境可能是“一个人站在悬崖边上”，而可能的后续情况可能包括“他跳了下去”或“他退后一步”。正确的答案反映了对常识和物理世界的理解。

SWAG stands for Situations with Adversarial Generations is a dataset consisting of 113k multiple- choice questions about a rich spectrum of grounded situations.

Swag: A Large Scale Adversarial Dataset for Grounded Commonsense Inference

According to NLP research paper on SWAG is “Given a partial description like “he opened the hood of the car,” humans can reason about the situation and anticipate what might come next (“then, he examined the engine”). In this paper, you introduce the task of grounded commonsense inference, unifying natural language inference(NLI), and common-sense reasoning.

We present SWAG, a dataset with 113k multiple-choice questions about the rich spectrum of grounded positions. To address recurring challenges of annotation artifacts and human biases found in many existing datasets, we propose AF(Adversarial Filtering), a novel procedure that constructs a de-biased dataset by iteratively training an ensemble of stylistic classifiers, and using them to filter the data. To account for the aggressive adversarial filtering, we use state-of-the-art language models to oversample a diverse set of potential counterfactuals massively. Empirical results present that while humans can solve the resulting inference problems with high accuracy (88%), various competitive models make an effort on our task. We provide a comprehensive analysis that indicates significant opportunities for future research.

When we read a tale, we bring to it a large body of implied knowledge about the physical world. For instance, given the context “on stage, a man takes a seat at the piano,” we can easily infer what the situation might look like: a man is giving a piano performance, with a crowd watching him. We can furthermore infer him likely next action: he will most likely set his fingers on the piano key and start playing.

This type of natural language inference(NLI) requires common-sense reasoning, substantially broadening the scope of prior work that focused primarily on linguistic entailment. Whereas the

dominant entailment paradigm asks if 2 natural language sentences (the ‘premise’ and the ‘hypothesis’) describe the same set of possible worlds, here we focus on whether a (multiple-choice) ending represents a possible (future) world that can a from the situation described in the premise, even when it is not strictly entailed. Making such inference necessitates a rich understanding of everyday physical conditions, including object affordances and frame semantics.

Q3. What is the Pix2Pix network?

Pix2Pix是一个图像到图像转换的网络，基于条件生成对抗网络（Conditional Generative Adversarial Networks, cGANs）来完成任务。Pix2Pix网络能够接收一种类型的图像（如草图、地图或黑白照片）并转换成另一种类型的图像（如彩色照片、街景或彩色照片）。它在一系列图像转换任务中表现出色，如将白天的照片转换成夜景，将素描转换成照片，或者将卫星图像转换成地图。Pix2Pix的核心是一个条件对抗网络，其中生成器尝试创建真实的图像，而判别器尝试区分真实的图像和生成器创建的图像。通过这种方式，生成器被训练来产生越来越逼真的图像。

Pix2Pix network: It is a Conditional GANs (cGAN) that learn the mapping from an input image to output an image.

Image-To-Image Translation is the process for translating one representation of the image into another representation.

The image-to-image translation is another example of a task that GANs (Generative Adversarial Networks) are ideally suited for. These are tasks in which it is nearly impossible to hard-code a loss function. Studies on GANs are concerned with novel image synthesis, translating from a random vector z into an image. Image-to-Image translation converts one image to another like the edges of the bag below to the photo image. Another exciting example of this is shown below:

In Pix2Pix Dual Objective Function with an Adversarial and L1 Loss

A naive way to do Image-to-Image translation would be to discard the adversarial framework altogether. A source image would just be passed through a parametric function, and the difference in the resulting image and the ground truth output would be used to update the weights of the network. However, designing this loss function with standard distance measures such as L1 and L2 will fail to capture many of the essential distinctive characteristics between these images. However,  authors do find some value to the L1 loss function as a weighted sidekick to the adversarial loss function.

The Conditional-Adversarial Loss (Generator versus Discriminator) is very popularly formatted as follows:

Q4. Explain UNet Architecture?

U-Net architecture: It is built upon the Fully Convolutional Network and modified in a way that it yields better segmentation in medical imaging. Compared to FCN-8, the two main differences are (a) U-net is symmetric and (b) the skip connections between the downsampling path and upsampling path apply a concatenation operator instead of a sum. These skip connections intend to provide local information to the global information while upsampling. Because of its symmetry, the network has a large number of feature maps in the upsampling path, which allows transferring information. By comparison, the underlying FCN architecture only had the number of classes feature maps in its upsampling way.

UNet是一种流行的卷积神经网络（CNN）架构，最初是为了满足医学图像分割的需求而设计的。其名称来源于其U形结构。UNet由两部分组成：收缩路径（用于捕捉上下文）和对称的扩展路径（用于精确定位）。这种设计使得网络可以在图像分割任务中有效地工作，特别是在需要精确边缘定位时。

• 收缩路径：由卷积层和最大池化层交替组成，随着网络深度的增加，图像的空间尺寸减小，而特征数量增加。
• 扩展路径：由卷积层和上采样层交替组成，它将特征图逐步恢复到输入图像的尺寸，同时将特征图从收缩路径通过跳跃连接（skip connections）拼接，这有助于恢复图像的定位信息。

UNet因其在医学图像分割领域的成功应用而备受推崇，并且已被广泛应用于其他图像分割任务中。

The UNet architecture looks like a ‘U,’ which justifies its name. This UNet architecture consists of 3 sections: The contraction, the bottleneck, and the expansion section. The contraction section is made of many contraction blocks. Each block takes an input that applies two 3X3 convolution layers, followed by a 2X2 max pooling. The number of features or kernel maps after each block doubles so that UNet architecture can learn complex structures. Bottommost layer mediates between the contraction layer and the expansion layer. It uses two 3X3 CNN layers followed by 2X2 up convolution layer.

But the heart of this architecture lies in the expansion section. Similar to the contraction layer, it also has several expansion blocks. Each block passes input to two 3X3 CNN layers, followed by a 2X2 upsampling layer. After each block number of feature maps used by the convolutional layer, get half to maintain symmetry. However, every time input is also get appended by feature maps of the corresponding contraction layer. This action would ensure that features that are learned while contracting the image will be used to reconstruct it. The number of expansion blocks is as same as the number of contraction blocks. After that, the resultant mapping passes through another 3X3 CNN layer, with the number of feature maps equal to the number of segments desired.

Q5. What is pair2vec?

pair2vec是NLP中的一种模型，旨在捕获词对之间的复杂关系。与word2vec类似，它将单词对表示为向量，但不同之处在于pair2vec专注于学习词对之间的语义和语法关系的嵌入，例如（"Paris", "France"）和（"Berlin", "Germany"）之间的关系。

pair2vec的目标是提供一种方法，通过该方法可以使用向量算术来解决类比问题，如“Paris 对于 France 就像 Berlin 对于 Germany”。这种模型对于理解语言中的关系和属性非常有用，可以支持复杂的推理任务。

This paper pre trains word pair representations by maximizing pointwise mutual information of pairs of words with their context. This encourages a model to learn more meaningful representations of word pairs than with more general objectives, like modeling. The pre-trained representations are useful in tasks like SQuAD and MultiNLI that require cross-sentence inference. You can expect to see more pretraining tasks that capture properties particularly suited to specific downstream tasks and are complementary to more general-purpose tasks like language modeling.

Reasoning about implied relationships between pairs of words is crucial for cross sentences inference problems like question answering (QA) and natural language inference (NLI). In NLI, e.g., given a premise such as “golf is prohibitively expensive,” inferring that the hypothesis “golf is a cheap pastime” is a contradiction requires one to know that expensive and cheap are antonyms. Recent work has shown that current models, which rely heavily on unsupervised single-word embeddings, struggle to grasp such relationships. In this pair2vec paper, we show that they can be learned with word pair2vec(pair vector), which are trained, unsupervised, at a huge scale, and which significantly improve performance when added to existing cross-sentence attention mechanisms.

Unlike single word representations, which are typically trained by modeling the co-occurrence of a target word x with its context c, our word-pair representations are learned by modeling the three-way co-occurrence between two words (x,y) and the context c that ties them together, as illustrated in above Table. While similar training signal has been used to learn models for ontology construction and knowledge base completion, this paper shows, for the first time, that considerable scale learning of pairwise embeddings can be used to improve the performance of neural cross- sentence inference models directly.

Q6. What is Meta-Learning?

Meta-learning，有时被称为“学会学习”，是机器学习中的一种概念，它涉及设计模型，这些模型可以学习如何更有效地学习和适应新任务。Meta-learning的目的是使模型能够利用以前的经验快速适应新的、未见过的任务或数据分布。

在传统的机器学习中，模型通常针对一个特定的任务进行训练。与此相反，meta-learning方法训练模型在多个任务上学习，从而获得跨任务转移知识的能力。

• 模型无关的Meta-learning（MAML）：一种流行的meta-learning方法，旨在找到一个好的模型参数初始化，使模型能够通过少量梯度更新迅速适应新任务。

• 少样本学习（Few-shot learning）：meta-learning常用于这种情况，即当只有很少的数据可用于新任务时，模型仍然能够表现良好。

Meta-learning: It is an exciting area of research that tackles the problem of learning to learn. The goal is to design models that can learn new skills or fastly to adapt to new environments with minimum training examples. Not only does this dramatically speed up and improve the design of ML(Machine Learning) pipelines or neural architectures, but it also allows us to replace hand-engineered algorithms with novel approaches learned in a data-driven way.

The goal of meta-learning is to train the model on a variety of learning tasks, such that it can solve new learning tasks with only a small number of training samples. It tends to focus on finding model agnostic solutions, whereas multi-task learning remains deeply tied to model architecture.

Thus, meta-level AI algorithms make AI systems:

· Learn faster

· Generalizable to many tasks

· Adaptable to environmental changes like in Reinforcement Learning

One can solve any problem with a single model, but meta-learning should not be confused with one- shot learning.

Q7. What is ALiPy(Active Learning in Python)?

Supervised ML methods usually require a large set of labeled examples for model training. However, in many real applications, there are ample unlabeled data but limited labeled data; and acquisition of labels is costly. Active learning (AL) reduces labeling costs by iteratively selecting the most valuable data to query their labels from the annotator.

Active learning is the leading approach to learning with limited labeled data. It tries to reduce human efforts on data annotation by actively querying the most prominent examples.

ALiPy is a Python toolbox for active learning(AL), which is suitable for various users. On the one hand, the entire process of active learning has been well implemented. Users can efficiently perform experiments by many lines of codes to finish the entire process from data pre-processes to result in visualization. More than 20 commonly used active learning(AL) methods have been implemented in the toolbox, providing users many choices.

Q8.What is the Lingvo model?

Lingvo: It is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models. These models are composed of modular building blocks that are flexible and easily extensible, and experiment configurations are centralized and highly customizable. Distributed training and quantized inference are supported directly within a framework, and it contains existing implementations of an ample number of utilities, helper functions, and newest research ideas. This model has been used in collaboration by dozens of researchers in more than 20 papers over the last two years.

Lingvo是一个用于构建序列到序列模型的TensorFlow框架，专注于语言任务，如机器翻译、语音识别和图片字幕。它由Google开发，旨在提供一个模块化和可扩展的架构，以支持大规模和复杂的深度学习模型。Lingvo模型的设计注重于在多个GPU和TPU上进行分布式训练，以及高效地处理大量数据。

为什么Lingvo研究重要？

Lingvo的研究重要性在于其为研究人员和开发者提供了一个健壮、灵活且经过验证的框架来进行复杂的语言建模任务。该框架的设计使得研究和实验能够在统一的环境中进行，这有助于推动语言处理技术的进步，尤其是在处理资源密集型任务和大规模应用时。

Why does this Lingvo research matter?

The process of establishing a new deep learning(DL) system is quite complicated. It involves exploring an ample space of design choices involving training data, data processing logic, the size, and type of model components, the optimization procedures, and the path to deployment. This complexity requires the framework that quickly facilitates the production of new combinations and the modifications from existing documents and experiments and shares these new results. It is a workspace ready to be used by deep learning researchers or developers. Nguyen Says: “We have researchers working on state-of-the-art(SOTA) products and research algorithms, basing their research off of the same codebase. This ensures that code is battle-tested. Our collective experience is encoded in means of good defaults and primitives that we have found useful over these tasks.”

Q9. What is Dropout Neural Networks?

Dropout是一种正则化技术，用于防止神经网络中的过拟合。在训练过程中，Dropout会随机“丢弃”（即临时移除）网络中的一些节点（包括其连接），这意味着这些节点在前向传播和反向传播中暂时不会被更新。这种随机移除的效果相当于训练了多个网络并将它们的预测结果进行平均。

为什么我们需要Dropout？

我们需要Dropout来增加网络的泛化能力，减少模型对训练数据的过度拟合。Dropout通过减少神经元间复杂的共适应关系，迫使网络学习更加鲁棒的特征，这些特征不会过度依赖于任何单一的输入特征或神经元。这样，模型在新的、未见过的数据上表现得更好，泛化误差更小。

The term “dropout” refers to dropping out units (both hidden and visible) in a neural network.

At each training stage, individual nodes are either dropped out of the net with probability 1-p or kept with probability p, so that a reduced network is left; incoming and outgoing edges to a dropped-out node are also removed.

Why do we need Dropout?

The answer to these questions is “to prevent over-fitting.”

A fully connected layer occupies most of the parameters, and hence, neurons develop co-dependency amongst each other during training, which curbs the individual power of each neuron leading to over- fitting of training data.

Q10. What is GAN?

GAN是一种由两部分组成的生成模型：生成器（Generator）和判别器（Discriminator）。生成器的任务是生成尽可能接近真实数据的新数据，而判别器的任务是区分生成的数据和真实数据。这两个模型以对抗的方式同时训练：生成器试图欺骗判别器，判别器试图不被欺骗。

GAN的例子

一个经典的GAN应用是在图像生成领域。例如，GAN可以从一组真实的人脸图片中学习，并生成全新的、看起来真实的人脸图片。这些生成的人脸从未在训练数据中出现过，但它们看起来与真实人脸难以区分。其他示例包括风格迁移（将一种艺术风格应用到图像上）、超分辨率（从低分辨率图像生成高分辨率图像）和文本到图像的生成（根据文本描述生成相应的图像）。

A generative adversarial network (GAN): It is a class of machine learning systems invented by Ian Goodfellow and his colleagues in 2014. Two neural networks are contesting with each other in a game (in the idea of game theory, often but not always in the form of a zero-sum game). Given a training set, this technique learns to generate new data with the same statistics as the training set. E.g., a GAN trained on photographs can produce original pictures that look at least superficially authentic to human observers, having many realistic characteristics. Though initially proposed as a form of a generative model for unsupervised learning, GANs have also proven useful for semi- supervised learning,[2] fully supervised learning, and reinforcement learning.

Example of GAN

• Given an image of a face, the network can construct an image that represents how that person could look when they are old.

Generative Adversarial Networks takes up a game-theoretic approach, unlike a conventional neural network. The network learns to generate from a training distribution through a 2-player game. The two entities are Generator and Discriminator. These two adversaries are in constant battle throughout the training process.