阿正的梦工坊

第一周的字幕，quiz的答案以及作业的代码

第一周的笔记，字幕，quiz的答案，lab的代码

Supervised Machine Learning: Regression and Classification第一周的课程笔记

This example demonstrates how to infer the covariance matrix of a Gaussian distribution in a Bayesian framework, highlighting the role of the Wishart distribution as a conjugate prior.

This article demonstrated the Bayesian inference process for the mean vector of a multivariate Gaussian distribution and highlighted the importance of conjugate priors.

The conjugate relationship between the Dirichlet distribution and multinomial distribution provides a simple and efficient Bayesian inference framework for modeling categorical probabilities.

In Bayesian statistics, the Beta distribution and Binomial distribution form one of the most classic conjugate prior-posterior pairs.

A conjugate distribution is a type of prior distribution that, when combined with a specific likelihood function, results in a posterior distribution that has the same form as the prior.

The Bayesian approach provides a flexible and systematic way to incorporate prior knowledge and beliefs, which can be incredibly useful when data is scarce or uncertain.

This article introduces the background, properties, and real-world applications of exponential family distributions, illustrated through examples such as univariate and multivariate Gaussian distributions, Dirichlet distribution, and Wishart distribution.

In exponential family distributions, due to the linear relationship between the log-likelihood function and the sufficient statistic, Maximum Likelihood Estimation (MLE) often has a closed-form solution.

Thanks to the closed-form expression of the kernel matrix, the gradient of the objective function in the dual form of SVM can be computed efficiently. This avoids the need for complicated numerical optimization procedures

Closed-form solutions play a vital role in simplifying and solving problems in mathematics, statistics, and machine learning. They provide precise results with minimal computational cost, making them indispensable for problems where they are applicable.

A sufficient statistic is a function of a dataset that captures all the information about a parameter of interest contained within the data.

In statistics, the Exponential Family of Distributions is a widely used and mathematically convenient class of distributions that includes many common ones, such as the normal, binomial, and Poisson distributions.

t分布最初由英国统计学家戈塞特（William Sealy Gosset）提出，目的是为了解决小样本情况下的统计推断问题。它的长尾特性允许分布对离群点更加鲁棒，成为了处理小样本数据、异常值和不确定性问题的重要工具。

A Detailed Guide to the Gaussian-Wishart Distribution: Background, Application, and Mathematical Formulation

Gaussian-Gamma Distribution: Bayesian Inference with Unknown Mean and Variance

模型的误差并非只有一个来源，而是可以分解为三部分：不可约误差（Irreducible Error）、偏差（Bias） 和 方差（Variance）。

有参模型通过有限的参数数量来定义模型，而非参模型通过数据本身或无固定数量的参数来描述问题。

协方差矩阵的值：决定了马氏距离的尺度和方向敏感性。分布形状的建模：通过调整协方差矩阵，可以控制分布的拉伸和旋转，以更精确地拟合数据。

主题模型是一种用于发现文档集合中潜在主题的概率生成模型。其中，LDA（Latent Dirichlet Allocation, 潜在狄利克雷分配）是最著名的主题模型之一。在 LDA 中，狄利克雷分布起到了核心作用，用于建模文档-主题分布和主题-单词分布。

狄利克雷分布是多项分布参数的先验分布，在贝叶斯统计和机器学习中扮演着重要角色

伽马分布是一个通用的分布，广泛用于描述随机变量的总和；卡方分布是伽马分布的特例，专注于正态变量平方和的情形；

伽马分布描述的是一维随机变量的分布，而威沙特分布则描述协方差矩阵的分布。它们的联系可以看作是一种从标量到矩阵的推广过程。

A Detailed Analysis of SGD, RMSProp, and Adam Optimizers, and Their Memory Consumption

Both aim to improve training efficiency by adapting learning rates for each parameter, but they do so in different ways.

Detailed Analysis of Adam and AdamW Optimizers and Their Memory Consumption with float32 and bfloat16 Precision

Understanding the Additional Memory Requirements of Adam Optimizer: Memory Consumption Breakdown for Model Parameters and Optimizer States

Curriculum learning aims to improve the learning efficiency of models by gradually increasing the difficulty of the training tasks.

在贝叶斯统计中，后验分布表示在观察到数据后，对参数的更新后的信念。

Both involve the probability of data given a set of parameters, but they are used in different contexts and have distinct meanings.

Bayesian statistics is a framework that uses Bayes' theorem to update our beliefs about parameters or models based on observed data.

KL散度（Kullback-Leibler Divergence）可衡量两个概率分布之间差异

By using a held-out set, we ensure that the evaluation results are reliable and not influenced by overfitting to the training or development data.

upsampling and downsampling refer to adjusting the number of samples in a dataset

信息论和机器学习

A dictionary is a set of basic elements, typically represented as a matrix, where each column is a basic element (or atom). These dictionary atoms are the fundamental components used to construct the original data.

第三周的字幕，quiz的答案以及作业的代码。完结撒花。

第二周的字幕，quiz的答案以及作业的代码