初识回归分析Regression Analysis

• Regression Analysis

  In statistical model, regression analysis is a set of statistical processes for estimating the relationship between a dependent variable (often called the “outcome variable”) and one or more independent variables (often called “predictors”, “covariates”, or “features”).

  理解difference between Regression Analysis and Machine Learning

  Regression analysis is primarily used for two conceptually distinct purpose:

  1. Regression analysis is widely used for prediction and forecasting, where its use has substantial overlap with the field of machine learning;
  2. In some situations regression analysis can be used to infer causal relationships between the independent and dependent variables;

• Regression History

  The earliest form of regression was the method of least squares, which was published by Legendre in 1805 and Gauss in 1809.

  The term “regression” was coined by Francis Galton in the nineteenth century to describe a biological phenomenon.

  Galton also created the statistical concept of correlation and widely promoted regression toward the mean.

  The phenomenon was that the heights of descendants of tall ancestors tend to regress down towards a normal average ( a phenomenon also knwon as regression toward the mean).

• Underlying Assumptions

  By itself, a regression is simply a calculation using the data.

  In order to interpret the output of a regression as a meaningful statistical quantity that measures real-world relationships, researchers often rely on a number of classical assumptions:

  1. The sample is representative of the population at large
  2. The independent variables are measured with no error
  3. Deviations from the model have an expected value of zero, conditional on covariates: $E(e_i|X_i)=0$;
  4. The variance of the residuals $e_i$ is constant across observations
  5. The residual $e_i$ are uncorrelated with one another.

• Regression model

  In practice, researchers first select a model they would like to estimate and then use their chosen method to estimate the parameters of that model. Regression models involve the following components:

  1. The unknown parameters, often denoted as a scalar or vector $\beta$;
  2. The independent variables, which are observed in data and are often denoted as a vecotr $X_i$(where $i$ denotes a row of data);
  3. The dependent variable, which are observed in data and often denoted using the scalar $Y_i$;
  4. The error terms, which are not directly observerd in data and are often denoted using the scalar $e_i$

• Regression in Finance

  Regression is a statistical method used in finance, investing, and other disciplines that attempts to determine the strength and character of the relationship between one dependent variable (usually denoted by Y) and a series of other variables (known as independent varibles).

• References

1. Investopedia: Regression Definition
2. 你应该掌握的 7 种回归模型！