Exponentially Weighted Averages

Exponentially Weighted Averages

$v _{t} = \beta v _{t - 1} + \left (1 - \beta \right ) \theta _{t}$
$= \beta \left [ \beta v _{t - 2} + \left (1 - \beta \right ) \theta _{t - 1} \right ] + \left (1 - \beta \right ) \theta _{t}$
$= \beta ^2 v _{t - 2} + \beta \left (1 - \beta \right ) \theta _{t - 1} + \left (1 - \beta \right ) \theta _{t}$
$= \beta ^{n} v _{t - n} + \left (1 - \beta \right ) \sum \limits_{i = 0} ^{n - 1} \beta ^{i} \theta _{t - i}$
令 $n = \left \lfloor \dfrac {1} {1 - \beta} \right \rfloor,$ 则
$\lim \limits_{ \beta \to 1} \beta ^{n} = \lim \limits_{ \beta \to 1} \beta ^{\left \lfloor \dfrac {1} {1 - \beta} \right \rfloor} = \lim \limits_{ \beta \to 1} \beta ^{ \dfrac {1} {1 - \beta} } = \dfrac {1} {e}$
因此 $v _{t} \approx \dfrac {1} {e} v _{t - n} + \dfrac {1} {n} \sum \limits_{i = 0} ^{n - 1} \beta ^{i} \theta _{t - i}$
$\approx \dfrac {1} {n} \sum \limits_{i = 0} ^{n - 1} \theta _{t - i}$

Bias Correction

$v _{t} = \begin{cases} 0, & t = 0 \\ \dfrac {1} {1 - \beta ^ t} \left [ \beta v _{t - 1} + \left (1 - \beta \right ) \theta _{t} \right ], & \text{otherwise} \end{cases}$

性质

$v _{1} = \theta _{1}$
$\lim \limits_{ t \to + \infty} \left (1 - \beta ^ t \right ) = 1$