2023-09-30-连续系统的LQR变分法推导

连续系统的LQR变分法推导

优化控制问题

考虑下述优化问题：

$$\begin{array}{rlr}J=h(x(t_f),t_f)+{\int }_{t_0}^{t_f}g(x(t),u(t),t)dt& & \\ \text{subject to}& \dot{x}(t)=a(x(t),u(t),t)& \\ & t_0,x(t_0)& \text{fixed}\\ & t_f& \text{free}\\ & x(t_f)& \text{free or fixed}\end{array}\text{\hspace{1em}(1)}$$

其中$\dot{x}(t)=a(x(t),u(t),t)$可以被看作一种约束，因此我们定义拉格朗日乘子$p(t)$，同时对原有的代价函数$J$进行增广，得到如下增广代价函数：

J a = h ( x ( t f ) , t f ) + ∫ t 0 t f [ g ( x ( t ) , u ( t ) , t ) + p ( t ) T { a ( x ( t ) , u ( t ) , t ) − x ˙ ( t ) } ] d t (2) J_a=h(x(t_f),t_f)+\int_{t_0}^{t_f}[g(x(t),u(t),t)+p(t)^T\{a(x(t),u(t),t)-\dot{x}(t)\}]dt \tag{2} Ja​=h(x(tf​),tf​)+∫t0​tf​​[g(x(t),u(t),t)+p(t)T{ a(x(t),u(t),t)−x˙(t)}]dt(2)

对$J_a$取变分（Variation），得到如下表达式：

$$\delta J_a=h_{x_f}\delta x_f+h_{t_f}\delta t_f+{\int }_{t_0}^{t_f}[g_x\delta x+g_u\delta u+(a-\dot{x}{)}^T\delta p(t)+p^T(t)a_x\delta x+a_u\delta u-\delta \dot{x}]dt+[g+p^T(a-\dot{x})](t_f)\delta t_f\text{\hspace{1em}(3)}$$

其中：

$$\begin{array}{rl}{x}_f& =x(t_f)\\ {\dot{x}}_f& =\dot{x}(t_f)\\ u_f& =u(t_f)\\ p_f& =p(t_f)\\ h_{x_f}& =\frac{\partial h(x,t)}{\partial x}(x_f,t_f)\\ h_{tf}& =\frac{\partial h(x,t)}{\partial t}(x_f,t_f)\\ g_x& =\frac{\partial g(x,u,t)}{\partial x}\\ g_u& =\frac{\partial g(x,u,t)}{\partial u}\\ [g+p^T(a-\dot{x})](t_f)& =g(x_f,u_f,t_f)+p_f^T(a(x_f,u_f,t_f)-{\dot{x}}_f)\end{array}\text{\hspace{1em}(4)}$$

下面我们定义哈密顿量（Hamiltonian）：

$$H(x,u,p,t)=g(x(t),u(t),t)+p^T(t)a(x(t),u(t),t)\text{\hspace{1em}(5)}$$

将定义的哈密顿量代入到公式（3）的变分中可以得到：

$$\begin{array}{rl}\delta J_a& =h_{x_f}\delta x_f+[h_{t_f}+g+p^T(a-\dot{x})](t_f)\delta t\\ & +{\int }_{t_0}^{t_f}[H_x\delta x+H_u\delta u+(a-\dot{x}{)}^T\delta p(t)\underset{(6.1)}{\underbrace{-p^T(t)\delta \dot{x}}}]dt\end{array}\text{\hspace{1em}(6)}$$

我们需要将变分表达式中的变分项进行合并，（6.1）中的$\delta \dot{x}$可以通过分部积分法（Integragint by Parts）将导数从变分项中移出，具体操作如下：

− ∫ t 0 t f p T ( t ) δ x ˙ d t = − ∫ t 0 t f p T ( t ) d δ x = − p T δ x ∣ t 0 t f + ∫ t 0 t f ( d p ( t ) d t ) T δ x d t = − p T ( t f ) δ x ( t f ) + ∫ t 0 t f p ˙ T δ x d t = − p T ( t f ) ( δ x f − x ˙ ( t f ) δ