信息矩阵在slam的一些理解

Orange_qaq~

于 2023-07-14 15:16:53 发布

阅读量425

点赞数

文章标签：矩阵算法人工智能

本文链接：https://blog.csdn.net/weixin_48572752/article/details/131724366

版权

文章目录

前言
1 :book: 信息矩阵
2 :book: 信息矩阵与最小二乘的关联
- 2-1 :bookmark: 先谈一谈常规的最小二乘
- 2-2 :bookmark: 包含信息矩阵的最小二乘
3 :book: 信息矩阵如何更新的？
0 :book: 参考资料

前言

本文主要谈哪些问题？本文主要以信息矩阵为核心，发散畅谈~

谈一谈信息矩阵的作用，以及与Hessian矩阵的关系与作用。
谈一谈舒尔补的边缘化原理与本质
在舒尔补下信息矩阵的更新，先验的定义与本质。
信息矩阵更新牵涉到的零空间问题，以及FEJ的引入~
（参考资料较多，吸取了知乎等大佬的回答，也包含一家之言，欢迎批评指正）

1 📖 信息矩阵

1-1 🔖 信息矩阵是什么？有什么作用？

本小节参考: 信息矩阵在图优化slam里面的作用表示感谢

信息矩阵是一个scalar 表达不确定性

       e 
      
     
       i 
      
     
    
      ( 
     
    
      x 
     
    
      ) 
     
    
      = 
     
     
     
       e 
      
     
       i 
      
     
    
      ( 
     
    
      x 
     
     
     
       ) 
      
     
       T 
      
     
     
     
       Ω 
      
     
       i 
      
     
     
     
       e 
      
     
       i 
      
     
    
      ( 
     
    
      x 
     
    
      ) 
     
    
   
     e_{i}(\mathbf{x})=\mathbf{e}_{i}(\mathbf{x})^{T} \mathbf{\Omega}_{i} \mathbf{e}_{i}(\mathbf{x}) 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord"><span class="mord mathdefault">e</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.311664em;"><span class="" style="top: -2.55em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mopen">(</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 1.14133em; vertical-align: -0.25em;"></span><span class="mord"><span class="mord"><span class="mord mathbf">e</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.311664em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mopen">(</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mclose"><span class="mclose">)</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.891331em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.13889em;">T</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord mathbf">Ω</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.311664em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord mathbf">e</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.311664em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mopen">(</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mclose">)</span></span></span></span></span></span><br> Described by information matrix <span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
   
     Ω 
    
   
  
    \Omega 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.68333em; vertical-align: 0em;"></span><span class="mord">Ω</span></span></span></span></span><br> <span class="katex--display"><span class="katex-display"><span class="katex"><span class="katex-mathml"> 
  
   
    
    
      Ω 
     
    
      = 
     
     
     
       Σ 
      
      
      
        − 
       
      
        1 
       
      
     
    
   
     \Omega=\Sigma^{-1} 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.68333em; vertical-align: 0em;"></span><span class="mord">Ω</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 0.864108em; vertical-align: 0em;"></span><span class="mord"><span class="mord">Σ</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.864108em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">−</span><span class="mord mtight">1</span></span></span></span></span></span></span></span></span></span></span></span></span></span><br> and information vector <span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
   
     ξ 
    
   
  
    \xi 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.88888em; vertical-align: -0.19444em;"></span><span class="mord mathdefault" style="margin-right: 0.04601em;">ξ</span></span></span></span></span><br> <span class="katex--display"><span class="katex-display"><span class="katex"><span class="katex-mathml"> 
  
   
    
    
      ξ 
     
    
      = 
     
     
     
       Σ 
      
      
      
        − 
       
      
        1 
       
      
     
    
      μ 
     
    
   
     \xi=\Sigma^{-1} \mu 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.88888em; vertical-align: -0.19444em;"></span><span class="mord mathdefault" style="margin-right: 0.04601em;">ξ</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 1.05855em; vertical-align: -0.19444em;"></span><span class="mord"><span class="mord">Σ</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.864108em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">−</span><span class="mord mtight">1</span></span></span></span></span></span></span></span></span><span class="mord mathdefault">μ</span></span></span></span></span></span><br> <strong>那为什么需要信息矩阵呢？</strong><br> <span class="katex--display"><span class="katex-display"><span class="katex"><span class="katex-mathml"> 
  
   
    
     
      
       
        
        
          x 
         
        
          ∗ 
         
        
       
      
      
       
        
         
        
          = 
         
         
          
          
            argmin 
           
          
            ⁡ 
           
          
         
           x 
          
         
        
          F 
         
        
          ( 
         
        
          x 
         
        
          ) 
         
        
          − 
         
        
          − 
         
        
          − 
         
        
          − 
         
        
          − 
         
         
        
          &nbsp;global&nbsp;error&nbsp;(scalar)&nbsp; 
         
        
       
      
     
     
      
       
        
       
      
      
       
        
         
        
          = 
         
         
          
          
            argmin 
           
          
            ⁡ 
           
          
         
           x 
          
         
         
         
           ∑ 
          
         
           i 
          
         
         
         
           e 
          
         
           i 
          
         
        
          ( 
         
        
          x 
         
        
          ) 
         
        
          − 
         
        
          − 
         
        
          − 
         
        
          − 
         
        
          − 
         
        
          &nbsp;squared&nbsp;error&nbsp;terms&nbsp;(scalar)&nbsp; 
         
        
       
      
     
     
      
       
        
       
      
      
       
        
         
        
          = 
         
         
          
          
            argmin 
           
          
            ⁡ 
           
          
         
           x 
          
         
         
         
           ∑ 
          
         
           i 
          
         
         
         
           e 
          
         
           i 
          
         
           T 
          
         
        
          ( 
         
        
          x 
         
        
          ) 
         
         
         
           Ω 
          
         
           i 
          
         
         
         
           e 
          
         
           i 
          
         
        
          ( 
         
        
          x 
         
        
          ) 
         
        
          − 
         
        
          − 
         
        
          − 
         
        
          error&nbsp;terms&nbsp;(vector)&nbsp; 
         
        
       
      
     
    
   
     <span class="MathJax_Preview" style="color: inherit; display: none;"></span><div class="MathJax_Display"><span class="MathJax MathJax_FullWidth" id="MathJax-Element-1-Frame" tabindex="0" style="position: relative;" data-mathml="<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot; display=&quot;block&quot;><mtable columnalign=&quot;right left right left right left right left right left right left&quot; rowspacing=&quot;3pt&quot; columnspacing=&quot;0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em&quot; displaystyle=&quot;true&quot;><mtr><mtd><msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>x</mi></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo>&amp;#x2217;</mo></mrow></msup></mtd><mtd><mi></mi><mo>=</mo><munder><mi>argmin</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>x</mi></mrow></munder><mi>F</mi><mo stretchy=&quot;false&quot;>(</mo><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>x</mi></mrow><mo stretchy=&quot;false&quot;>)</mo><mo>&amp;#x2212;</mo><mo>&amp;#x2212;</mo><mo>&amp;#x2212;</mo><mo>&amp;#x2212;</mo><mo>&amp;#x2212;</mo><mspace width=&quot;1em&quot; /><mtext>&amp;#xA0;global error (scalar)&amp;#xA0;</mtext></mtd></mtr><mtr><mtd /><mtd><mi></mi><mo>=</mo><munder><mi>argmin</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>x</mi></mrow></munder><munder><mo>&amp;#x2211;</mo><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>i</mi></mrow></munder><msub><mi>e</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>i</mi></mrow></msub><mo stretchy=&quot;false&quot;>(</mo><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>x</mi></mrow><mo stretchy=&quot;false&quot;>)</mo><mo>&amp;#x2212;</mo><mo>&amp;#x2212;</mo><mo>&amp;#x2212;</mo><mo>&amp;#x2212;</mo><mo>&amp;#x2212;</mo><mtext>&amp;#xA0;squared error terms (scalar)&amp;#xA0;</mtext></mtd></mtr><mtr><mtd /><mtd><mi></mi><mo>=</mo><munder><mi>argmin</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>x</mi></mrow></munder><munder><mo>&amp;#x2211;</mo><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>i</mi></mrow></munder><msubsup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>e</mi></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>i</mi></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>T</mi></mrow></msubsup><mo stretchy=&quot;false&quot;>(</mo><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>x</mi></mrow><mo stretchy=&quot;false&quot;>)</mo><msub><mi mathvariant=&quot;normal&quot;>&amp;#x03A9;</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>i</mi></mrow></msub><msub><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>e</mi></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>i</mi></mrow></msub><mo stretchy=&quot;false&quot;>(</mo><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>x</mi></mrow><mo stretchy=&quot;false&quot;>)</mo><mo>&amp;#x2212;</mo><mo>&amp;#x2212;</mo><mo>&amp;#x2212;</mo><mtext>error terms (vector)&amp;#xA0;</mtext></mtd></mtr></mtable></math>" role="presentation"><nobr aria-hidden="true"><span class="math" id="MathJax-Span-1" style="width: 100%; display: inline-block; min-width: 27.541em;"><span style="display: inline-block; position: relative; width: 100%; height: 0px; font-size: 102%;"><span style="position: absolute; clip: rect(3.9em, 1026.73em, 10.785em, -999.997em); top: -7.591em; left: 0em; width: 100%;"><span class="mrow" id="MathJax-Span-2"><span style="display: inline-block; position: relative; width: 100%; height: 0px;"><span style="position: absolute; clip: rect(3.9em, 1026.73em, 10.785em, -999.997em); top: -7.591em; left: 50%; margin-left: -13.463em;"><span class="mtable" id="MathJax-Span-3" style="padding-left: 0.154em;"><span style="display: inline-block; position: relative; width: 26.833em; height: 0px;"><span style="position: absolute; clip: rect(2.686em, 1001.01em, 8.355em, -999.997em); top: -6.325em; left: 0em;"><span style="display: inline-block; position: relative; width: 1.015em; height: 0px;"><span style="position: absolute; width: 100%; clip: rect(3.091em, 1001.01em, 4.154em, -999.997em); top: -6.73em; left: 0em;"><span class="mtd" id="MathJax-Span-4"><span class="mrow" id="MathJax-Span-5"><span style="display: inline-block; position: relative; width: 100%; height: 0px;"><span style="position: absolute; clip: rect(3.091em, 1001.01em, 4.154em, -999.997em); top: -3.997em; left: 50%; margin-left: -0.504em;"><span class="msubsup" id="MathJax-Span-6"><span style="display: inline-block; position: relative; width: 1.015em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="texatom" id="MathJax-Span-7"><span class="mrow" id="MathJax-Span-8"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-9" style="font-family: MathJax_Main-bold;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.61em;"><span class="texatom" id="MathJax-Span-10"><span class="mrow" id="MathJax-Span-11"><span style="display: inline-block; position: relative; width: 0.357em; height: 0px;"><span style="position: absolute; clip: rect(3.546em, 1000.31em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mo" id="MathJax-Span-12" style="font-size: 70.7%; font-family: MathJax_Main;">∗</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.85em, 1000em, 4.154em, -999.997em); top: -4.604em; right: 0em;"><span class="mtd" id="MathJax-Span-35"><span class="mrow" id="MathJax-Span-36"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.85em, 1000em, 4.154em, -999.997em); top: -2.124em; right: 0em;"><span class="mtd" id="MathJax-Span-67"><span class="mrow" id="MathJax-Span-68"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span><span style="display: inline-block; width: 0px; height: 6.33em;"></span></span><span style="position: absolute; clip: rect(3.9em, 1025.57em, 10.735em, -999.997em); top: -7.54em; left: 1.015em;"><span style="display: inline-block; position: relative; width: 25.769em; height: 0px;"><span style="position: absolute; width: 100%; clip: rect(3.091em, 1021.47em, 5.065em, -999.997em); top: -6.73em; left: 0em;"><span class="mtd" id="MathJax-Span-13"><span class="mrow" id="MathJax-Span-14"><span style="display: inline-block; position: relative; width: 100%; height: 0px;"><span style="position: absolute; clip: rect(3.091em, 1021.47em, 5.065em, -999.997em); top: -3.997em; left: 50%; margin-left: -10.831em;"><span class="mi" id="MathJax-Span-15"></span><span class="mo" id="MathJax-Span-16" style="font-family: MathJax_Main; padding-left: 0.256em;">=</span><span class="munder" id="MathJax-Span-17" style="padding-left: 0.256em;"><span style="display: inline-block; position: relative; width: 3.04em; height: 0px;"><span style="position: absolute; clip: rect(3.192em, 1003.04em, 4.356em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-18" style="font-family: MathJax_Main;">argmin</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.546em, 1000.36em, 4.255em, -999.997em); top: -3.187em; left: 1.369em;"><span class="texatom" id="MathJax-Span-19"><span class="mrow" id="MathJax-Span-20"><span style="display: inline-block; position: relative; width: 0.357em; height: 0px;"><span style="position: absolute; clip: rect(3.546em, 1000.36em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-21" style="font-size: 70.7%; font-family: MathJax_Main;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="mi" id="MathJax-Span-22" style="font-family: MathJax_Math-italic; padding-left: 0.154em;">F<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span class="mo" id="MathJax-Span-23" style="font-family: MathJax_Main;">(</span><span class="texatom" id="MathJax-Span-24"><span class="mrow" id="MathJax-Span-25"><span style="display: inline-block; position: relative; width: 0.509em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-26" style="font-family: MathJax_Main;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span class="mo" id="MathJax-Span-27" style="font-family: MathJax_Main;">)</span><span class="mo" id="MathJax-Span-28" style="font-family: MathJax_Main; padding-left: 0.205em;">−</span><span class="mo" id="MathJax-Span-29" style="font-family: MathJax_Main; padding-left: 0.205em;">−</span><span class="mo" id="MathJax-Span-30" style="font-family: MathJax_Main; padding-left: 0.205em;">−</span><span class="mo" id="MathJax-Span-31" style="font-family: MathJax_Main; padding-left: 0.205em;">−</span><span class="mo" id="MathJax-Span-32" style="font-family: MathJax_Main; padding-left: 0.205em;">−</span><span class="mspace" id="MathJax-Span-33" style="height: 0em; vertical-align: 0em; width: 1.015em; display: inline-block; overflow: hidden;"></span><span class="mtext" id="MathJax-Span-34" style="font-family: MathJax_Main; padding-left: 0.205em;">&nbsp;global error (scalar)&nbsp;</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; width: 100%; clip: rect(2.888em, 1025.57em, 5.369em, -999.997em); top: -4.604em; left: 0em;"><span class="mtd" id="MathJax-Span-37"><span class="mrow" id="MathJax-Span-38"><span style="display: inline-block; position: relative; width: 100%; height: 0px;"><span style="position: absolute; clip: rect(2.888em, 1025.57em, 5.369em, -999.997em); top: -3.997em; left: 50%; margin-left: -12.906em;"><span class="mi" id="MathJax-Span-39"></span><span class="mo" id="MathJax-Span-40" style="font-family: MathJax_Main; padding-left: 0.256em;">=</span><span class="munder" id="MathJax-Span-41" style="padding-left: 0.256em;"><span style="display: inline-block; position: relative; width: 3.04em; height: 0px;"><span style="position: absolute; clip: rect(3.192em, 1003.04em, 4.356em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-42" style="font-family: MathJax_Main;">argmin</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.546em, 1000.36em, 4.255em, -999.997em); top: -3.187em; left: 1.369em;"><span class="texatom" id="MathJax-Span-43"><span class="mrow" id="MathJax-Span-44"><span style="display: inline-block; position: relative; width: 0.357em; height: 0px;"><span style="position: absolute; clip: rect(3.546em, 1000.36em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-45" style="font-size: 70.7%; font-family: MathJax_Main;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="munderover" id="MathJax-Span-46" style="padding-left: 0.154em;"><span style="display: inline-block; position: relative; width: 1.471em; height: 0px;"><span style="position: absolute; clip: rect(2.888em, 1001.37em, 4.609em, -999.997em); top: -3.997em; left: 0em;"><span class="mo" id="MathJax-Span-47" style="font-family: MathJax_Size2; vertical-align: 0em;">∑</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.394em, 1000.21em, 4.255em, -999.997em); top: -2.934em; left: 0.61em;"><span class="texatom" id="MathJax-Span-48"><span class="mrow" id="MathJax-Span-49"><span style="display: inline-block; position: relative; width: 0.256em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.21em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-50" style="font-size: 70.7%; font-family: MathJax_Math-italic;">i</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="msubsup" id="MathJax-Span-51" style="padding-left: 0.154em;"><span style="display: inline-block; position: relative; width: 0.812em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.41em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-52" style="font-family: MathJax_Math-italic;">e</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -3.845em; left: 0.458em;"><span class="texatom" id="MathJax-Span-53"><span class="mrow" id="MathJax-Span-54"><span style="display: inline-block; position: relative; width: 0.256em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.21em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-55" style="font-size: 70.7%; font-family: MathJax_Math-italic;">i</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="mo" id="MathJax-Span-56" style="font-family: MathJax_Main;">(</span><span class="texatom" id="MathJax-Span-57"><span class="mrow" id="MathJax-Span-58"><span style="display: inline-block; position: relative; width: 0.509em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-59" style="font-family: MathJax_Main;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span class="mo" id="MathJax-Span-60" style="font-family: MathJax_Main;">)</span><span class="mo" id="MathJax-Span-61" style="font-family: MathJax_Main; padding-left: 0.205em;">−</span><span class="mo" id="MathJax-Span-62" style="font-family: MathJax_Main; padding-left: 0.205em;">−</span><span class="mo" id="MathJax-Span-63" style="font-family: MathJax_Main; padding-left: 0.205em;">−</span><span class="mo" id="MathJax-Span-64" style="font-family: MathJax_Main; padding-left: 0.205em;">−</span><span class="mo" id="MathJax-Span-65" style="font-family: MathJax_Main; padding-left: 0.205em;">−</span><span class="mtext" id="MathJax-Span-66" style="font-family: MathJax_Main; padding-left: 0.205em;">&nbsp;squared error terms (scalar)&nbsp;</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; width: 100%; clip: rect(2.888em, 1023.24em, 5.369em, -999.997em); top: -2.124em; left: 0em;"><span class="mtd" id="MathJax-Span-69"><span class="mrow" id="MathJax-Span-70"><span style="display: inline-block; position: relative; width: 100%; height: 0px;"><span style="position: absolute; clip: rect(2.888em, 1023.24em, 5.369em, -999.997em); top: -3.997em; left: 50%; margin-left: -11.742em;"><span class="mi" id="MathJax-Span-71"></span><span class="mo" id="MathJax-Span-72" style="font-family: MathJax_Main; padding-left: 0.256em;">=</span><span class="munder" id="MathJax-Span-73" style="padding-left: 0.256em;"><span style="display: inline-block; position: relative; width: 3.04em; height: 0px;"><span style="position: absolute; clip: rect(3.192em, 1003.04em, 4.356em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-74" style="font-family: MathJax_Main;">argmin</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.546em, 1000.36em, 4.255em, -999.997em); top: -3.187em; left: 1.369em;"><span class="texatom" id="MathJax-Span-75"><span class="mrow" id="MathJax-Span-76"><span style="display: inline-block; position: relative; width: 0.357em; height: 0px;"><span style="position: absolute; clip: rect(3.546em, 1000.36em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-77" style="font-size: 70.7%; font-family: MathJax_Main;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="munderover" id="MathJax-Span-78" style="padding-left: 0.154em;"><span style="display: inline-block; position: relative; width: 1.471em; height: 0px;"><span style="position: absolute; clip: rect(2.888em, 1001.37em, 4.609em, -999.997em); top: -3.997em; left: 0em;"><span class="mo" id="MathJax-Span-79" style="font-family: MathJax_Size2; vertical-align: 0em;">∑</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.394em, 1000.21em, 4.255em, -999.997em); top: -2.934em; left: 0.61em;"><span class="texatom" id="MathJax-Span-80"><span class="mrow" id="MathJax-Span-81"><span style="display: inline-block; position: relative; width: 0.256em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.21em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-82" style="font-size: 70.7%; font-family: MathJax_Math-italic;">i</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="msubsup" id="MathJax-Span-83" style="padding-left: 0.154em;"><span style="display: inline-block; position: relative; width: 1.015em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.41em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="texatom" id="MathJax-Span-84"><span class="mrow" id="MathJax-Span-85"><span style="display: inline-block; position: relative; width: 0.458em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.41em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-86" style="font-family: MathJax_Main;">e</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.394em, 1000.56em, 4.154em, -999.997em); top: -4.351em; left: 0.458em;"><span class="texatom" id="MathJax-Span-87"><span class="mrow" id="MathJax-Span-88"><span style="display: inline-block; position: relative; width: 0.509em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-89" style="font-size: 70.7%; font-family: MathJax_Math-italic;">T<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.394em, 1000.31em, 4.154em, -999.997em); top: -3.693em; left: 0.458em;"><span class="texatom" id="MathJax-Span-90"><span class="mrow" id="MathJax-Span-91"><span style="display: inline-block; position: relative; width: 0.256em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.21em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-92" style="font-size: 70.7%; font-family: MathJax_Math-italic;">i</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="mo" id="MathJax-Span-93" style="font-family: MathJax_Main;">(</span><span class="texatom" id="MathJax-Span-94"><span class="mrow" id="MathJax-Span-95"><span style="display: inline-block; position: relative; width: 0.509em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-96" style="font-family: MathJax_Main;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span class="mo" id="MathJax-Span-97" style="font-family: MathJax_Main;">)</span><span class="msubsup" id="MathJax-Span-98"><span style="display: inline-block; position: relative; width: 1.066em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.66em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-99" style="font-family: MathJax_Main;">Ω</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -3.845em; left: 0.711em;"><span class="texatom" id="MathJax-Span-100"><span class="mrow" id="MathJax-Span-101"><span style="display: inline-block; position: relative; width: 0.256em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.21em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-102" style="font-size: 70.7%; font-family: MathJax_Math-italic;">i</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="msubsup" id="MathJax-Span-103"><span style="display: inline-block; position: relative; width: 0.762em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.41em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="texatom" id="MathJax-Span-104"><span class="mrow" id="MathJax-Span-105"><span style="display: inline-block; position: relative; width: 0.458em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.41em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-106" style="font-family: MathJax_Main;">e</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -3.845em; left: 0.458em;"><span class="texatom" id="MathJax-Span-107"><span class="mrow" id="MathJax-Span-108"><span style="display: inline-block; position: relative; width: 0.256em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.21em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-109" style="font-size: 70.7%; font-family: MathJax_Math-italic;">i</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="mo" id="MathJax-Span-110" style="font-family: MathJax_Main;">(</span><span class="texatom" id="MathJax-Span-111"><span class="mrow" id="MathJax-Span-112"><span style="display: inline-block; position: relative; width: 0.509em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-113" style="font-family: MathJax_Main;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span class="mo" id="MathJax-Span-114" style="font-family: MathJax_Main;">)</span><span class="mo" id="MathJax-Span-115" style="font-family: MathJax_Main; padding-left: 0.205em;">−</span><span class="mo" id="MathJax-Span-116" style="font-family: MathJax_Main; padding-left: 0.205em;">−</span><span class="mo" id="MathJax-Span-117" style="font-family: MathJax_Main; padding-left: 0.205em;">−</span><span class="mtext" id="MathJax-Span-118" style="font-family: MathJax_Main; padding-left: 0.205em;">error terms (vector)&nbsp;</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span><span style="display: inline-block; width: 0px; height: 7.545em;"></span></span></span></span><span style="display: inline-block; width: 0px; height: 7.596em;"></span></span></span></span><span style="display: inline-block; width: 0px; height: 7.596em;"></span></span></span><span style="display: inline-block; overflow: hidden; vertical-align: -3.147em; border-left: 0px solid; width: 0px; height: 6.818em;"></span></span></nobr><span class="MJX_Assistive_MathML MJX_Assistive_MathML_Block" role="presentation"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mtable columnalign="right left right left right left right left right left right left" rowspacing="3pt" columnspacing="0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em" displaystyle="true"><mtr><mtd><msup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">x</mi></mrow><mrow class="MJX-TeXAtom-ORD"><mo>∗</mo></mrow></msup></mtd><mtd><mi></mi><mo>=</mo><munder><mi>argmin</mi><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">x</mi></mrow></munder><mi>F</mi><mo stretchy="false">(</mo><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">x</mi></mrow><mo stretchy="false">)</mo><mo>−</mo><mo>−</mo><mo>−</mo><mo>−</mo><mo>−</mo><mspace width="1em"></mspace><mtext>&nbsp;global error (scalar)&nbsp;</mtext></mtd></mtr><mtr><mtd></mtd><mtd><mi></mi><mo>=</mo><munder><mi>argmin</mi><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">x</mi></mrow></munder><munder><mo>∑</mo><mrow class="MJX-TeXAtom-ORD"><mi>i</mi></mrow></munder><msub><mi>e</mi><mrow class="MJX-TeXAtom-ORD"><mi>i</mi></mrow></msub><mo stretchy="false">(</mo><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">x</mi></mrow><mo stretchy="false">)</mo><mo>−</mo><mo>−</mo><mo>−</mo><mo>−</mo><mo>−</mo><mtext>&nbsp;squared error terms (scalar)&nbsp;</mtext></mtd></mtr><mtr><mtd></mtd><mtd><mi></mi><mo>=</mo><munder><mi>argmin</mi><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">x</mi></mrow></munder><munder><mo>∑</mo><mrow class="MJX-TeXAtom-ORD"><mi>i</mi></mrow></munder><msubsup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">e</mi></mrow><mrow class="MJX-TeXAtom-ORD"><mi>i</mi></mrow><mrow class="MJX-TeXAtom-ORD"><mi>T</mi></mrow></msubsup><mo stretchy="false">(</mo><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">x</mi></mrow><mo stretchy="false">)</mo><msub><mi mathvariant="normal">Ω</mi><mrow class="MJX-TeXAtom-ORD"><mi>i</mi></mrow></msub><msub><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">e</mi></mrow><mrow class="MJX-TeXAtom-ORD"><mi>i</mi></mrow></msub><mo stretchy="false">(</mo><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">x</mi></mrow><mo stretchy="false">)</mo><mo>−</mo><mo>−</mo><mo>−</mo><mtext>error terms (vector)&nbsp;</mtext></mtd></mtr></mtable></math></span></span></div><script type="math/tex; mode=display" id="MathJax-Element-1">\begin{aligned} \mathbf{x}^{*} &=\underset{\mathrm{x}}{\operatorname{argmin}} F(\mathrm{x}) ----- \quad \text { global error (scalar) } \\ &=\underset{\mathrm{x}}{\operatorname{argmin}} \sum_{i} e_{i}(\mathrm{x}) ----- \text { squared error terms (scalar) } \\ &=\underset{\mathrm{x}}{\operatorname{argmin}} \sum_{i} \mathrm{e}_{i}^{T}(\mathrm{x}) \Omega_{i} \mathrm{e}_{i}(\mathrm{x}) --- \text {error terms (vector) } \end{aligned}</script> 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 7.28979em; vertical-align: -3.39489em;"></span><span class="mord"><span class="mtable"><span class="col-align-r"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 3.89489em;"><span class="" style="top: -6.1049em;"><span class="pstrut" style="height: 3.05001em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord mathbf">x</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.738696em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">∗</span></span></span></span></span></span></span></span></span></span></span><span class="" style="top: -3.86045em;"><span class="pstrut" style="height: 3.05001em;"></span><span class="mord"></span></span><span class="" style="top: -1.23278em;"><span class="pstrut" style="height: 3.05001em;"></span><span class="mord"></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 3.39489em;"><span class=""></span></span></span></span></span><span class="col-align-l"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 3.89489em;"><span class="" style="top: -6.1049em;"><span class="pstrut" style="height: 3.05001em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord"><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.66786em;"><span class="" style="top: -2.20556em; margin-left: 0em;"><span class="pstrut" style="height: 3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mathrm mtight">x</span></span></span></span></span><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class=""><span class="mop"><span class="mop"><span class="mord mathrm">a</span><span class="mord mathrm">r</span><span class="mord mathrm" style="margin-right: 0.01389em;">g</span><span class="mord mathrm">m</span><span class="mord mathrm">i</span><span class="mord mathrm">n</span></span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.89444em;"><span class=""></span></span></span></span></span></span><span class="mord mathdefault" style="margin-right: 0.13889em;">F</span><span class="mopen">(</span><span class="mord"><span class="mord mathrm">x</span></span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mspace" style="margin-right: 1em;"></span><span class="mord text"><span class="mord">&nbsp;global&nbsp;error&nbsp;(scalar)&nbsp;</span></span></span></span><span class="" style="top: -3.86045em;"><span class="pstrut" style="height: 3.05001em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord"><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.66786em;"><span class="" style="top: -2.20556em; margin-left: 0em;"><span class="pstrut" style="height: 3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mathrm mtight">x</span></span></span></span></span><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class=""><span class="mop"><span class="mop"><span class="mord mathrm">a</span><span class="mord mathrm">r</span><span class="mord mathrm" style="margin-right: 0.01389em;">g</span><span class="mord mathrm">m</span><span class="mord mathrm">i</span><span class="mord mathrm">n</span></span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.89444em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.05001em;"><span class="" style="top: -1.87233em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span><span class="" style="top: -3.05001em;"><span class="pstrut" style="height: 3.05em;"></span><span class=""><span class="mop op-symbol large-op">∑</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 1.27767em;"><span class=""></span></span></span></span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mord"><span class="mord mathdefault">e</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.311664em;"><span class="" style="top: -2.55em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mopen">(</span><span class="mord"><span class="mord mathrm">x</span></span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord text"><span class="mord">&nbsp;squared&nbsp;error&nbsp;terms&nbsp;(scalar)&nbsp;</span></span></span></span><span class="" style="top: -1.23278em;"><span class="pstrut" style="height: 3.05001em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord"><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.66786em;"><span class="" style="top: -2.20556em; margin-left: 0em;"><span class="pstrut" style="height: 3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mathrm mtight">x</span></span></span></span></span><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class=""><span class="mop"><span class="mop"><span class="mord mathrm">a</span><span class="mord mathrm">r</span><span class="mord mathrm" style="margin-right: 0.01389em;">g</span><span class="mord mathrm">m</span><span class="mord mathrm">i</span><span class="mord mathrm">n</span></span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.89444em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.05001em;"><span class="" style="top: -1.87233em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span><span class="" style="top: -3.05001em;"><span class="pstrut" style="height: 3.05em;"></span><span class=""><span class="mop op-symbol large-op">∑</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 1.27767em;"><span class=""></span></span></span></span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mord"><span class="mord"><span class="mord mathrm">e</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.891331em;"><span class="" style="top: -2.453em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.13889em;">T</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mopen">(</span><span class="mord"><span class="mord mathrm">x</span></span><span class="mclose">)</span><span class="mord"><span class="mord">Ω</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.311664em;"><span class="" style="top: -2.55em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord mathrm">e</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.311664em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mopen">(</span><span class="mord"><span class="mord mathrm">x</span></span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord text"><span class="mord">error&nbsp;terms&nbsp;(vector)&nbsp;</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 3.39489em;"><span class=""></span></span></span></span></span></span></span></span></span></span></span></span></p>

系统可能有很多传感器，传感器精度越高，对应的information matrix里面的系数会很大（这里是越大越好，因为它是协方差矩阵的逆矩阵），系数越大代表权重越高，表达的信息越多，在优化的过程中就越会被重视。用一个形象的数学表达式表达就是：

const int INT_MAX=1e9;
argmin( INT_MAX*(x-3)^2+1/INT_MAX*(x-1)^2)) 

 
 
 
 1
2

那么INT MAX就代表我们的精确传感器，那么优化的结果肯定是 x=3;也就是说，我们更加相信我们好的传感器

1-2 🔖 信息矩阵与Hessian矩阵的关系

本小节参考链接：
文章
[SLAM的滑动窗口算法中，在边缘化时，高斯牛顿法的信息矩阵为什么是优化变量协方差的逆？]

[Why is the observed Fisher information defined as the Hessian of the log-likelihood?]
PDF
Maximum Likelihood Estimation (MLE).pdf

先放出结论：

Hessian矩阵在最大似然（MLE）问题中被认为约等于信息矩阵，所以一般也会将Hessian矩阵直接当做信息矩阵对待。

协方差的逆=信息矩阵，这个结论是建立在假设分布是高斯分布这个前提下，计算最大后验概率（MAP）优化问题得出来的

1-2-1 Hessian矩阵和H矩阵的关系

Hessian矩阵平时接触的可能不多, 但是Hessian矩阵的近似矩阵H矩阵就比较多了, 因为总是在求解优化问题，必不可少的就会接触到优化问题的H矩阵, 通常我们见到的都是最小二乘问题中的H矩阵, 如下有：

         E 
        
       
      
      
       
        
         
        
          = 
         
        
          ∥ 
         
        
          z 
         
        
          − 
         
        
          f 
         
        
          ( 
         
        
          x 
         
        
          ) 
         
         
         
           ∥ 
          
         
           w 
          
         
           2 
          
         
        
          = 
         
        
          ∥ 
         
        
          z 
         
        
          − 
         
        
          f 
         
        
          ( 
         
        
          x 
         
        
          ) 
         
        
          + 
         
        
          J 
         
        
          δ 
         
        
          x 
         
         
         
           ∥ 
          
         
           w 
          
         
           2 
          
         
        
       
      
     
     
      
       
        
       
      
      
       
        
         
        
          = 
         
        
          ( 
         
        
          e 
         
        
          + 
         
        
          J 
         
        
          δ 
         
        
          x 
         
         
         
           ) 
          
         
           T 
          
         
        
          W 
         
        
          ( 
         
        
          e 
         
        
          + 
         
        
          J 
         
        
          δ 
         
        
          x 
         
        
          ) 
         
        
       
      
     
     
      
       
        
       
      
      
       
        
         
        
          = 
         
         
         
           e 
          
         
           T 
          
         
        
          w 
         
        
          e 
         
        
          + 
         
        
          δ 
         
         
         
           x 
          
         
           T 
          
         
         
         
           J 
          
         
           T 
          
         
        
          W 
         
        
          e 
         
        
          + 
         
         
         
           e 
          
         
           T 
          
         
        
          W 
         
        
          J 
         
        
          δ 
         
        
          x 
         
        
          + 
         
        
          δ 
         
         
         
           x 
          
         
           T 
          
         
         
         
           J 
          
         
           T 
          
         
        
          W 
         
        
          J 
         
        
          δ 
         
        
          x 
         
        
       
      
     
    
   
     <span class="MathJax_Preview" style="color: inherit; display: none;"></span><div class="MathJax_Display"><span class="MathJax MathJax_FullWidth" id="MathJax-Element-2-Frame" tabindex="0" style="position: relative;" data-mathml="<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot; display=&quot;block&quot;><mtable columnalign=&quot;right left right left right left right left right left right left&quot; rowspacing=&quot;3pt&quot; columnspacing=&quot;0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em&quot; displaystyle=&quot;true&quot;><mtr><mtd><mi>E</mi></mtd><mtd><mi></mi><mo>=</mo><mo fence=&quot;false&quot; stretchy=&quot;false&quot;>&amp;#x2016;</mo><mi>z</mi><mo>&amp;#x2212;</mo><mi>f</mi><mo stretchy=&quot;false&quot;>(</mo><mi>x</mi><mo stretchy=&quot;false&quot;>)</mo><msubsup><mo fence=&quot;false&quot; stretchy=&quot;false&quot;>&amp;#x2016;</mo><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>w</mi></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mn>2</mn></mrow></msubsup><mo>=</mo><mo fence=&quot;false&quot; stretchy=&quot;false&quot;>&amp;#x2016;</mo><mi>z</mi><mo>&amp;#x2212;</mo><mi>f</mi><mo stretchy=&quot;false&quot;>(</mo><mi>x</mi><mo stretchy=&quot;false&quot;>)</mo><mo>+</mo><mi>J</mi><mi>&amp;#x03B4;</mi><mi>x</mi><msubsup><mo fence=&quot;false&quot; stretchy=&quot;false&quot;>&amp;#x2016;</mo><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>w</mi></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mn>2</mn></mrow></msubsup></mtd></mtr><mtr><mtd /><mtd><mi></mi><mo>=</mo><mo stretchy=&quot;false&quot;>(</mo><mi>e</mi><mo>+</mo><mi>J</mi><mi>&amp;#x03B4;</mi><mi>x</mi><msup><mo stretchy=&quot;false&quot;>)</mo><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>T</mi></mrow></msup><mi>W</mi><mo stretchy=&quot;false&quot;>(</mo><mi>e</mi><mo>+</mo><mi>J</mi><mi>&amp;#x03B4;</mi><mi>x</mi><mo stretchy=&quot;false&quot;>)</mo></mtd></mtr><mtr><mtd /><mtd><mi></mi><mo>=</mo><msup><mi>e</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>T</mi></mrow></msup><mi>w</mi><mi>e</mi><mo>+</mo><mi>&amp;#x03B4;</mi><msup><mi>x</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>T</mi></mrow></msup><msup><mi>J</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>T</mi></mrow></msup><mi>W</mi><mi>e</mi><mo>+</mo><msup><mi>e</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>T</mi></mrow></msup><mi>W</mi><mi>J</mi><mi>&amp;#x03B4;</mi><mi>x</mi><mo>+</mo><mi>&amp;#x03B4;</mi><msup><mi>x</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>T</mi></mrow></msup><msup><mi>J</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi>T</mi></mrow></msup><mi>W</mi><mi>J</mi><mi>&amp;#x03B4;</mi><mi>x</mi></mtd></mtr></mtable></math>" role="presentation"><nobr aria-hidden="true"><span class="math" id="MathJax-Span-119" style="width: 100%; display: inline-block; min-width: 22.175em;"><span style="display: inline-block; position: relative; width: 100%; height: 0px; font-size: 102%;"><span style="position: absolute; clip: rect(2.584em, 1021.67em, 6.786em, -999.997em); top: -4.908em; left: 0em; width: 100%;"><span class="mrow" id="MathJax-Span-120"><span style="display: inline-block; position: relative; width: 100%; height: 0px;"><span style="position: absolute; clip: rect(2.584em, 1021.67em, 6.786em, -999.997em); top: -4.908em; left: 50%; margin-left: -10.831em;"><span class="mtable" id="MathJax-Span-121" style="padding-left: 0.154em;"><span style="display: inline-block; position: relative; width: 21.568em; height: 0px;"><span style="position: absolute; clip: rect(2.331em, 1000.76em, 6.229em, -999.997em); top: -4.554em; left: 0em;"><span style="display: inline-block; position: relative; width: 0.762em; height: 0px;"><span style="position: absolute; width: 100%; clip: rect(3.192em, 1000.76em, 4.154em, -999.997em); top: -5.363em; left: 0em;"><span class="mtd" id="MathJax-Span-122"><span class="mrow" id="MathJax-Span-123"><span style="display: inline-block; position: relative; width: 100%; height: 0px;"><span style="position: absolute; clip: rect(3.192em, 1000.76em, 4.154em, -999.997em); top: -3.997em; left: 50%; margin-left: -0.402em;"><span class="mi" id="MathJax-Span-124" style="font-family: MathJax_Math-italic;">E<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.053em;"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.85em, 1000em, 4.154em, -999.997em); top: -3.946em; right: 0em;"><span class="mtd" id="MathJax-Span-164"><span class="mrow" id="MathJax-Span-165"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.85em, 1000em, 4.154em, -999.997em); top: -2.478em; right: 0em;"><span class="mtd" id="MathJax-Span-189"><span class="mrow" id="MathJax-Span-190"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span><span style="display: inline-block; width: 0px; height: 4.559em;"></span></span><span style="position: absolute; clip: rect(2.432em, 1020.76em, 6.482em, -999.997em); top: -4.756em; left: 0.762em;"><span style="display: inline-block; position: relative; width: 20.808em; height: 0px;"><span style="position: absolute; width: 100%; clip: rect(3.04em, 1015.7em, 4.407em, -999.997em); top: -5.363em; left: 0em;"><span class="mtd" id="MathJax-Span-125"><span class="mrow" id="MathJax-Span-126"><span style="display: inline-block; position: relative; width: 100%; height: 0px;"><span style="position: absolute; clip: rect(3.04em, 1015.7em, 4.407em, -999.997em); top: -3.997em; left: 50%; margin-left: -7.844em;"><span class="mi" id="MathJax-Span-127"></span><span class="mo" id="MathJax-Span-128" style="font-family: MathJax_Main; padding-left: 0.256em;">=</span><span class="mo" id="MathJax-Span-129" style="font-family: MathJax_Main; padding-left: 0.256em;">∥</span><span class="mi" id="MathJax-Span-130" style="font-family: MathJax_Math-italic;">z<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.003em;"></span></span><span class="mo" id="MathJax-Span-131" style="font-family: MathJax_Main; padding-left: 0.205em;">−</span><span class="mi" id="MathJax-Span-132" style="font-family: MathJax_Math-italic; padding-left: 0.205em;">f<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.053em;"></span></span><span class="mo" id="MathJax-Span-133" style="font-family: MathJax_Main;">(</span><span class="mi" id="MathJax-Span-134" style="font-family: MathJax_Math-italic;">x</span><span class="mo" id="MathJax-Span-135" style="font-family: MathJax_Main;">)</span><span class="msubsup" id="MathJax-Span-136"><span style="display: inline-block; position: relative; width: 1.066em; height: 0px;"><span style="position: absolute; clip: rect(3.091em, 1000.36em, 4.407em, -999.997em); top: -3.997em; left: 0em;"><span class="mo" id="MathJax-Span-137" style="font-family: MathJax_Main;">∥</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.394em, 1000.41em, 4.154em, -999.997em); top: -4.351em; left: 0.509em;"><span class="texatom" id="MathJax-Span-138"><span class="mrow" id="MathJax-Span-139"><span style="display: inline-block; position: relative; width: 0.357em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.31em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mn" id="MathJax-Span-140" style="font-size: 70.7%; font-family: MathJax_Main;">2</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.546em, 1000.56em, 4.154em, -999.997em); top: -3.845em; left: 0.509em;"><span class="texatom" id="MathJax-Span-141"><span class="mrow" id="MathJax-Span-142"><span style="display: inline-block; position: relative; width: 0.509em; height: 0px;"><span style="position: absolute; clip: rect(3.546em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-143" style="font-size: 70.7%; font-family: MathJax_Math-italic;">w</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="mo" id="MathJax-Span-144" style="font-family: MathJax_Main; padding-left: 0.256em;">=</span><span class="mo" id="MathJax-Span-145" style="font-family: MathJax_Main; padding-left: 0.256em;">∥</span><span class="mi" id="MathJax-Span-146" style="font-family: MathJax_Math-italic;">z<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.003em;"></span></span><span class="mo" id="MathJax-Span-147" style="font-family: MathJax_Main; padding-left: 0.205em;">−</span><span class="mi" id="MathJax-Span-148" style="font-family: MathJax_Math-italic; padding-left: 0.205em;">f<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.053em;"></span></span><span class="mo" id="MathJax-Span-149" style="font-family: MathJax_Main;">(</span><span class="mi" id="MathJax-Span-150" style="font-family: MathJax_Math-italic;">x</span><span class="mo" id="MathJax-Span-151" style="font-family: MathJax_Main;">)</span><span class="mo" id="MathJax-Span-152" style="font-family: MathJax_Main; padding-left: 0.205em;">+</span><span class="mi" id="MathJax-Span-153" style="font-family: MathJax_Math-italic; padding-left: 0.205em;">J<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span class="mi" id="MathJax-Span-154" style="font-family: MathJax_Math-italic;">δ<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.003em;"></span></span><span class="mi" id="MathJax-Span-155" style="font-family: MathJax_Math-italic;">x</span><span class="msubsup" id="MathJax-Span-156"><span style="display: inline-block; position: relative; width: 1.066em; height: 0px;"><span style="position: absolute; clip: rect(3.091em, 1000.36em, 4.407em, -999.997em); top: -3.997em; left: 0em;"><span class="mo" id="MathJax-Span-157" style="font-family: MathJax_Main;">∥</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.394em, 1000.41em, 4.154em, -999.997em); top: -4.351em; left: 0.509em;"><span class="texatom" id="MathJax-Span-158"><span class="mrow" id="MathJax-Span-159"><span style="display: inline-block; position: relative; width: 0.357em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.31em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mn" id="MathJax-Span-160" style="font-size: 70.7%; font-family: MathJax_Main;">2</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.546em, 1000.56em, 4.154em, -999.997em); top: -3.845em; left: 0.509em;"><span class="texatom" id="MathJax-Span-161"><span class="mrow" id="MathJax-Span-162"><span style="display: inline-block; position: relative; width: 0.509em; height: 0px;"><span style="position: absolute; clip: rect(3.546em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-163" style="font-size: 70.7%; font-family: MathJax_Math-italic;">w</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; width: 100%; clip: rect(2.939em, 1010.99em, 4.407em, -999.997em); top: -3.946em; left: 0em;"><span class="mtd" id="MathJax-Span-166"><span class="mrow" id="MathJax-Span-167"><span style="display: inline-block; position: relative; width: 100%; height: 0px;"><span style="position: absolute; clip: rect(2.939em, 1010.99em, 4.407em, -999.997em); top: -3.997em; left: 50%; margin-left: -5.566em;"><span class="mi" id="MathJax-Span-168"></span><span class="mo" id="MathJax-Span-169" style="font-family: MathJax_Main; padding-left: 0.256em;">=</span><span class="mo" id="MathJax-Span-170" style="font-family: MathJax_Main; padding-left: 0.256em;">(</span><span class="mi" id="MathJax-Span-171" style="font-family: MathJax_Math-italic;">e</span><span class="mo" id="MathJax-Span-172" style="font-family: MathJax_Main; padding-left: 0.205em;">+</span><span class="mi" id="MathJax-Span-173" style="font-family: MathJax_Math-italic; padding-left: 0.205em;">J<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span class="mi" id="MathJax-Span-174" style="font-family: MathJax_Math-italic;">δ<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.003em;"></span></span><span class="mi" id="MathJax-Span-175" style="font-family: MathJax_Math-italic;">x</span><span class="msubsup" id="MathJax-Span-176"><span style="display: inline-block; position: relative; width: 0.964em; height: 0px;"><span style="position: absolute; clip: rect(3.091em, 1000.31em, 4.407em, -999.997em); top: -3.997em; left: 0em;"><span class="mo" id="MathJax-Span-177" style="font-family: MathJax_Main;">)</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.408em;"><span class="texatom" id="MathJax-Span-178"><span class="mrow" id="MathJax-Span-179"><span style="display: inline-block; position: relative; width: 0.509em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-180" style="font-size: 70.7%; font-family: MathJax_Math-italic;">T<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="mi" id="MathJax-Span-181" style="font-family: MathJax_Math-italic;">W<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span class="mo" id="MathJax-Span-182" style="font-family: MathJax_Main;">(</span><span class="mi" id="MathJax-Span-183" style="font-family: MathJax_Math-italic;">e</span><span class="mo" id="MathJax-Span-184" style="font-family: MathJax_Main; padding-left: 0.205em;">+</span><span class="mi" id="MathJax-Span-185" style="font-family: MathJax_Math-italic; padding-left: 0.205em;">J<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span class="mi" id="MathJax-Span-186" style="font-family: MathJax_Math-italic;">δ<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.003em;"></span></span><span class="mi" id="MathJax-Span-187" style="font-family: MathJax_Math-italic;">x</span><span class="mo" id="MathJax-Span-188" style="font-family: MathJax_Main;">)</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; width: 100%; clip: rect(2.939em, 1020.76em, 4.255em, -999.997em); top: -2.478em; left: 0em;"><span class="mtd" id="MathJax-Span-191"><span class="mrow" id="MathJax-Span-192"><span style="display: inline-block; position: relative; width: 100%; height: 0px;"><span style="position: absolute; clip: rect(2.939em, 1020.76em, 4.255em, -999.997em); top: -3.997em; left: 50%; margin-left: -10.375em;"><span class="mi" id="MathJax-Span-193"></span><span class="mo" id="MathJax-Span-194" style="font-family: MathJax_Main; padding-left: 0.256em;">=</span><span class="msubsup" id="MathJax-Span-195" style="padding-left: 0.256em;"><span style="display: inline-block; position: relative; width: 1.066em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.41em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-196" style="font-family: MathJax_Math-italic;">e</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.458em;"><span class="texatom" id="MathJax-Span-197"><span class="mrow" id="MathJax-Span-198"><span style="display: inline-block; position: relative; width: 0.509em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-199" style="font-size: 70.7%; font-family: MathJax_Math-italic;">T<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="mi" id="MathJax-Span-200" style="font-family: MathJax_Math-italic;">w</span><span class="mi" id="MathJax-Span-201" style="font-family: MathJax_Math-italic;">e</span><span class="mo" id="MathJax-Span-202" style="font-family: MathJax_Main; padding-left: 0.205em;">+</span><span class="mi" id="MathJax-Span-203" style="font-family: MathJax_Math-italic; padding-left: 0.205em;">δ<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.003em;"></span></span><span class="msubsup" id="MathJax-Span-204"><span style="display: inline-block; position: relative; width: 1.167em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-205" style="font-family: MathJax_Math-italic;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.559em;"><span class="texatom" id="MathJax-Span-206"><span class="mrow" id="MathJax-Span-207"><span style="display: inline-block; position: relative; width: 0.509em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-208" style="font-size: 70.7%; font-family: MathJax_Math-italic;">T<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="msubsup" id="MathJax-Span-209"><span style="display: inline-block; position: relative; width: 1.268em; height: 0px;"><span style="position: absolute; clip: rect(3.192em, 1000.66em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-210" style="font-family: MathJax_Math-italic;">J<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.711em;"><span class="texatom" id="MathJax-Span-211"><span class="mrow" id="MathJax-Span-212"><span style="display: inline-block; position: relative; width: 0.509em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-213" style="font-size: 70.7%; font-family: MathJax_Math-italic;">T<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="mi" id="MathJax-Span-214" style="font-family: MathJax_Math-italic;">W<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span class="mi" id="MathJax-Span-215" style="font-family: MathJax_Math-italic;">e</span><span class="mo" id="MathJax-Span-216" style="font-family: MathJax_Main; padding-left: 0.205em;">+</span><span class="msubsup" id="MathJax-Span-217" style="padding-left: 0.205em;"><span style="display: inline-block; position: relative; width: 1.066em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.41em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-218" style="font-family: MathJax_Math-italic;">e</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.458em;"><span class="texatom" id="MathJax-Span-219"><span class="mrow" id="MathJax-Span-220"><span style="display: inline-block; position: relative; width: 0.509em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-221" style="font-size: 70.7%; font-family: MathJax_Math-italic;">T<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="mi" id="MathJax-Span-222" style="font-family: MathJax_Math-italic;">W<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span class="mi" id="MathJax-Span-223" style="font-family: MathJax_Math-italic;">J<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span class="mi" id="MathJax-Span-224" style="font-family: MathJax_Math-italic;">δ<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.003em;"></span></span><span class="mi" id="MathJax-Span-225" style="font-family: MathJax_Math-italic;">x</span><span class="mo" id="MathJax-Span-226" style="font-family: MathJax_Main; padding-left: 0.205em;">+</span><span class="mi" id="MathJax-Span-227" style="font-family: MathJax_Math-italic; padding-left: 0.205em;">δ<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.003em;"></span></span><span class="msubsup" id="MathJax-Span-228"><span style="display: inline-block; position: relative; width: 1.167em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-229" style="font-family: MathJax_Math-italic;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.559em;"><span class="texatom" id="MathJax-Span-230"><span class="mrow" id="MathJax-Span-231"><span style="display: inline-block; position: relative; width: 0.509em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-232" style="font-size: 70.7%; font-family: MathJax_Math-italic;">T<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="msubsup" id="MathJax-Span-233"><span style="display: inline-block; position: relative; width: 1.268em; height: 0px;"><span style="position: absolute; clip: rect(3.192em, 1000.66em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-234" style="font-family: MathJax_Math-italic;">J<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.711em;"><span class="texatom" id="MathJax-Span-235"><span class="mrow" id="MathJax-Span-236"><span style="display: inline-block; position: relative; width: 0.509em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-237" style="font-size: 70.7%; font-family: MathJax_Math-italic;">T<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="mi" id="MathJax-Span-238" style="font-family: MathJax_Math-italic;">W<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span class="mi" id="MathJax-Span-239" style="font-family: MathJax_Math-italic;">J<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span class="mi" id="MathJax-Span-240" style="font-family: MathJax_Math-italic;">δ<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.003em;"></span></span><span class="mi" id="MathJax-Span-241" style="font-family: MathJax_Math-italic;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span><span style="display: inline-block; width: 0px; height: 4.761em;"></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.913em;"></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.913em;"></span></span></span><span style="display: inline-block; overflow: hidden; vertical-align: -1.805em; border-left: 0px solid; width: 0px; height: 4.082em;"></span></span></nobr><span class="MJX_Assistive_MathML MJX_Assistive_MathML_Block" role="presentation"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mtable columnalign="right left right left right left right left right left right left" rowspacing="3pt" columnspacing="0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em" displaystyle="true"><mtr><mtd><mi>E</mi></mtd><mtd><mi></mi><mo>=</mo><mo fence="false" stretchy="false">‖</mo><mi>z</mi><mo>−</mo><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><msubsup><mo fence="false" stretchy="false">‖</mo><mrow class="MJX-TeXAtom-ORD"><mi>w</mi></mrow><mrow class="MJX-TeXAtom-ORD"><mn>2</mn></mrow></msubsup><mo>=</mo><mo fence="false" stretchy="false">‖</mo><mi>z</mi><mo>−</mo><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mo>+</mo><mi>J</mi><mi>δ</mi><mi>x</mi><msubsup><mo fence="false" stretchy="false">‖</mo><mrow class="MJX-TeXAtom-ORD"><mi>w</mi></mrow><mrow class="MJX-TeXAtom-ORD"><mn>2</mn></mrow></msubsup></mtd></mtr><mtr><mtd></mtd><mtd><mi></mi><mo>=</mo><mo stretchy="false">(</mo><mi>e</mi><mo>+</mo><mi>J</mi><mi>δ</mi><mi>x</mi><msup><mo stretchy="false">)</mo><mrow class="MJX-TeXAtom-ORD"><mi>T</mi></mrow></msup><mi>W</mi><mo stretchy="false">(</mo><mi>e</mi><mo>+</mo><mi>J</mi><mi>δ</mi><mi>x</mi><mo stretchy="false">)</mo></mtd></mtr><mtr><mtd></mtd><mtd><mi></mi><mo>=</mo><msup><mi>e</mi><mrow class="MJX-TeXAtom-ORD"><mi>T</mi></mrow></msup><mi>w</mi><mi>e</mi><mo>+</mo><mi>δ</mi><msup><mi>x</mi><mrow class="MJX-TeXAtom-ORD"><mi>T</mi></mrow></msup><msup><mi>J</mi><mrow class="MJX-TeXAtom-ORD"><mi>T</mi></mrow></msup><mi>W</mi><mi>e</mi><mo>+</mo><msup><mi>e</mi><mrow class="MJX-TeXAtom-ORD"><mi>T</mi></mrow></msup><mi>W</mi><mi>J</mi><mi>δ</mi><mi>x</mi><mo>+</mo><mi>δ</mi><msup><mi>x</mi><mrow class="MJX-TeXAtom-ORD"><mi>T</mi></mrow></msup><msup><mi>J</mi><mrow class="MJX-TeXAtom-ORD"><mi>T</mi></mrow></msup><mi>W</mi><mi>J</mi><mi>δ</mi><mi>x</mi></mtd></mtr></mtable></math></span></span></div><script type="math/tex; mode=display" id="MathJax-Element-2">\begin{aligned} E &=\|z-f(x)\|_{w}^{2}=\|z-f(x)+J \delta x\|_{w}^{2} \\ &=(e+J \delta x)^{T} W(e+J \delta x) \\ &=e^{T} w e+\delta x^{T} J^{T} W e+e^{T} W J \delta x+\delta x^{T} J^{T} W J \delta x \end{aligned}</script> 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 4.62677em; vertical-align: -2.06339em;"></span><span class="mord"><span class="mtable"><span class="col-align-r"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 2.56339em;"><span class="" style="top: -4.69928em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord mathdefault" style="margin-right: 0.05764em;">E</span></span></span><span class="" style="top: -3.14795em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"></span></span><span class="" style="top: -1.59661em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 2.06339em;"><span class=""></span></span></span></span></span><span class="col-align-l"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 2.56339em;"><span class="" style="top: -4.69928em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord">∥</span><span class="mord mathdefault" style="margin-right: 0.04398em;">z</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord mathdefault" style="margin-right: 0.10764em;">f</span><span class="mopen">(</span><span class="mord mathdefault">x</span><span class="mclose">)</span><span class="mord"><span class="mord">∥</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.864108em;"><span class="" style="top: -2.453em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.02691em;">w</span></span></span></span><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">2</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord">∥</span><span class="mord mathdefault" style="margin-right: 0.04398em;">z</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord mathdefault" style="margin-right: 0.10764em;">f</span><span class="mopen">(</span><span class="mord mathdefault">x</span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord mathdefault" style="margin-right: 0.09618em;">J</span><span class="mord mathdefault" style="margin-right: 0.03785em;">δ</span><span class="mord mathdefault">x</span><span class="mord"><span class="mord">∥</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.864108em;"><span class="" style="top: -2.453em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.02691em;">w</span></span></span></span><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">2</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span></span></span><span class="" style="top: -3.14795em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mopen">(</span><span class="mord mathdefault">e</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord mathdefault" style="margin-right: 0.09618em;">J</span><span class="mord mathdefault" style="margin-right: 0.03785em;">δ</span><span class="mord mathdefault">x</span><span class="mclose"><span class="mclose">)</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.891331em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.13889em;">T</span></span></span></span></span></span></span></span></span><span class="mord mathdefault" style="margin-right: 0.13889em;">W</span><span class="mopen">(</span><span class="mord mathdefault">e</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord mathdefault" style="margin-right: 0.09618em;">J</span><span class="mord mathdefault" style="margin-right: 0.03785em;">δ</span><span class="mord mathdefault">x</span><span class="mclose">)</span></span></span><span class="" style="top: -1.59661em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord"><span class="mord mathdefault">e</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.891331em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.13889em;">T</span></span></span></span></span></span></span></span></span><span class="mord mathdefault" style="margin-right: 0.02691em;">w</span><span class="mord mathdefault">e</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord mathdefault" style="margin-right: 0.03785em;">δ</span><span class="mord"><span class="mord mathdefault">x</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.891331em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.13889em;">T</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord mathdefault" style="margin-right: 0.09618em;">J</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.891331em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.13889em;">T</span></span></span></span></span></span></span></span></span><span class="mord mathdefault" style="margin-right: 0.13889em;">W</span><span class="mord mathdefault">e</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord"><span class="mord mathdefault">e</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.891331em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.13889em;">T</span></span></span></span></span></span></span></span></span><span class="mord mathdefault" style="margin-right: 0.13889em;">W</span><span class="mord mathdefault" style="margin-right: 0.09618em;">J</span><span class="mord mathdefault" style="margin-right: 0.03785em;">δ</span><span class="mord mathdefault">x</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord mathdefault" style="margin-right: 0.03785em;">δ</span><span class="mord"><span class="mord mathdefault">x</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.891331em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.13889em;">T</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord mathdefault" style="margin-right: 0.09618em;">J</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.891331em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.13889em;">T</span></span></span></span></span></span></span></span></span><span class="mord mathdefault" style="margin-right: 0.13889em;">W</span><span class="mord mathdefault" style="margin-right: 0.09618em;">J</span><span class="mord mathdefault" style="margin-right: 0.03785em;">δ</span><span class="mord mathdefault">x</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 2.06339em;"><span class=""></span></span></span></span></span></span></span></span></span></span></span></span><br> 其中 <span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
    
      J 
     
    
      T 
     
    
   
     W 
    
   
     J 
    
   
  
    J^{T} W J 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.841331em; vertical-align: 0em;"></span><span class="mord"><span class="mord mathdefault" style="margin-right: 0.09618em;">J</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.841331em;"><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.13889em;">T</span></span></span></span></span></span></span></span></span><span class="mord mathdefault" style="margin-right: 0.13889em;">W</span><span class="mord mathdefault" style="margin-right: 0.09618em;">J</span></span></span></span></span> 就称为 <span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
   
     H 
    
   
  
    \mathrm{H} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.68333em; vertical-align: 0em;"></span><span class="mord"><span class="mord mathrm">H</span></span></span></span></span></span> 矩阵。<br> Hessian矩阵其实说白了就是 <span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
   
     E 
    
   
  
    E 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.68333em; vertical-align: 0em;"></span><span class="mord mathdefault" style="margin-right: 0.05764em;">E</span></span></span></span></span> 对于状态变量 <span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
   
     x 
    
   
  
    x 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.43056em; vertical-align: 0em;"></span><span class="mord mathdefault">x</span></span></span></span></span> 的二阶偏导数。而H矩阵是对Hessian矩阵的近似, 主要是为了加速计算。</p>

1-2-2 Hessian或H矩阵和信息矩阵的关系

结论：在最大似然估计问题中，Hessian矩阵通常被用来表示Information矩阵。(结论所参考的资料如下)

[Why is the observed Fisher information defined as the Hessian of the log-likelihood?]
注意：我们的最小二乘就是建立在最大似然估计的基础上的，这也是为什么在一些SLAM框架中，直接将Hessian矩阵当做了信息矩阵。而在最大似然估计中，就是将Hessian矩阵近似为了信息矩阵

Maximum Likelihood Estimation (MLE).pdf
里面的公式(68)， The Information matrix is the negative of the expectation of the Hessian. 信息矩阵是Hessian期望的负值。

根据参考资料得出：

对于似然分布

       p 
      
     
       ( 
      
     
       y 
      
     
       ∣ 
      
     
       x 
      
     
       ) 
      
     
    
      p(y|x) 
     
    
  </span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord mathdefault">p</span><span class="mopen">(</span><span class="mord mathdefault" style="margin-right: 0.03588em;">y</span><span class="mord">∣</span><span class="mord mathdefault">x</span><span class="mclose">)</span></span></span></span></span>而言</strong>，Information矩阵就是负对数似然问题的Hessian矩阵的期望；</li><li><strong>对于分布<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
   
    
     
     
       p 
      
     
       ( 
      
     
       x 
      
     
       ) 
      
     
    
      p(x) 
     
    
  </span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord mathdefault">p</span><span class="mopen">(</span><span class="mord mathdefault">x</span><span class="mclose">)</span></span></span></span></span>而言</strong>，负对数似然问题的协方差矩阵的逆就是Hessian矩阵；</li></ol>

VINS-mono应该就是采用这样的思路，直接将hessian矩阵作为了信息矩阵(information)（原因写在下面了）

问题：信息矩阵=协方差的逆，但是为什么有的还说Hessian矩阵在高斯牛顿中被近似认为是信息矩阵？哪一个是对的？

答：都对，信息矩阵就是协方差的逆，这个不是非线性优化推导出的，是假设分布是高斯分布，计算最大后验估计得到的一个优化问题。（参考后面信息矩阵与最小二乘的关系，就会发现推导过程中，假设分布属于高斯分布）

2 📖 信息矩阵与最小二乘的关联

2-1 🔖 先谈一谈常规的最小二乘

残差函数

     f 
    
   
     ( 
    
   
     x 
    
   
     ) 
    
   
  
    \mathbf{f}(\mathbf{x}) 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord"><span class="mord mathbf" style="margin-right: 0.10903em;">f</span></span><span class="mopen">(</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mclose">)</span></span></span></span></span> 为非线性函数，对其一阶泰勒近似有:<br> <span class="katex--display"><span class="katex-display"><span class="katex"><span class="katex-mathml"> 
  
   
    
    
      f 
     
    
      ( 
     
    
      x 
     
    
      + 
     
    
      Δ 
     
    
      x 
     
    
      ) 
     
    
      ≈ 
     
    
      ℓ 
     
    
      ( 
     
    
      Δ 
     
    
      x 
     
    
      ) 
     
    
      ≡ 
     
    
      f 
     
    
      ( 
     
    
      x 
     
    
      ) 
     
    
      + 
     
    
      J 
     
    
      Δ 
     
    
      x 
     
    
   
     \mathbf{f}(\mathbf{x}+\Delta \mathbf{x}) \approx \ell(\Delta \mathbf{x}) \equiv \mathbf{f}(\mathbf{x})+\mathbf{J} \Delta \mathbf{x} 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord"><span class="mord mathbf" style="margin-right: 0.10903em;">f</span></span><span class="mopen">(</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.222222em;"></span></span><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord">Δ</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">≈</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord">ℓ</span><span class="mopen">(</span><span class="mord">Δ</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">≡</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord"><span class="mord mathbf" style="margin-right: 0.10903em;">f</span></span><span class="mopen">(</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.222222em;"></span></span><span class="base"><span class="strut" style="height: 0.68611em; vertical-align: 0em;"></span><span class="mord"><span class="mord mathbf">J</span></span><span class="mord">Δ</span><span class="mord"><span class="mord mathbf">x</span></span></span></span></span></span></span><br> <font color="blue">请特别注意, 这里的 <span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
  
   
    
    
      J 
     
    
   
     \mathrm{J} 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.68333em; vertical-align: 0em;"></span><span class="mord"><span class="mord mathrm">J</span></span></span></span></span></span> 是残差函数 <span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
  
   
    
    
      f 
     
    
   
     \mathrm{f} 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.69444em; vertical-align: 0em;"></span><span class="mord"><span class="mord mathrm" style="margin-right: 0.07778em;">f</span></span></span></span></span></span> 的雅克比矩阵</font>。代入损失函数：<br> <span class="katex--display"><span class="katex-display"><span class="katex"><span class="katex-mathml"> 
  
   
    
     
      
       
        
        
          F 
         
        
          ( 
         
        
          x 
         
        
          + 
         
        
          Δ 
         
        
          x 
         
        
          ) 
         
        
          ≈ 
         
        
          L 
         
        
          ( 
         
        
          Δ 
         
        
          x 
         
        
          ) 
         
        
       
      
      
       
        
         
        
          ≡ 
         
         
         
           1 
          
         
           2 
          
         
        
          ℓ 
         
        
          ( 
         
        
          Δ 
         
        
          x 
         
         
         
           ) 
          
         
           ⊤ 
          
         
        
          ℓ 
         
        
          ( 
         
        
          Δ 
         
        
          x 
         
        
          ) 
         
        
       
      
     
     
      
       
        
       
      
      
       
        
         
        
          = 
         
         
         
           1 
          
         
           2 
          
         
         
         
           f 
          
         
           ⊤ 
          
         
        
          f 
         
        
          + 
         
        
          Δ 
         
         
         
           x 
          
         
           ⊤ 
          
         
         
         
           J 
          
         
           ⊤ 
          
         
        
          f 
         
        
          + 
         
         
         
           1 
          
         
           2 
          
         
        
          Δ 
         
         
         
           x 
          
         
           ⊤ 
          
         
         
         
           J 
          
         
           ⊤ 
          
         
        
          J 
         
        
          Δ 
         
        
          x 
         
        
       
      
     
     
      
       
        
       
      
      
       
        
         
        
          = 
         
        
          F 
         
        
          ( 
         
        
          x 
         
        
          ) 
         
        
          + 
         
        
          Δ 
         
         
         
           x 
          
         
           ⊤ 
          
         
         
         
           J 
          
         
           ⊤ 
          
         
        
          f 
         
        
          + 
         
         
         
           1 
          
         
           2 
          
         
        
          Δ 
         
         
         
           x 
          
         
           ⊤ 
          
         
         
         
           J 
          
         
           ⊤ 
          
         
        
          J 
         
        
          Δ 
         
        
          x 
         
        
       
      
     
    
   
     <span class="MathJax_Preview" style="color: inherit; display: none;"></span><div class="MathJax_Display"><span class="MathJax MathJax_FullWidth" id="MathJax-Element-3-Frame" tabindex="0" style="position: relative;" data-mathml="<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot; display=&quot;block&quot;><mtable columnalign=&quot;right left right left right left right left right left right left&quot; rowspacing=&quot;3pt&quot; columnspacing=&quot;0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em&quot; displaystyle=&quot;true&quot;><mtr><mtd><mi>F</mi><mo stretchy=&quot;false&quot;>(</mo><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>x</mi></mrow><mo>+</mo><mi mathvariant=&quot;normal&quot;>&amp;#x0394;</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>x</mi></mrow><mo stretchy=&quot;false&quot;>)</mo><mo>&amp;#x2248;</mo><mi>L</mi><mo stretchy=&quot;false&quot;>(</mo><mi mathvariant=&quot;normal&quot;>&amp;#x0394;</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>x</mi></mrow><mo stretchy=&quot;false&quot;>)</mo></mtd><mtd><mi></mi><mo>&amp;#x2261;</mo><mfrac><mn>1</mn><mn>2</mn></mfrac><mi>&amp;#x2113;</mi><mo stretchy=&quot;false&quot;>(</mo><mi mathvariant=&quot;normal&quot;>&amp;#x0394;</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>x</mi></mrow><msup><mo stretchy=&quot;false&quot;>)</mo><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>&amp;#x22A4;</mi></mrow></msup><mi>&amp;#x2113;</mi><mo stretchy=&quot;false&quot;>(</mo><mi mathvariant=&quot;normal&quot;>&amp;#x0394;</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>x</mi></mrow><mo stretchy=&quot;false&quot;>)</mo></mtd></mtr><mtr><mtd /><mtd><mi></mi><mo>=</mo><mfrac><mn>1</mn><mn>2</mn></mfrac><msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>f</mi></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>&amp;#x22A4;</mi></mrow></msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>f</mi></mrow><mo>+</mo><mi mathvariant=&quot;normal&quot;>&amp;#x0394;</mi><msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>x</mi></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>&amp;#x22A4;</mi></mrow></msup><msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>J</mi></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>&amp;#x22A4;</mi></mrow></msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>f</mi></mrow><mo>+</mo><mfrac><mn>1</mn><mn>2</mn></mfrac><mi mathvariant=&quot;normal&quot;>&amp;#x0394;</mi><msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>x</mi></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>&amp;#x22A4;</mi></mrow></msup><msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>J</mi></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>&amp;#x22A4;</mi></mrow></msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>J</mi></mrow><mi mathvariant=&quot;normal&quot;>&amp;#x0394;</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>x</mi></mrow></mtd></mtr><mtr><mtd /><mtd><mi></mi><mo>=</mo><mi>F</mi><mo stretchy=&quot;false&quot;>(</mo><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>x</mi></mrow><mo stretchy=&quot;false&quot;>)</mo><mo>+</mo><mi mathvariant=&quot;normal&quot;>&amp;#x0394;</mi><msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>x</mi></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>&amp;#x22A4;</mi></mrow></msup><msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>J</mi></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>&amp;#x22A4;</mi></mrow></msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>f</mi></mrow><mo>+</mo><mfrac><mn>1</mn><mn>2</mn></mfrac><mi mathvariant=&quot;normal&quot;>&amp;#x0394;</mi><msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>x</mi></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>&amp;#x22A4;</mi></mrow></msup><msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>J</mi></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;normal&quot;>&amp;#x22A4;</mi></mrow></msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>J</mi></mrow><mi mathvariant=&quot;normal&quot;>&amp;#x0394;</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mi mathvariant=&quot;bold&quot;>x</mi></mrow></mtd></mtr></mtable></math>" role="presentation"><nobr aria-hidden="true"><span class="math" id="MathJax-Span-242" style="width: 100%; display: inline-block; min-width: 25.567em;"><span style="display: inline-block; position: relative; width: 100%; height: 0px; font-size: 102%;"><span style="position: absolute; clip: rect(3.951em, 1025.06em, 10.937em, -999.997em); top: -7.692em; left: 0em; width: 100%;"><span class="mrow" id="MathJax-Span-243"><span style="display: inline-block; position: relative; width: 100%; height: 0px;"><span style="position: absolute; clip: rect(3.951em, 1025.06em, 10.937em, -999.997em); top: -7.692em; left: 50%; margin-left: -12.552em;"><span class="mtable" id="MathJax-Span-244" style="padding-left: 0.154em;"><span style="display: inline-block; position: relative; width: 24.909em; height: 0px;"><span style="position: absolute; clip: rect(3.293em, 1008.86em, 9.013em, -999.997em); top: -6.427em; left: 0em;"><span style="display: inline-block; position: relative; width: 8.963em; height: 0px;"><span style="position: absolute; width: 100%; clip: rect(3.091em, 1008.86em, 4.407em, -999.997em); top: -6.224em; left: 0em;"><span class="mtd" id="MathJax-Span-245"><span class="mrow" id="MathJax-Span-246"><span style="display: inline-block; position: relative; width: 100%; height: 0px;"><span style="position: absolute; clip: rect(3.091em, 1008.86em, 4.407em, -999.997em); top: -3.997em; left: 50%; margin-left: -4.452em;"><span class="mi" id="MathJax-Span-247" style="font-family: MathJax_Math-italic;">F<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span class="mo" id="MathJax-Span-248" style="font-family: MathJax_Main;">(</span><span class="texatom" id="MathJax-Span-249"><span class="mrow" id="MathJax-Span-250"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-251" style="font-family: MathJax_Main-bold;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span class="mo" id="MathJax-Span-252" style="font-family: MathJax_Main; padding-left: 0.205em;">+</span><span class="mi" id="MathJax-Span-253" style="font-family: MathJax_Main; padding-left: 0.205em;">Δ</span><span class="texatom" id="MathJax-Span-254"><span class="mrow" id="MathJax-Span-255"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-256" style="font-family: MathJax_Main-bold;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span class="mo" id="MathJax-Span-257" style="font-family: MathJax_Main;">)</span><span class="mo" id="MathJax-Span-258" style="font-family: MathJax_Main; padding-left: 0.256em;">≈</span><span class="mi" id="MathJax-Span-259" style="font-family: MathJax_Math-italic; padding-left: 0.256em;">L</span><span class="mo" id="MathJax-Span-260" style="font-family: MathJax_Main;">(</span><span class="mi" id="MathJax-Span-261" style="font-family: MathJax_Main;">Δ</span><span class="texatom" id="MathJax-Span-262"><span class="mrow" id="MathJax-Span-263"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-264" style="font-family: MathJax_Main-bold;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span class="mo" id="MathJax-Span-265" style="font-family: MathJax_Main;">)</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.85em, 1000em, 4.154em, -999.997em); top: -3.895em; right: 0em;"><span class="mtd" id="MathJax-Span-291"><span class="mrow" id="MathJax-Span-292"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.85em, 1000em, 4.154em, -999.997em); top: -1.567em; right: 0em;"><span class="mtd" id="MathJax-Span-355"><span class="mrow" id="MathJax-Span-356"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span><span style="display: inline-block; width: 0px; height: 6.432em;"></span></span><span style="position: absolute; clip: rect(3.951em, 1015.95em, 10.937em, -999.997em); top: -7.692em; left: 8.963em;"><span style="display: inline-block; position: relative; width: 15.949em; height: 0px;"><span style="position: absolute; width: 100%; clip: rect(2.483em, 1007.95em, 4.862em, -999.997em); top: -6.224em; left: 0em;"><span class="mtd" id="MathJax-Span-266"><span class="mrow" id="MathJax-Span-267"><span style="display: inline-block; position: relative; width: 100%; height: 0px;"><span style="position: absolute; clip: rect(2.483em, 1007.95em, 4.862em, -999.997em); top: -3.997em; left: 50%; margin-left: -3.997em;"><span class="mi" id="MathJax-Span-268"></span><span class="mo" id="MathJax-Span-269" style="font-family: MathJax_Main; padding-left: 0.256em;">≡</span><span class="mfrac" id="MathJax-Span-270" style="padding-left: 0.256em;"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px; margin-right: 0.104em; margin-left: 0.104em;"><span style="position: absolute; clip: rect(3.192em, 1000.41em, 4.154em, -999.997em); top: -4.655em; left: 50%; margin-left: -0.251em;"><span class="mn" id="MathJax-Span-271" style="font-family: MathJax_Main;">1</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.192em, 1000.46em, 4.154em, -999.997em); top: -3.288em; left: 50%; margin-left: -0.251em;"><span class="mn" id="MathJax-Span-272" style="font-family: MathJax_Main;">2</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(0.863em, 1000.61em, 1.217em, -999.997em); top: -1.263em; left: 0em;"><span style="display: inline-block; overflow: hidden; vertical-align: 0em; border-top: 1.3px solid; width: 0.61em; height: 0px;"></span><span style="display: inline-block; width: 0px; height: 1.066em;"></span></span></span></span><span class="mi" id="MathJax-Span-273" style="font-family: MathJax_Main;">ℓ</span><span class="mo" id="MathJax-Span-274" style="font-family: MathJax_Main;">(</span><span class="mi" id="MathJax-Span-275" style="font-family: MathJax_Main;">Δ</span><span class="texatom" id="MathJax-Span-276"><span class="mrow" id="MathJax-Span-277"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-278" style="font-family: MathJax_Main-bold;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span class="msubsup" id="MathJax-Span-279"><span style="display: inline-block; position: relative; width: 1.015em; height: 0px;"><span style="position: absolute; clip: rect(3.091em, 1000.31em, 4.407em, -999.997em); top: -3.997em; left: 0em;"><span class="mo" id="MathJax-Span-280" style="font-family: MathJax_Main;">)</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.408em;"><span class="texatom" id="MathJax-Span-281"><span class="mrow" id="MathJax-Span-282"><span style="display: inline-block; position: relative; width: 0.559em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-283" style="font-size: 70.7%; font-family: MathJax_Main;">⊤</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="mi" id="MathJax-Span-284" style="font-family: MathJax_Main;">ℓ</span><span class="mo" id="MathJax-Span-285" style="font-family: MathJax_Main;">(</span><span class="mi" id="MathJax-Span-286" style="font-family: MathJax_Main;">Δ</span><span class="texatom" id="MathJax-Span-287"><span class="mrow" id="MathJax-Span-288"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-289" style="font-family: MathJax_Main-bold;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span class="mo" id="MathJax-Span-290" style="font-family: MathJax_Main;">)</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; width: 100%; clip: rect(2.483em, 1015.95em, 4.862em, -999.997em); top: -3.895em; left: 0em;"><span class="mtd" id="MathJax-Span-293"><span class="mrow" id="MathJax-Span-294"><span style="display: inline-block; position: relative; width: 100%; height: 0px;"><span style="position: absolute; clip: rect(2.483em, 1015.95em, 4.862em, -999.997em); top: -3.997em; left: 50%; margin-left: -7.996em;"><span class="mi" id="MathJax-Span-295"></span><span class="mo" id="MathJax-Span-296" style="font-family: MathJax_Main; padding-left: 0.256em;">=</span><span class="mfrac" id="MathJax-Span-297" style="padding-left: 0.256em;"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px; margin-right: 0.104em; margin-left: 0.104em;"><span style="position: absolute; clip: rect(3.192em, 1000.41em, 4.154em, -999.997em); top: -4.655em; left: 50%; margin-left: -0.251em;"><span class="mn" id="MathJax-Span-298" style="font-family: MathJax_Main;">1</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.192em, 1000.46em, 4.154em, -999.997em); top: -3.288em; left: 50%; margin-left: -0.251em;"><span class="mn" id="MathJax-Span-299" style="font-family: MathJax_Main;">2</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(0.863em, 1000.61em, 1.217em, -999.997em); top: -1.263em; left: 0em;"><span style="display: inline-block; overflow: hidden; vertical-align: 0em; border-top: 1.3px solid; width: 0.61em; height: 0px;"></span><span style="display: inline-block; width: 0px; height: 1.066em;"></span></span></span></span><span class="msubsup" id="MathJax-Span-300"><span style="display: inline-block; position: relative; width: 1.066em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.46em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="texatom" id="MathJax-Span-301"><span class="mrow" id="MathJax-Span-302"><span style="display: inline-block; position: relative; width: 0.458em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.46em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-303" style="font-family: MathJax_Main-bold;">f<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.458em;"><span class="texatom" id="MathJax-Span-304"><span class="mrow" id="MathJax-Span-305"><span style="display: inline-block; position: relative; width: 0.559em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-306" style="font-size: 70.7%; font-family: MathJax_Main;">⊤</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="texatom" id="MathJax-Span-307"><span class="mrow" id="MathJax-Span-308"><span style="display: inline-block; position: relative; width: 0.458em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.46em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-309" style="font-family: MathJax_Main-bold;">f<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span class="mo" id="MathJax-Span-310" style="font-family: MathJax_Main; padding-left: 0.205em;">+</span><span class="mi" id="MathJax-Span-311" style="font-family: MathJax_Main; padding-left: 0.205em;">Δ</span><span class="msubsup" id="MathJax-Span-312"><span style="display: inline-block; position: relative; width: 1.217em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="texatom" id="MathJax-Span-313"><span class="mrow" id="MathJax-Span-314"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-315" style="font-family: MathJax_Main-bold;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.61em;"><span class="texatom" id="MathJax-Span-316"><span class="mrow" id="MathJax-Span-317"><span style="display: inline-block; position: relative; width: 0.559em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-318" style="font-size: 70.7%; font-family: MathJax_Main;">⊤</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="msubsup" id="MathJax-Span-319"><span style="display: inline-block; position: relative; width: 1.217em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="texatom" id="MathJax-Span-320"><span class="mrow" id="MathJax-Span-321"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-322" style="font-family: MathJax_Main-bold;">J</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.61em;"><span class="texatom" id="MathJax-Span-323"><span class="mrow" id="MathJax-Span-324"><span style="display: inline-block; position: relative; width: 0.559em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-325" style="font-size: 70.7%; font-family: MathJax_Main;">⊤</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="texatom" id="MathJax-Span-326"><span class="mrow" id="MathJax-Span-327"><span style="display: inline-block; position: relative; width: 0.458em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.46em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-328" style="font-family: MathJax_Main-bold;">f<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span class="mo" id="MathJax-Span-329" style="font-family: MathJax_Main; padding-left: 0.205em;">+</span><span class="mfrac" id="MathJax-Span-330" style="padding-left: 0.205em;"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px; margin-right: 0.104em; margin-left: 0.104em;"><span style="position: absolute; clip: rect(3.192em, 1000.41em, 4.154em, -999.997em); top: -4.655em; left: 50%; margin-left: -0.251em;"><span class="mn" id="MathJax-Span-331" style="font-family: MathJax_Main;">1</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.192em, 1000.46em, 4.154em, -999.997em); top: -3.288em; left: 50%; margin-left: -0.251em;"><span class="mn" id="MathJax-Span-332" style="font-family: MathJax_Main;">2</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(0.863em, 1000.61em, 1.217em, -999.997em); top: -1.263em; left: 0em;"><span style="display: inline-block; overflow: hidden; vertical-align: 0em; border-top: 1.3px solid; width: 0.61em; height: 0px;"></span><span style="display: inline-block; width: 0px; height: 1.066em;"></span></span></span></span><span class="mi" id="MathJax-Span-333" style="font-family: MathJax_Main;">Δ</span><span class="msubsup" id="MathJax-Span-334"><span style="display: inline-block; position: relative; width: 1.217em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="texatom" id="MathJax-Span-335"><span class="mrow" id="MathJax-Span-336"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-337" style="font-family: MathJax_Main-bold;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.61em;"><span class="texatom" id="MathJax-Span-338"><span class="mrow" id="MathJax-Span-339"><span style="display: inline-block; position: relative; width: 0.559em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-340" style="font-size: 70.7%; font-family: MathJax_Main;">⊤</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="msubsup" id="MathJax-Span-341"><span style="display: inline-block; position: relative; width: 1.217em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="texatom" id="MathJax-Span-342"><span class="mrow" id="MathJax-Span-343"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-344" style="font-family: MathJax_Main-bold;">J</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.61em;"><span class="texatom" id="MathJax-Span-345"><span class="mrow" id="MathJax-Span-346"><span style="display: inline-block; position: relative; width: 0.559em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-347" style="font-size: 70.7%; font-family: MathJax_Main;">⊤</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="texatom" id="MathJax-Span-348"><span class="mrow" id="MathJax-Span-349"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-350" style="font-family: MathJax_Main-bold;">J</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span class="mi" id="MathJax-Span-351" style="font-family: MathJax_Main;">Δ</span><span class="texatom" id="MathJax-Span-352"><span class="mrow" id="MathJax-Span-353"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-354" style="font-family: MathJax_Main-bold;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; width: 100%; clip: rect(2.483em, 1015.7em, 4.862em, -999.997em); top: -1.567em; left: 0em;"><span class="mtd" id="MathJax-Span-357"><span class="mrow" id="MathJax-Span-358"><span style="display: inline-block; position: relative; width: 100%; height: 0px;"><span style="position: absolute; clip: rect(2.483em, 1015.7em, 4.862em, -999.997em); top: -3.997em; left: 50%; margin-left: -7.844em;"><span class="mi" id="MathJax-Span-359"></span><span class="mo" id="MathJax-Span-360" style="font-family: MathJax_Main; padding-left: 0.256em;">=</span><span class="mi" id="MathJax-Span-361" style="font-family: MathJax_Math-italic; padding-left: 0.256em;">F<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span class="mo" id="MathJax-Span-362" style="font-family: MathJax_Main;">(</span><span class="texatom" id="MathJax-Span-363"><span class="mrow" id="MathJax-Span-364"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-365" style="font-family: MathJax_Main-bold;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span class="mo" id="MathJax-Span-366" style="font-family: MathJax_Main;">)</span><span class="mo" id="MathJax-Span-367" style="font-family: MathJax_Main; padding-left: 0.205em;">+</span><span class="mi" id="MathJax-Span-368" style="font-family: MathJax_Main; padding-left: 0.205em;">Δ</span><span class="msubsup" id="MathJax-Span-369"><span style="display: inline-block; position: relative; width: 1.217em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="texatom" id="MathJax-Span-370"><span class="mrow" id="MathJax-Span-371"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-372" style="font-family: MathJax_Main-bold;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.61em;"><span class="texatom" id="MathJax-Span-373"><span class="mrow" id="MathJax-Span-374"><span style="display: inline-block; position: relative; width: 0.559em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-375" style="font-size: 70.7%; font-family: MathJax_Main;">⊤</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="msubsup" id="MathJax-Span-376"><span style="display: inline-block; position: relative; width: 1.217em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="texatom" id="MathJax-Span-377"><span class="mrow" id="MathJax-Span-378"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-379" style="font-family: MathJax_Main-bold;">J</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.61em;"><span class="texatom" id="MathJax-Span-380"><span class="mrow" id="MathJax-Span-381"><span style="display: inline-block; position: relative; width: 0.559em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-382" style="font-size: 70.7%; font-family: MathJax_Main;">⊤</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="texatom" id="MathJax-Span-383"><span class="mrow" id="MathJax-Span-384"><span style="display: inline-block; position: relative; width: 0.458em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.46em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-385" style="font-family: MathJax_Main-bold;">f<span style="display: inline-block; overflow: hidden; height: 1px; width: 0.104em;"></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span class="mo" id="MathJax-Span-386" style="font-family: MathJax_Main; padding-left: 0.205em;">+</span><span class="mfrac" id="MathJax-Span-387" style="padding-left: 0.205em;"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px; margin-right: 0.104em; margin-left: 0.104em;"><span style="position: absolute; clip: rect(3.192em, 1000.41em, 4.154em, -999.997em); top: -4.655em; left: 50%; margin-left: -0.251em;"><span class="mn" id="MathJax-Span-388" style="font-family: MathJax_Main;">1</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(3.192em, 1000.46em, 4.154em, -999.997em); top: -3.288em; left: 50%; margin-left: -0.251em;"><span class="mn" id="MathJax-Span-389" style="font-family: MathJax_Main;">2</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; clip: rect(0.863em, 1000.61em, 1.217em, -999.997em); top: -1.263em; left: 0em;"><span style="display: inline-block; overflow: hidden; vertical-align: 0em; border-top: 1.3px solid; width: 0.61em; height: 0px;"></span><span style="display: inline-block; width: 0px; height: 1.066em;"></span></span></span></span><span class="mi" id="MathJax-Span-390" style="font-family: MathJax_Main;">Δ</span><span class="msubsup" id="MathJax-Span-391"><span style="display: inline-block; position: relative; width: 1.217em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="texatom" id="MathJax-Span-392"><span class="mrow" id="MathJax-Span-393"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-394" style="font-family: MathJax_Main-bold;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.61em;"><span class="texatom" id="MathJax-Span-395"><span class="mrow" id="MathJax-Span-396"><span style="display: inline-block; position: relative; width: 0.559em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-397" style="font-size: 70.7%; font-family: MathJax_Main;">⊤</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="msubsup" id="MathJax-Span-398"><span style="display: inline-block; position: relative; width: 1.217em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="texatom" id="MathJax-Span-399"><span class="mrow" id="MathJax-Span-400"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-401" style="font-family: MathJax_Main-bold;">J</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span><span style="position: absolute; top: -4.402em; left: 0.61em;"><span class="texatom" id="MathJax-Span-402"><span class="mrow" id="MathJax-Span-403"><span style="display: inline-block; position: relative; width: 0.559em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-404" style="font-size: 70.7%; font-family: MathJax_Main;">⊤</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span><span class="texatom" id="MathJax-Span-405"><span class="mrow" id="MathJax-Span-406"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.141em, 1000.51em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-407" style="font-family: MathJax_Main-bold;">J</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span class="mi" id="MathJax-Span-408" style="font-family: MathJax_Main;">Δ</span><span class="texatom" id="MathJax-Span-409"><span class="mrow" id="MathJax-Span-410"><span style="display: inline-block; position: relative; width: 0.61em; height: 0px;"><span style="position: absolute; clip: rect(3.394em, 1000.61em, 4.154em, -999.997em); top: -3.997em; left: 0em;"><span class="mi" id="MathJax-Span-411" style="font-family: MathJax_Main-bold;">x</span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span></span></span><span style="display: inline-block; width: 0px; height: 4.002em;"></span></span></span><span style="display: inline-block; width: 0px; height: 7.697em;"></span></span></span></span><span style="display: inline-block; width: 0px; height: 7.697em;"></span></span></span></span><span style="display: inline-block; width: 0px; height: 7.697em;"></span></span></span><span style="display: inline-block; overflow: hidden; vertical-align: -3.199em; border-left: 0px solid; width: 0px; height: 6.922em;"></span></span></nobr><span class="MJX_Assistive_MathML MJX_Assistive_MathML_Block" role="presentation"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><mtable columnalign="right left right left right left right left right left right left" rowspacing="3pt" columnspacing="0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em" displaystyle="true"><mtr><mtd><mi>F</mi><mo stretchy="false">(</mo><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">x</mi></mrow><mo>+</mo><mi mathvariant="normal">Δ</mi><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">x</mi></mrow><mo stretchy="false">)</mo><mo>≈</mo><mi>L</mi><mo stretchy="false">(</mo><mi mathvariant="normal">Δ</mi><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">x</mi></mrow><mo stretchy="false">)</mo></mtd><mtd><mi></mi><mo>≡</mo><mfrac><mn>1</mn><mn>2</mn></mfrac><mi>ℓ</mi><mo stretchy="false">(</mo><mi mathvariant="normal">Δ</mi><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">x</mi></mrow><msup><mo stretchy="false">)</mo><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">⊤</mi></mrow></msup><mi>ℓ</mi><mo stretchy="false">(</mo><mi mathvariant="normal">Δ</mi><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">x</mi></mrow><mo stretchy="false">)</mo></mtd></mtr><mtr><mtd></mtd><mtd><mi></mi><mo>=</mo><mfrac><mn>1</mn><mn>2</mn></mfrac><msup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">f</mi></mrow><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">⊤</mi></mrow></msup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">f</mi></mrow><mo>+</mo><mi mathvariant="normal">Δ</mi><msup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">x</mi></mrow><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">⊤</mi></mrow></msup><msup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">J</mi></mrow><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">⊤</mi></mrow></msup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">f</mi></mrow><mo>+</mo><mfrac><mn>1</mn><mn>2</mn></mfrac><mi mathvariant="normal">Δ</mi><msup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">x</mi></mrow><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">⊤</mi></mrow></msup><msup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">J</mi></mrow><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">⊤</mi></mrow></msup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">J</mi></mrow><mi mathvariant="normal">Δ</mi><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">x</mi></mrow></mtd></mtr><mtr><mtd></mtd><mtd><mi></mi><mo>=</mo><mi>F</mi><mo stretchy="false">(</mo><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">x</mi></mrow><mo stretchy="false">)</mo><mo>+</mo><mi mathvariant="normal">Δ</mi><msup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">x</mi></mrow><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">⊤</mi></mrow></msup><msup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">J</mi></mrow><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">⊤</mi></mrow></msup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">f</mi></mrow><mo>+</mo><mfrac><mn>1</mn><mn>2</mn></mfrac><mi mathvariant="normal">Δ</mi><msup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">x</mi></mrow><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">⊤</mi></mrow></msup><msup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">J</mi></mrow><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="normal">⊤</mi></mrow></msup><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">J</mi></mrow><mi mathvariant="normal">Δ</mi><mrow class="MJX-TeXAtom-ORD"><mi mathvariant="bold">x</mi></mrow></mtd></mtr></mtable></math></span></span></div><script type="math/tex; mode=display" id="MathJax-Element-3">\begin{aligned} F(\mathbf{x}+\Delta \mathbf{x}) \approx L(\Delta \mathbf{x}) & \equiv \frac{1}{2} \ell(\Delta \mathbf{x})^{\top} \ell(\Delta \mathbf{x}) \\ &=\frac{1}{2} \mathbf{f}^{\top} \mathbf{f}+\Delta \mathbf{x}^{\top} \mathbf{J}^{\top} \mathbf{f}+\frac{1}{2} \Delta \mathbf{x}^{\top} \mathbf{J}^{\top} \mathbf{J} \Delta \mathbf{x} \\ &=F(\mathbf{x})+\Delta \mathbf{x}^{\top} \mathbf{J}^{\top} \mathbf{f}+\frac{1}{2} \Delta \mathbf{x}^{\top} \mathbf{J}^{\top} \mathbf{J} \Delta \mathbf{x} \end{aligned}</script> 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 6.92232em; vertical-align: -3.21116em;"></span><span class="mord"><span class="mtable"><span class="col-align-r"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 3.71116em;"><span class="" style="top: -5.71116em;"><span class="pstrut" style="height: 3.32144em;"></span><span class="mord"><span class="mord mathdefault" style="margin-right: 0.13889em;">F</span><span class="mopen">(</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord">Δ</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">≈</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord mathdefault">L</span><span class="mopen">(</span><span class="mord">Δ</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mclose">)</span></span></span><span class="" style="top: -3.40372em;"><span class="pstrut" style="height: 3.32144em;"></span><span class="mord"></span></span><span class="" style="top: -1.09628em;"><span class="pstrut" style="height: 3.32144em;"></span><span class="mord"></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 3.21116em;"><span class=""></span></span></span></span></span><span class="col-align-l"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 3.71116em;"><span class="" style="top: -5.71116em;"><span class="pstrut" style="height: 3.32144em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">≡</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.32144em;"><span class="" style="top: -2.314em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">2</span></span></span><span class="" style="top: -3.23em;"><span class="pstrut" style="height: 3em;"></span><span class="frac-line" style="border-bottom-width: 0.04em;"></span></span><span class="" style="top: -3.677em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.686em;"><span class=""></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mord">ℓ</span><span class="mopen">(</span><span class="mord">Δ</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mclose"><span class="mclose">)</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.899108em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">⊤</span></span></span></span></span></span></span></span></span><span class="mord">ℓ</span><span class="mopen">(</span><span class="mord">Δ</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mclose">)</span></span></span><span class="" style="top: -3.40372em;"><span class="pstrut" style="height: 3.32144em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.32144em;"><span class="" style="top: -2.314em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">2</span></span></span><span class="" style="top: -3.23em;"><span class="pstrut" style="height: 3em;"></span><span class="frac-line" style="border-bottom-width: 0.04em;"></span></span><span class="" style="top: -3.677em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.686em;"><span class=""></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mord"><span class="mord"><span class="mord mathbf" style="margin-right: 0.10903em;">f</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.899108em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">⊤</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord mathbf" style="margin-right: 0.10903em;">f</span></span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord">Δ</span><span class="mord"><span class="mord"><span class="mord mathbf">x</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.899108em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">⊤</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord mathbf">J</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.899108em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">⊤</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord mathbf" style="margin-right: 0.10903em;">f</span></span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.32144em;"><span class="" style="top: -2.314em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">2</span></span></span><span class="" style="top: -3.23em;"><span class="pstrut" style="height: 3em;"></span><span class="frac-line" style="border-bottom-width: 0.04em;"></span></span><span class="" style="top: -3.677em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.686em;"><span class=""></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mord">Δ</span><span class="mord"><span class="mord"><span class="mord mathbf">x</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.899108em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">⊤</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord mathbf">J</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.899108em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">⊤</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord mathbf">J</span></span><span class="mord">Δ</span><span class="mord"><span class="mord mathbf">x</span></span></span></span><span class="" style="top: -1.09628em;"><span class="pstrut" style="height: 3.32144em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord mathdefault" style="margin-right: 0.13889em;">F</span><span class="mopen">(</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord">Δ</span><span class="mord"><span class="mord"><span class="mord mathbf">x</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.899108em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">⊤</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord mathbf">J</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.899108em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">⊤</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord mathbf" style="margin-right: 0.10903em;">f</span></span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.32144em;"><span class="" style="top: -2.314em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">2</span></span></span><span class="" style="top: -3.23em;"><span class="pstrut" style="height: 3em;"></span><span class="frac-line" style="border-bottom-width: 0.04em;"></span></span><span class="" style="top: -3.677em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.686em;"><span class=""></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mord">Δ</span><span class="mord"><span class="mord"><span class="mord mathbf">x</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.899108em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">⊤</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord mathbf">J</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.899108em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">⊤</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord mathbf">J</span></span><span class="mord">Δ</span><span class="mord"><span class="mord mathbf">x</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 3.21116em;"><span class=""></span></span></span></span></span></span></span></span></span></span></span></span><br> 这样损失函数就近似成了一个二次函数，并且如果雅克比是满秩的， 则 <span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
    
      J 
     
    
      ⊤ 
     
    
   
     J 
    
   
  
    \mathbf{J}^{\top} \mathbf{J} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.849108em; vertical-align: 0em;"></span><span class="mord"><span class="mord"><span class="mord mathbf">J</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.849108em;"><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">⊤</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord mathbf">J</span></span></span></span></span></span> 正定，损失函数有最小值。</p>

另外, 易得：

      F 
     
    
      ′ 
     
    
   
     ( 
    
   
     x 
    
   
     ) 
    
   
     = 
    
    
     
     
       ( 
      
      
      
        J 
       
      
        ⊤ 
       
      
     
       f 
      
     
       ) 
      
     
    
      ⊤ 
     
    
   
  
    F^{\prime}(\mathbf{x})=\left(\mathbf{J}^{\top} \mathbf{f}\right)^{\top} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 1.00189em; vertical-align: -0.25em;"></span><span class="mord"><span class="mord mathdefault" style="margin-right: 0.13889em;">F</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.751892em;"><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">′</span></span></span></span></span></span></span></span></span><span class="mopen">(</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 1.43902em; vertical-align: -0.35001em;"></span><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size1">(</span></span><span class="mord"><span class="mord"><span class="mord mathbf">J</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.849108em;"><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">⊤</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord mathbf" style="margin-right: 0.10903em;">f</span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size1">)</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 1.08901em;"><span class="" style="top: -3.3029em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">⊤</span></span></span></span></span></span></span></span></span></span></span></span></span>, 以及 <span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
    
      F 
     
     
     
       ′ 
      
     
       ′ 
      
     
    
   
     ( 
    
   
     x 
    
   
     ) 
    
   
     ≈ 
    
    
    
      J 
     
    
      ⊤ 
     
    
   
     J 
    
   
  
    F^{\prime \prime}(\mathbf{x}) \approx \mathbf{J}^{\top} \mathbf{J} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 1.00189em; vertical-align: -0.25em;"></span><span class="mord"><span class="mord mathdefault" style="margin-right: 0.13889em;">F</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.751892em;"><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">′</span><span class="mord mtight">′</span></span></span></span></span></span></span></span></span><span class="mopen">(</span><span class="mord"><span class="mord mathbf">x</span></span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">≈</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 0.849108em; vertical-align: 0em;"></span><span class="mord"><span class="mord"><span class="mord mathbf">J</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.849108em;"><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">⊤</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord mathbf">J</span></span></span></span></span></span>.</p>

2-2 🔖 包含信息矩阵的最小二乘

本小节主要回答下面问题：

有时候我们不写中间的协方差的逆，为什么？？多出来的协方差的逆是怎么多出来的？？

没有写那是假设其为1了。

SLAM问题建模

考虑某个状态

     ξ 
    
   
  
    \boldsymbol{\xi} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.88888em; vertical-align: -0.19444em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span></span></span></span></span>, 以及一次与该变量相关的观测 <span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
    
      r 
     
     
     
       i 
      
     
       ∘ 
      
     
    
   
  
    \mathbf{r}_{i \circ} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.59444em; vertical-align: -0.15em;"></span><span class="mord"><span class="mord"><span class="mord mathbf">r</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.311664em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span><span class="mord mtight">∘</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span></span></span></span></span> 由于噪声的存在, 观测服从概率分布 <span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
   
     p 
    
    
    
      ( 
     
     
     
       r 
      
     
       i 
      
     
    
      ∣ 
     
    
      ξ 
     
    
      ) 
     
    
   
  
    p\left(\mathbf{r}_{i} \mid \boldsymbol{\xi}\right) 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord mathdefault">p</span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord"><span class="mord"><span class="mord mathbf">r</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.311664em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">∣</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mclose delimcenter" style="top: 0em;">)</span></span></span></span></span></span> 。 多次观测时，各个测量值相互独立, 则多个测量相互独立，则多个测量 <span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
   
     r 
    
   
     = 
    
    
     
     
       ( 
      
      
      
        r 
       
      
        1 
       
      
     
       , 
      
     
       … 
      
     
       , 
      
      
      
        r 
       
      
        n 
       
      
     
       ) 
      
     
    
      ⊤ 
     
    
   
  
    \mathbf{r}=\left(\mathbf{r}_{1}, \ldots, \mathbf{r}_{n}\right)^{\top} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.44444em; vertical-align: 0em;"></span><span class="mord"><span class="mord mathbf">r</span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 1.23901em; vertical-align: -0.25em;"></span><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord"><span class="mord"><span class="mord mathbf">r</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.301108em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="minner">…</span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mord"><span class="mord"><span class="mord mathbf">r</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.151392em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">n</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;">)</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.989008em;"><span class="" style="top: -3.2029em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">⊤</span></span></span></span></span></span></span></span></span></span></span></span></span> 构建的似然概率为：<br> <span class="katex--display"><span class="katex-display"><span class="katex"><span class="katex-mathml"> 
  
   
    
    
      p 
     
    
      ( 
     
    
      r 
     
    
      ∣ 
     
    
      ξ 
     
    
      ) 
     
    
      = 
     
     
     
       ∏ 
      
     
       i 
      
     
    
      p 
     
     
     
       ( 
      
      
      
        r 
       
      
        i 
       
      
     
       ∣ 
      
     
       ξ 
      
     
       ) 
      
     
    
   
     p(\mathbf{r} \mid \boldsymbol{\xi})=\prod_{i} p\left(\mathbf{r}_{i} \mid \boldsymbol{\xi}\right) 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord mathdefault">p</span><span class="mopen">(</span><span class="mord"><span class="mord mathbf">r</span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">∣</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 2.32767em; vertical-align: -1.27767em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.05001em;"><span class="" style="top: -1.87233em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span><span class="" style="top: -3.05001em;"><span class="pstrut" style="height: 3.05em;"></span><span class=""><span class="mop op-symbol large-op">∏</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 1.27767em;"><span class=""></span></span></span></span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mord mathdefault">p</span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord"><span class="mord"><span class="mord mathbf">r</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.311664em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">∣</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mclose delimcenter" style="top: 0em;">)</span></span></span></span></span></span></span><br> 如果知道机器人状态的先验信息 <span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
   
     p 
    
   
     ( 
    
   
     ξ 
    
   
     ) 
    
   
  
    p(\boldsymbol{\xi}) 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord mathdefault">p</span><span class="mopen">(</span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mclose">)</span></span></span></span></span>, 如 GPS, 车轮码盘信息等, 则 根据 Bayes 法则，有后验概率：<br> <span class="katex--display"><span class="katex-display"><span class="katex"><span class="katex-mathml"> 
  
   
    
    
      p 
     
    
      ( 
     
    
      ξ 
     
    
      ∣ 
     
    
      r 
     
    
      ) 
     
    
      = 
     
     
      
      
        p 
       
      
        ( 
       
      
        r 
       
      
        ∣ 
       
      
        ξ 
       
      
        ) 
       
      
        p 
       
      
        ( 
       
      
        ξ 
       
      
        ) 
       
      
      
      
        p 
       
      
        ( 
       
      
        r 
       
      
        ) 
       
      
     
    
   
     p(\boldsymbol{\xi} \mid \mathbf{r})=\frac{p(\mathbf{r} \mid \boldsymbol{\xi}) p(\boldsymbol{\xi})}{p(\mathbf{r})} 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord mathdefault">p</span><span class="mopen">(</span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">∣</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord"><span class="mord mathbf">r</span></span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 2.363em; vertical-align: -0.936em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.427em;"><span class="" style="top: -2.314em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord mathdefault">p</span><span class="mopen">(</span><span class="mord"><span class="mord mathbf">r</span></span><span class="mclose">)</span></span></span><span class="" style="top: -3.23em;"><span class="pstrut" style="height: 3em;"></span><span class="frac-line" style="border-bottom-width: 0.04em;"></span></span><span class="" style="top: -3.677em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord mathdefault">p</span><span class="mopen">(</span><span class="mord"><span class="mord mathbf">r</span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">∣</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mclose">)</span><span class="mord mathdefault">p</span><span class="mopen">(</span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mclose">)</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.936em;"><span class=""></span></span></span></span></span><span class="mclose nulldelimiter"></span></span></span></span></span></span></span><br> 通过最大后验估计，获得系统状态的最优估计：<br> <span class="katex--display"><span class="katex-display"><span class="katex"><span class="katex-mathml"> 
  
   
    
     
     
       ξ 
      
      
      
        M 
       
      
        A 
       
      
        P 
       
      
     
    
      = 
     
    
      arg 
     
    
      ⁡ 
     
     
      
      
        max 
       
      
        ⁡ 
       
      
     
       ξ 
      
     
    
      p 
     
    
      ( 
     
    
      ξ 
     
    
      ∣ 
     
    
      r 
     
    
      ) 
     
    
   
     \boldsymbol{\xi}_{\mathrm{MAP}}=\arg \max _{\xi} p(\boldsymbol{\xi} \mid \mathbf{r}) 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.93858em; vertical-align: -0.24414em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.234191em;"><span class="" style="top: -2.45586em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mathrm mtight">M</span><span class="mord mathrm mtight">A</span><span class="mord mathrm mtight">P</span></span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.24414em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 1.63822em; vertical-align: -0.888216em;"></span><span class="mop">ar<span style="margin-right: 0.01389em;">g</span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.43056em;"><span class="" style="top: -2.04789em; margin-left: 0em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.04601em;">ξ</span></span></span></span><span class="" style="top: -2.7em;"><span class="pstrut" style="height: 2.7em;"></span><span class=""><span class="mop">max</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.888216em;"><span class=""></span></span></span></span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mord mathdefault">p</span><span class="mopen">(</span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">∣</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord"><span class="mord mathbf">r</span></span><span class="mclose">)</span></span></span></span></span></span><br> 后验公式中分母跟状态量无关，舍弃。最大后验变成了：<br> <span class="katex--display"><span class="katex-display"><span class="katex"><span class="katex-mathml"> 
  
   
    
     
     
       ξ 
      
      
      
        M 
       
      
        A 
       
      
        P 
       
      
     
    
      = 
     
    
      arg 
     
    
      ⁡ 
     
     
      
      
        max 
       
      
        ⁡ 
       
      
     
       ξ 
      
     
     
     
       ∏ 
      
     
       i 
      
     
    
      p 
     
     
     
       ( 
      
      
      
        r 
       
      
        i 
       
      
     
       ∣ 
      
     
       ξ 
      
     
       ) 
      
     
    
      p 
     
    
      ( 
     
    
      ξ 
     
    
      ) 
     
    
   
     \boldsymbol{\xi}_{\mathrm{MAP}}=\arg \max _{\xi} \prod_{i} p\left(\mathbf{r}_{i} \mid \boldsymbol{\xi}\right) p(\boldsymbol{\xi}) 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.93858em; vertical-align: -0.24414em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.234191em;"><span class="" style="top: -2.45586em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mathrm mtight">M</span><span class="mord mathrm mtight">A</span><span class="mord mathrm mtight">P</span></span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.24414em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 2.32767em; vertical-align: -1.27767em;"></span><span class="mop">ar<span style="margin-right: 0.01389em;">g</span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.43056em;"><span class="" style="top: -2.04789em; margin-left: 0em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.04601em;">ξ</span></span></span></span><span class="" style="top: -2.7em;"><span class="pstrut" style="height: 2.7em;"></span><span class=""><span class="mop">max</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.888216em;"><span class=""></span></span></span></span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.05001em;"><span class="" style="top: -1.87233em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span><span class="" style="top: -3.05001em;"><span class="pstrut" style="height: 3.05em;"></span><span class=""><span class="mop op-symbol large-op">∏</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 1.27767em;"><span class=""></span></span></span></span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mord mathdefault">p</span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord"><span class="mord"><span class="mord mathbf">r</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.311664em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">∣</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mclose delimcenter" style="top: 0em;">)</span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mord mathdefault">p</span><span class="mopen">(</span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mclose">)</span></span></span></span></span></span><br> 即<br> <span class="katex--display"><span class="katex-display"><span class="katex"><span class="katex-mathml"> 
  
   
    
     
     
       ξ 
      
      
      
        M 
       
      
        A 
       
      
        P 
       
      
     
    
      = 
     
    
      arg 
     
    
      ⁡ 
     
     
      
      
        min 
       
      
        ⁡ 
       
      
     
       ξ 
      
     
     
     
       [ 
      
     
       − 
      
      
      
        ∑ 
       
      
        i 
       
      
     
       log 
      
     
       ⁡ 
      
     
       p 
      
      
      
        ( 
       
       
       
         r 
        
       
         i 
        
       
      
        ∣ 
       
      
        ξ 
       
      
        ) 
       
      
     
       − 
      
     
       log 
      
     
       ⁡ 
      
     
       p 
      
     
       ( 
      
     
       ξ 
      
     
       ) 
      
     
       ] 
      
     
    
   
     \boldsymbol{\xi}_{\mathrm{MAP}}=\arg \min _{\xi}\left[-\sum_{i} \log p\left(\mathbf{r}_{i} \mid \boldsymbol{\xi}\right)-\log p(\boldsymbol{\xi})\right] 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.93858em; vertical-align: -0.24414em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.234191em;"><span class="" style="top: -2.45586em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mathrm mtight">M</span><span class="mord mathrm mtight">A</span><span class="mord mathrm mtight">P</span></span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.24414em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 3.02767em; vertical-align: -1.27767em;"></span><span class="mop">ar<span style="margin-right: 0.01389em;">g</span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.66786em;"><span class="" style="top: -2.04789em; margin-left: 0em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.04601em;">ξ</span></span></span></span><span class="" style="top: -2.7em;"><span class="pstrut" style="height: 2.7em;"></span><span class=""><span class="mop">min</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.888216em;"><span class=""></span></span></span></span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size4">[</span></span><span class="mord">−</span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.05001em;"><span class="" style="top: -1.87233em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span><span class="" style="top: -3.05001em;"><span class="pstrut" style="height: 3.05em;"></span><span class=""><span class="mop op-symbol large-op">∑</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 1.27767em;"><span class=""></span></span></span></span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mop">lo<span style="margin-right: 0.01389em;">g</span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mord mathdefault">p</span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord"><span class="mord"><span class="mord mathbf">r</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.311664em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">∣</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mclose delimcenter" style="top: 0em;">)</span></span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mop">lo<span style="margin-right: 0.01389em;">g</span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mord mathdefault">p</span><span class="mopen">(</span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mclose">)</span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size4">]</span></span></span></span></span></span></span></span><br> 如果假设观测值服从多元高斯分布：<br> <span class="katex--display"><span class="katex-display"><span class="katex"><span class="katex-mathml"> 
  
   
    
    
      p 
     
     
     
       ( 
      
      
      
        r 
       
      
        i 
       
      
     
       ∣ 
      
     
       ξ 
      
     
       ) 
      
     
    
      = 
     
    
      N 
     
     
     
       ( 
      
      
      
        μ 
       
      
        i 
       
      
     
       , 
      
      
      
        Σ 
       
      
        i 
       
      
     
       ) 
      
     
    
      , 
     
    
      p 
     
    
      ( 
     
    
      ξ 
     
    
      ) 
     
    
      = 
     
    
      N 
     
     
     
       ( 
      
      
      
        μ 
       
      
        ξ 
       
      
     
       , 
      
      
      
        Σ 
       
      
        ξ 
       
      
     
       ) 
      
     
    
   
     p\left(\mathbf{r}_{i} \mid \boldsymbol{\xi}\right)=\mathcal{N}\left(\boldsymbol{\mu}_{i}, \boldsymbol{\Sigma}_{i}\right), p(\boldsymbol{\xi})=\mathcal{N}\left(\boldsymbol{\mu}_{\xi}, \boldsymbol{\Sigma}_{\xi}\right) 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord mathdefault">p</span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord"><span class="mord"><span class="mord mathbf">r</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.311664em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">∣</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mclose delimcenter" style="top: 0em;">)</span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord"><span class="mord mathcal" style="margin-right: 0.14736em;">N</span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">μ</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.217524em;"><span class="" style="top: -2.45586em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.24414em;"><span class=""></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.05451em;">Σ</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.311664em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;">)</span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mord mathdefault">p</span><span class="mopen">(</span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 1.23025em; vertical-align: -0.380248em;"></span><span class="mord"><span class="mord mathcal" style="margin-right: 0.14736em;">N</span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size1">(</span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">μ</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.241968em;"><span class="" style="top: -2.45586em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.04601em;">ξ</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.380248em;"><span class=""></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.05451em;">Σ</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.336108em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.04601em;">ξ</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.286108em;"><span class=""></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size1">)</span></span></span></span></span></span></span></span><br> 则有：<br> <span class="katex--display"><span class="katex-display"><span class="katex"><span class="katex-mathml"> 
  
   
    
     
     
       ξ 
      
      
      
        M 
       
      
        A 
       
      
        P 
       
      
     
    
      = 
     
     
      
      
        argmin 
       
      
        ⁡ 
       
      
     
       ξ 
      
     
     
     
       ∑ 
      
     
       i 
      
     
     
      
      
        ∥ 
       
       
       
         r 
        
       
         i 
        
       
      
        − 
       
       
       
         μ 
        
       
         i 
        
       
      
        ∥ 
       
      
      
      
        Σ 
       
      
        i 
       
      
     
       2 
      
     
    
      + 
     
     
      
      
        ∥ 
       
      
        ξ 
       
      
        − 
       
       
       
         μ 
        
       
         ξ 
        
       
      
        ∥ 
       
      
      
      
        Σ 
       
      
        ξ 
       
      
     
       2 
      
     
    
   
     \boldsymbol{\xi}_{\mathrm{MAP}}=\underset{\boldsymbol{\xi}}{\operatorname{argmin}} \sum_{i}\left\|\mathbf{r}_{i}-\boldsymbol{\mu}_{i}\right\|_{\boldsymbol{\Sigma}_{i}}^{2}+\left\|\boldsymbol{\xi}-\boldsymbol{\mu}_{\xi}\right\|_{\Sigma_{\xi}}^{2} 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.93858em; vertical-align: -0.24414em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.234191em;"><span class="" style="top: -2.45586em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mathrm mtight">M</span><span class="mord mathrm mtight">A</span><span class="mord mathrm mtight">P</span></span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.24414em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 2.32767em; vertical-align: -1.27767em;"></span><span class="mord"><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.66786em;"><span class="" style="top: -2.15345em; margin-left: 0em;"><span class="pstrut" style="height: 3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.03021em;">ξ</span></span></span></span></span></span><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class=""><span class="mop"><span class="mop"><span class="mord mathrm">a</span><span class="mord mathrm">r</span><span class="mord mathrm" style="margin-right: 0.01389em;">g</span><span class="mord mathrm">m</span><span class="mord mathrm">i</span><span class="mord mathrm">n</span></span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 1.08266em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.05001em;"><span class="" style="top: -1.87233em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span><span class="" style="top: -3.05001em;"><span class="pstrut" style="height: 3.05em;"></span><span class=""><span class="mop op-symbol large-op">∑</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 1.27767em;"><span class=""></span></span></span></span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top: 0em;">∥</span><span class="mord"><span class="mord"><span class="mord mathbf">r</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.311664em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">μ</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.217524em;"><span class="" style="top: -2.45586em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.24414em;"><span class=""></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;">∥</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.954008em;"><span class="" style="top: -2.4003em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.05451em;">Σ</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.328086em;"><span class="" style="top: -2.357em; margin-right: 0.0714286em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mathdefault mtight">i</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.143em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="" style="top: -3.2029em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">2</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.3998em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.222222em;"></span></span><span class="base"><span class="strut" style="height: 1.69903em; vertical-align: -0.633028em;"></span><span class="minner"><span class="minner"><span class="mopen"><span class="delimsizing mult"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.86199em;"><span class="" style="top: -2.25599em;"><span class="pstrut" style="height: 2.606em;"></span><span class="delimsizinginner delim-size1"><span class="">∥</span></span></span><span class="" style="top: -2.86199em;"><span class="pstrut" style="height: 2.606em;"></span><span class="delimsizinginner delim-size1"><span class="">∥</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.35001em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.222222em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">μ</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.241968em;"><span class="" style="top: -2.45586em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.04601em;">ξ</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.380248em;"><span class=""></span></span></span></span></span></span><span class="mclose"><span class="delimsizing mult"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.86199em;"><span class="" style="top: -2.25599em;"><span class="pstrut" style="height: 2.606em;"></span><span class="delimsizinginner delim-size1"><span class="">∥</span></span></span><span class="" style="top: -2.86199em;"><span class="pstrut" style="height: 2.606em;"></span><span class="delimsizinginner delim-size1"><span class="">∥</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.35001em;"><span class=""></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.066em;"><span class="" style="top: -2.27005em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight">Σ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3448em;"><span class="" style="top: -2.34877em; margin-left: 0em; margin-right: 0.0714286em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.04601em;">ξ</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.290114em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="" style="top: -3.31489em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">2</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.633028em;"><span class=""></span></span></span></span></span></span></span></span></span></span></span></p>

注意右下角的

      ∥ 
     
    
      . 
     
    
      . 
     
     
     
       ∥ 
      
      
      
        Σ 
       
      
        ξ 
       
      
     
       2 
      
     
    
   
     \|..\|_{\Sigma_{\xi}}^{2} 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 1.29252em; vertical-align: -0.478411em;"></span><span class="mord">∥</span><span class="mord">.</span><span class="mord">.</span><span class="mord"><span class="mord">∥</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.814108em;"><span class="" style="top: -2.42467em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight">Σ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3448em;"><span class="" style="top: -2.34877em; margin-left: 0em; margin-right: 0.0714286em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.04601em;">ξ</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.290114em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">2</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.478411em;"><span class=""></span></span></span></span></span></span></span></span></span></span>表示<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
  
   
    
    
      ∥ 
     
    
      . 
     
    
      . 
     
     
     
       ∥ 
      
      
     
       2 
      
     
    
   
     \|..\|_{}^{2} 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 1.06411em; vertical-align: -0.25em;"></span><span class="mord">∥</span><span class="mord">.</span><span class="mord">.</span><span class="mord"><span class="mord">∥</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.814108em;"><span class="" style="top: -2.453em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"></span></span></span><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">2</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span></span></span></span></span>需要除以<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
  
   
    
     
     
       Σ 
      
     
       ξ 
      
     
    
   
     {\Sigma_{\xi}} 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.969438em; vertical-align: -0.286108em;"></span><span class="mord"><span class="mord"><span class="mord">Σ</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.336108em;"><span class="" style="top: -2.55em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.04601em;">ξ</span></span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.286108em;"><span class=""></span></span></span></span></span></span></span></span></span></span></span>，这也是为什么高斯牛顿中出现中间的协方差的逆的原因。不过有时候我们假设其为1了，也就是省略不写</p>

这个最小二乘的求解为：（下面结果如何推导出的，看下面的细节说明）

       J 
      
     
       T 
      
     
     
     
       Σ 
      
      
      
        − 
       
      
        1 
       
      
     
    
      J 
     
    
      δ 
     
    
      ξ 
     
    
      = 
     
    
      − 
     
     
     
       J 
      
     
       T 
      
     
     
     
       Σ 
      
      
      
        − 
       
      
        1 
       
      
     
    
      r 
     
    
   
     \mathbf{J}^{T} \mathbf{\Sigma}^{-1} \mathbf{J} \delta \boldsymbol{\xi}=-\mathbf{J}^{T} \mathbf{\Sigma}^{-1} \mathbf{r} 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 1.08577em; vertical-align: -0.19444em;"></span><span class="mord"><span class="mord"><span class="mord mathbf">J</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.891331em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.13889em;">T</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord mathbf">Σ</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.864108em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">−</span><span class="mord mtight">1</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord mathbf">J</span></span><span class="mord mathdefault" style="margin-right: 0.03785em;">δ</span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.03021em;">ξ</span></span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 0.974661em; vertical-align: -0.08333em;"></span><span class="mord">−</span><span class="mord"><span class="mord"><span class="mord mathbf">J</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.891331em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.13889em;">T</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord mathbf">Σ</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.864108em;"><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">−</span><span class="mord mtight">1</span></span></span></span></span></span></span></span></span><span class="mord"><span class="mord mathbf">r</span></span></span></span></span></span></span></p>

这里对上面省略掉的推到细节进行说明：

前期基础：

考虑一个任意的高维高斯分布

      x 
     
    
      ∼ 
     
    
      N 
     
    
      ( 
     
    
      μ 
     
    
      , 
     
    
      Σ 
     
    
      ) 
     
    
   
     \boldsymbol{x} \sim N(\boldsymbol{\mu}, \mathbf{\Sigma}) 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.44444em; vertical-align: 0em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol">x</span></span></span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">∼</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord mathdefault" style="margin-right: 0.10903em;">N</span><span class="mopen">(</span><span class="mord"><span class="mord"><span class="mord boldsymbol">μ</span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mord"><span class="mord mathbf">Σ</span></span><span class="mclose">)</span></span></span></span></span>它的概率密度函数展开形式为:<br> <span class="katex--display"><span class="katex-display"><span class="katex"><span class="katex-mathml"> 
   
    
     
     
       P 
      
     
       ( 
      
     
       x 
      
     
       ) 
      
     
       = 
      
      
      
        1 
       
       
        
        
          ( 
         
        
          2 
         
        
          π 
         
         
         
           ) 
          
         
           N 
          
         
        
          det 
         
        
          ⁡ 
         
        
          ( 
         
        
          Σ 
         
        
          ) 
         
        
       
      
     
       exp 
      
     
       ⁡ 
      
      
      
        ( 
       
      
        − 
       
       
       
         1 
        
       
         2 
        
       
      
        ( 
       
      
        x 
       
      
        − 
       
      
        μ 
       
       
       
         ) 
        
       
         T 
        
       
       
       
         Σ 
        
        
        
          − 
         
        
          1 
         
        
       
      
        ( 
       
      
        x 
       
      
        − 
       
      
        μ 
       
      
        ) 
       
      
        ) 
       
      
     
    
      P(x)=\frac{1}{\sqrt{(2 \pi)^{N} \operatorname{det}(\mathbf{\Sigma})}} \exp \left(-\frac{1}{2}(x-\mu)^{T} \mathbf{\Sigma}^{-1}(x-\mu)\right) 
     
    
  </span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord mathdefault" style="margin-right: 0.13889em;">P</span><span class="mopen">(</span><span class="mord mathdefault">x</span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 2.58em; vertical-align: -1.13em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.32144em;"><span class="" style="top: -2.16633em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord sqrt"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.943666em;"><span class="svg-align" style="top: -3.2em;"><span class="pstrut" style="height: 3.2em;"></span><span class="mord" style="padding-left: 1em;"><span class="mopen">(</span><span class="mord">2</span><span class="mord mathdefault" style="margin-right: 0.03588em;">π</span><span class="mclose"><span class="mclose">)</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.767331em;"><span class="" style="top: -2.989em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathdefault mtight" style="margin-right: 0.10903em;">N</span></span></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mop"><span class="mord mathrm">d</span><span class="mord mathrm">e</span><span class="mord mathrm">t</span></span><span class="mopen">(</span><span class="mord"><span class="mord mathbf">Σ</span></span><span class="mclose">)</span></span></span><span class="" style="top: -2.90367em;"><span class="pstrut" style="height: 3.2em;"></span><span class="hide-tail" style="min-width: 1.02em; height: 1.28em;"> 
                <svg width="400em" height="1.28em" viewBox="0 0 400000 1296" preserveAspectRatio="xMinYMin slice"> 
                 <path d="M263,681c0.7,0,18,39.7,52,119c34,79.3,68.167,

158.7,102.5,238c34.3,79.3,51.8,119.3,52.5,120c340,-704.7,510.7,-1060.3,512,-1067
c4.7,-7.3,11,-11,19,-11H40000v40H1012.3s-271.3,567,-271.3,567c-38.7,80.7,-84,
175,-136,283c-52,108,-89.167,185.3,-111.5,232c-22.3,46.7,-33.8,70.3,-34.5,71
c-4.7,4.7,-12.3,7,-23,7s-12,-1,-12,-1s-109,-253,-109,-253c-72.7,-168,-109.3,
-252,-110,-252c-10.7,8,-22,16.7,-34,26c-22,17.3,-33.3,26,-34,26s-26,-26,-26,-26
s76,-59,76,-59s76,-60,76,-60z M1001 80H40000v40H1012z">
1exp(−21(x−μ)TΣ−1(x−μ))
取它的负对数, 则变为:

       − 
      
     
       ln 
      
     
       ⁡ 
      
     
       ( 
      
     
       P 
      
     
       ( 
      
     
       x 
      
     
       ) 
      
     
       ) 
      
     
       = 
      
      
      
        1 
       
      
        2 
       
      
     
       ln 
      
     
       ⁡ 
      
      
      
        ( 
       
      
        ( 
       
      
        2 
       
      
        π 
       
       
       
         ) 
        
       
         N 
        
       
      
        det 
       
      
        ⁡ 
       
      
        ( 
       
      
        Σ 
       
      
        ) 
       
      
        ) 
       
      
     
       + 
      
      
      
        1 
       
      
        2 
       
      
     
       ( 
      
     
       x 
      
     
       − 
      
     
       μ 
      
      
      
        ) 
       
      
        T 
       
      
      
      
        Σ 
       
       
       
         − 
        
       
         1 
        
       
      
     
       ( 
      
     
       x 
      
     
       − 
      
     
       μ 
      
     
       ) 
      
     
       . 
      
     
    
      -\ln (P(x))=\frac{1}{2} \ln \left((2 \pi)^{N} \operatorname{det}(\mathbf{\Sigma})\right)+\frac{1}{2}(x-\boldsymbol{\mu})^{T} \boldsymbol{\Sigma}^{-1}(\boldsymbol{x}-\boldsymbol{\mu}) . 
     
    
  </span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord">−</span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mop">ln</span><span class="mopen">(</span><span class="mord mathdefault" style="margin-right: 0.13889em;">P</span><span class="mopen">(</span><span class="mord mathdefault">x</span><span class="mclose">)</span><span class="mclose">)</span><span class="mspace" style="margin-right: 0.277778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.277778em;"></span></span><span class="base"><span class="strut" style="height: 2.00744em; vertical-align: -0.686em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.32144em;"><span class="" style="top: -2.314em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">2</span></span></span><span class="" style="top: -3.23em;"><span class="pstrut" style="height: 3em;"></span><span class="frac-line" style="border-bottom-width: 0.04em;"></span></span><span class="" style="top: -3.677em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s"></span></span><span class="vlist-r"><span class="vlist" style="height: 0.686em;"><span class=""></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="mop">ln</span><span class="mspace" style="margin-right: 0.166667em;"></span><span class="minner"><span class="mopen del