Hyperbolic Representation Learning

Manifolds: A Gentle Introduction

Manifolds

• 关于 Manifold，我们可能想到的就是 Manifold Hypothesis：现实中的高维数据通常位于一个嵌入在高维空间里的低维流形，这解释了为什么 ML 能从高维数据中抽取出有效的低维特征。但究竟什么是 Manifold？

Manifolds belong to the branches of mathematics of topology and differential geometry.

---

Circles and Spheres as Manifolds

• Manifold 可以看作局部与欧氏空间同胚 (homeomorphic) 的拓扑空间，其中同胚是指能保留拓扑性质的连续双射 (当然欧氏空间也是 manifold)，因此 $n$ 维流形就是指局部能被 ${\mathbb{R}}^n$ 近似的空间。下图展示了几种嵌入在二维空间里的一维流形，它们的每条小圆弧都与一维线段近似，但二维空间里的任何有交点的空间都不是一维流形 (e.g. 8)，因为交点处并不满足对一维线段的局部近似
  下图展示了几种嵌入在三维空间里的二维流形。Similarly, any 2D surface (including a plane) that doesn’t self-intersect is also a 2D manifold.
  

---

A (slightly) More Formal Look at Manifolds

• 如下图所示，$X$ 为 manifold，$U_{\alpha },U_{\beta }$ 为两个不同的 “patches” or domains，每个 domain 都可以通过 chart ${\phi }_{\alpha },{\phi }_{\beta }$ 映射到低维欧氏空间。如果一个集合的 charts 对应的 domains 刚好覆盖整个 manifold，则该集合为 atlas. ${\phi }_{\alpha \beta }={\phi }_{\beta }\circ {\phi }_{\alpha }^{-1},{\phi }_{\beta \alpha }={\phi }_{\alpha }\circ {\phi }_{\beta }^{-1}$ 为 transition map，当 transition maps $k$ 次可微时，manifold 为 differentiable manifolds，当 transition maps 无穷可微时，manifold 为 smooth manifolds.
  

---

Example: A 1D Manifold with Multiple Charts

• 单位圆是一维流形，如下图所示，可以定义 4 个 charts $M\to \mathbb{R}$ 将一维流形映射到一维空间 (极坐标表示，$\theta \in [0,2\pi )$)，定义多个 charts 是因为 domain 必须为 open set
  $$\begin{array}{rl}{\phi }_1(r,\theta )=\theta & \theta \in \left(-\frac{\pi }{3},\frac{\pi }{3}\right)\\ {\phi }_2(r,\theta )=\theta & \theta \in \left(\frac{\pi }{6},\frac{5\pi }{6}\right)\\ {\phi }_3(r,\theta )=\theta & \theta \in \left(\frac{2\pi }{3},\frac{4\pi }{3}\right)\\ {\phi }_4(r,\theta )=\theta & \theta \in \left(\frac{7\pi }{6},\frac{11\pi }{6}\right)\end{array}$$
• 我们也可以通过 stereographic projection 定义 charts. 通过 “north” or “south” pole $(0,1),(0,-1)$ 可以定义两个 charts，将除了自身外的其余点投影到 $x$ 轴上
  $$\begin{gathered} u_1:={\phi }_1(p)=\frac{{\phi }_1(p)}{1}=\frac{x_p}{1-y_p} \\ {u}_2:={\phi }_2(q)=\frac{{\phi }_2(q)}{1}=\frac{x_q}{1+y_q} \end{gathered}$$联立等式 $x^2+y^2=1$ 可得逆映射
  $$\begin{gathered} x_p=\frac{2u_1}{u_1^2+1},y_p=\frac{u_1^2-1}{u_1^2+1} \\ {x}_q=\frac{2u_2}{u_2^2+1},y_q=\frac{1-u_2^2}{u_2^2+1} \end{gathered}$$由此可得 transition map ${\phi }_{\alpha \beta }$
  $$\begin{array}{rl}{u}_2& ={\phi }_{\alpha \beta }\left(u_1\right)\\ & ={\phi }_2\circ {\phi }_1^{-1}\left(u_1\right)\\ & ={\phi }_2\left({\phi }_1^{-1}\left(u_1\right)\right)\\ & ={\phi }_2\left(\left(\frac{2u_1}{u_1^2+1},\frac{u_1^2-1}{u_1^2+1}\right)\right)\\ & =\frac{\frac{2u_1}{u_1^2+1}}{1+\frac{u_1^2-1}{u_1^2+1}}\\ & =\frac{1}{u_1}\end{array}$$

---

Example: Stereographic Projections for $S^n$ (球极平面投影)

• $n$-dimensional sphere $S^n$ 是嵌入在 $(n+1)$ 维欧氏空间中的流形，下图展示了使用 stereographic projection 将 $S^2$ 映射到二维空间
  该方法可以推广到 $S^n$
  对于超球体上的任意点 $p=(\mathbf{x},z)\in S^n$，用 north pole 做投影，在超平面 $z=0$ 上得到的投影点坐标为 $({\mathbf{u}}_N,0)$
  $$\begin{array}{rl}{\mathbf{u}}_{\mathbf{N}}& :={\phi }_N(p)=\frac{\mathbf{x}}{1-z}\\ \mathbf{x}& =\frac{2{\mathbf{u}}_{\mathbf{N}}}{{\left|{\mathbf{u}}_{\mathbf{N}}\right|}^2+1}\\ z& =\frac{{\left|{\mathbf{u}}_{\mathbf{N}}\right|}^2-1}{{\left|{\mathbf{u}}_{\mathbf{N}}\right|}^2+1}\end{array}$$

Tangent Spaces

• 定义 Manifold $M$ 在 $x$ 处的 tangent space 为 $T_{x}M$，它由所有经过 $x$ 的 curves $\gamma (t)$ 的 tangent vector $v$ 张成。该空间可以被看作 $x$ 处对 $M$ 的一阶线性近似
  

---

Tangent Spaces as the Velocity of Curves

• 假设 $p\in M$，$M$ 为 smooth manifold，可以定义 chart $\phi :U\to {\mathbb{R}}^n$，其中 $U$ 为 $M$ 上包含 $p$ 的 open subset. 进一步可以定义一个 smooth parametric curve $\gamma :t\to M$ 用于将 $t\in [a,b]$ 映射到 $M$ 上过 $p$ 点的一条曲线，因此 $\phi \circ \gamma :t\to {\mathbb{R}}^n$ 可以将 $t$ 直接映射到欧氏空间上的点 $x$. 当 $t=t_0$ 时 (对应点 $p$)，有
  $$\text{ "velocity" at }p={\frac{d\phi \circ \gamma (t)}{dt}|}_{t=t_0}={\left[\frac{d{x}^1(t)}{dt},\dots ,\frac{d{x}^n(t)}{dt}\right]|}_{t=t_0}$$上述速度向量即可视为 tangent vector $v$，所有 tangent vector 张成的空间 $T_{p}M$ tangent space 可以被表示为 ${\mathbb{R}}^n$，因此我们可以进行常规的欧式向量空间操作

---

Basis of the Tangent Space

• 首先我们定义任意 test function $f:M\to \mathbb{R}$ 以及 smooth parametric curve $\gamma :t\to M$，那么 “velocity” relative to this test function 为
  $$\begin{array}{rlrl}\mathbf{v}f& =\frac{df\circ \gamma (t)}{dt}{|}_{t=t_0}& & \\ & =\frac{d(f\circ {\phi }^{-1}\circ \phi \circ \gamma )(t)}{dt}{|}_{t=t_0}& & \\ & =\frac{d((f\circ {\phi }^{-1})\circ (\phi \circ \gamma ))(t)}{dt}{|}_{t=t_0}& & \\ & =\sum _i\frac{\partial (f\circ {\phi }^{-1})(x)}{\partial x_i}{|}_{x=\phi \circ \gamma (t_0)}\frac{d(\phi \circ \gamma {)}^i(t)}{dt}{|}_{t=t_0}& & \text{chain rule}\\ & =\sum _i\frac{\partial (f\circ {\phi }^{-1})(x)}{\partial x_i}{|}_{x=\phi (p)}\frac{d(\phi \circ \gamma {)}^i(t)}{dt}{|}_{t=t_0}& & \text{since }p=\gamma (t_0)\\ & =\sum _i(\text{basis for component }i)(\text{"velocity" of component i wrt to }\phi )& & \end{array}$$
• 在 vector space 中，向量为向量各个坐标分量乘上对应 basis 的加权和，例如欧氏空间中常用的 basis 为 $e_1,e_2,...$. 上述式子中可以将 $\frac{d(\phi \circ \gamma {)}^i(t)}{dt}{|}_{t=t_0}$ 看作坐标分量，$\frac{\partial (f\circ {\phi }^{-1})(x)}{\partial x_i}{|}_{x=\phi (p)}$ 看作对应的 basis，可以简写为
  $$\begin{array}{r}(\frac{\partial }{\partial x^i}{)}_p(f):=\frac{\partial (f\circ {\phi }^{-1})(\phi (p))}{\partial x_i}\end{array}$$由于 $f$ 是任取的，因此我们甚至可以省略 $f$，对于任意切向量 $\mathbf{v}\in T_{p}M$，有
  $$\begin{array}{rl}\mathbf{v}& =\sum _{i=1}^{n}v(x^i)\cdot (\frac{\partial }{\partial x^i}{)}_p\\ & =\sum _{i=1}^n\frac{d(\phi \circ \gamma {)}^i(t)}{dt}{|}_{t=t_0}\cdot (\frac{\partial }{\partial x^i}{)}_p\end{array}$$

---

Change of Basis for Tangent Vectors

• 在 $d$ 维流形 $M$ 中定义两个 charts：
  $$\begin{array}{r}\phi (p)=(x^1(p),\dots ,x^d(p))\\ \vartheta (p)=(y^1(p),\dots ,y^d(p))\end{array}$$下面考虑如何将一个 chart 对应的 tangent space 变换到另一个 chart 对应的 tangent space
  $$\begin{array}{rlrl}\mathbf{v}f& =\sum _{i}v(x^i)\cdot (\frac{\partial }{\partial x^i}{)}_{p}f& & \\ & =\sum _{i}v(x^i)\cdot {\partial }_{x^i}(f\circ {\phi }^{-1})(\phi (p))& & \text{by definition}\\ & =\sum _{i}v(x^i)\cdot {\partial }_{x^i}(f\circ {\vartheta }^{-1}\circ \vartheta \circ {\phi }^{-1})(\phi (p))& & \text{introduce }\vartheta \text{ with identity trick}\\ & =\sum _{i}v(x^i)\cdot {\partial }_{x^i}((f\circ {\vartheta }^{-1})\circ (\vartheta \circ {\phi }^{-1}))(\phi (p))& & \\ & =\sum _{i}v(x^i)\cdot \sum _j{\partial }_{x^i}(\vartheta \circ {\phi }^{-1}{)}^j(\phi (p))\cdot {\partial }_{y^j}(f\circ {\vartheta }^{-1})(\vartheta \circ {\phi }^{-1}(\phi (p)))& & \text{chain rule}\\ & =\sum _{i}v(x^i)\cdot \sum _j{\partial }_{x^i}(\vartheta \circ {\phi }^{-1}{)}^j(\phi (p))\cdot {\partial }_{y^j}(f\circ {\vartheta }^{-1})(\vartheta (p))& & \text{simplifying}\\ & =\sum _{i}v(x^i)\cdot \sum _j{\partial }_{x^i}(\vartheta \circ {\phi }^{-1}{)}^j(\phi (p))\cdot (\frac{\partial }{\partial y^j}{)}_{p}f& & \text{by definition}\\ & =\sum _{i}v(x^i)\cdot \sum _j\frac{\partial y^j}{\partial x^i}{|}_{x=\phi (p)}\cdot (\frac{\partial }{\partial y^j}{)}_{p}f& & \text{since }{y}^j(x)=y^j({\phi }^{-1}(\phi (x)))\\ & =\sum _j\sum _{i}v(x^i)\cdot \frac{\partial y^j}{\partial x^i}{|}_{x=\phi (p)}\cdot (\frac{\partial }{\partial y^j}{)}_{p}f& & \\ \mathbf{v}f& =\sum _{j}v(y^j)\cdot (\frac{\partial }{\partial y^j}{)}_{p}f& & \end{array}$$其中，${\partial }_{x}f:=\frac{\partial f}{\partial x}$. 由上述两式可知 change of basis matrix 即为 Jacobian $J_{\mathbf{y}}$
  $$\begin{array}{r}\mathbf{v}(\mathbf{y})=\left[\begin{array}{c}v(y^1)\\ \dots \\ v(y^d)\end{array}\right]={\mathbf{J}}_{\mathbf{y}}\mathbf{v}(\mathbf{x})=\left[\begin{array}{ccc}\frac{\partial y^1}{\partial x^1}{|}_{x=\phi (p)}& \cdots & \frac{\partial y^1}{\partial x^d}{|}_{x=\phi (p)}\\ ⋮& \ddots & ⋮\\ \frac{\partial y^d}{\partial x^1}{|}_{x=\phi (p)}& \cdots & \frac{\partial y^d}{\partial x^d}{|}_{x=\phi (p)}\end{array}\right]\left[\begin{array}{c}v(x^1)\\ \dots \\ v(x^d)\end{array}\right]\end{array}$$

---

Example: Tangent Vectors on a Sphere

• 考虑如下的单位球体，定义 curve $\gamma (t)$ (红蓝交界处)，$\theta =\pi /4$
  $$\begin{array}{c}\gamma (t)=(\cos \frac{\pi }{4}\cos \pi t,\cos \frac{\pi }{4}\sin \pi t,\sin \frac{\pi }{4}),t\in [-1,1]\end{array}$$下面求 $p=\gamma (t_0=0)=(x,y,z)=(\frac{1}{\sqrt{2}},0,\frac{1}{\sqrt{2}})$ 处的 tangent vector
• 由前面的 “Example: Stereographic Projections for $S^n$” 可知，将单位球体由 north pole 投影到 $z=0$ 的超平面上得到的投影坐标为 $(u_1,u_2)$，i.e., $\phi$
  $$\begin{array}{rl}{u}_1(x,y,z)& =\frac{x}{1-z}\\ u_2(x,y,z)& =\frac{y}{1-z}\end{array}$$因此 $p$ 处的 tangent vector 坐标为
  $$\begin{array}{rl}\frac{d\phi \circ \gamma (t)}{dt}{|}_{t=t_0}& =[\frac{d{u}_1(\cos \frac{\pi }{4}\cos \pi t,\cos \frac{\pi }{4}\sin \pi t,\sin \frac{\pi }{4})}{dt},\frac{d{u}_2(\cos \frac{\pi }{4}\cos \pi t,\cos \frac{\pi }{4}\sin \pi t,\sin \frac{\pi }{4})}{dt},]{|}_{t=t_0}\\ & =[\frac{d((\sqrt{2}+1)\cos \pi t)}{dt},\frac{d((\sqrt{2}+1)\sin \pi t)}{dt}]{|}_{t=t_0}\\ & =[(\sqrt{2}+1)\pi (-\sin \pi t),(\sqrt{2}+1)\pi \cos \pi t]{|}_{t=t_0}\\ & =(0,(\sqrt{2}+1)\pi )\end{array}$$加上 basis 可以得到如下的 tangent vector
  $$\begin{array}{c}{\mathbf{T}}_{\phi }=0\cdot (\frac{\partial }{\partial u_1}{)}_p+(\sqrt{2}+1)\pi \cdot (\frac{\partial }{\partial u_2}{)}_p\end{array}$$
• 下面考虑将上述 tangent vector 变换到由 south pole 定义的 chart $\vartheta$. ${\phi }^{-1}$ 为
  $$\begin{array}{rl}x& =\frac{2u_1}{u_1^2+u_2^2+1}\\ y& =\frac{2u_2}{u_1^2+u_2^2+1}\\ z& =\frac{u_1^2+u_2^2-1}{u_1^2+u_2^2+1}\end{array}$$$\vartheta$ 为
  $$\begin{array}{rl}{w}_1(x,y,z)& =\frac{x}{1+z}\\ w_2(x,y,z)& =\frac{y}{1+z}\end{array}$$因此可以将 $w_i$ 用 $u_i$ 表示
  $$\begin{array}{rl}{w}_i(u_1,u_2)& =w_i\circ {\phi }^{-1}(u_1,u_2)\\ & =w_i(\frac{2u_1}{u_1^2+u_2^2+1},\frac{2u_2}{u_1^2+u_2^2+1},\frac{u_1^2+u_2^2-1}{u_1^2+u_2^2+1})\end{array}$$代入下式可得
  $$\begin{array}{rl}\mathbf{v}(\mathbf{w})& ={\mathbf{J}}_{\mathbf{u}}\mathbf{v}(\mathbf{u})\\ & =\left[\begin{array}{cc}\frac{\partial w_1}{\partial u_1}{|}_{u=\phi (p)}& \frac{\partial w_1}{\partial u_2}{|}_{u=\phi (p)}\\ \frac{\partial w_2}{\partial u_1}{|}_{u=\phi (p)}& \frac{\partial w_2}{\partial u_2}{|}_{u=\phi (p)}\end{array}\right]\left[\begin{array}{c}0\\ (\sqrt{2}+1)\pi \end{array}\right]\\ & =\left[\begin{array}{cc}\frac{u_2^2-u_1^2}{(u_1^2+u_2^2{)}^2}{|}_{u=\phi (p)}& -\frac{2u_1{u}_2}{(u_1^2+u_2^2{)}^2}{|}_{u=\phi (p)}\\ -\frac{2u_1{u}_2}{(u_1^2+u_2^2{)}^2}{|}_{u=\phi (p)}& \frac{u_1^2-u_2^2}{(u_1^2+u_2^2{)}^2}{|}_{u=\phi (p)}\end{array}\right]\left[\begin{array}{c}0\\ (\sqrt{2}+1)\pi \end{array}\right]\\ & =\left[\begin{array}{cc}\frac{-(\sqrt{2}+1{)}^2}{(\sqrt{2}+1{)}^4}& 0\\ 0& \frac{(\sqrt{2}+1{)}^2}{(\sqrt{2}+1{)}^4}\end{array}\right]\left[\begin{array}{c}0\\ (\sqrt{2}+1)\pi \end{array}\right]\\ & =\left[\begin{array}{c}0\\ (\sqrt{2}-1)\pi \end{array}\right]\end{array}$$

Metric Tensor

Covariant vs. Contravariant Tensors

• 假设向量空间中的一组基为 $\mathbf{v}$，向量 $u$ 可以被表示为 $\mathbf{u}=\mathbf{v}\mathbf{c}$，其中 $\mathbf{c}$ 可以看作是向量坐标。现在进行基变换得到一组新基 $\tilde{\mathbf{v}}$，对应的基变换为 $T$，i.e.，$\tilde{\mathbf{v}}=\mathbf{v}T$，此时有 $u=\tilde{\mathbf{v}}\tilde{\mathbf{c}}$
• Contravariant vector 是指在进行基变换时，向量坐标对应的变换为基变换的逆 $T^{-1}$，i.e.，$\tilde{\mathbf{c}}=T^{-1}\mathbf{c}$. 通常将其写为列向量的形式并用上标表示
  $$\begin{array}{c}{v}^{\alpha }=\left[\begin{array}{c}{v}^0\\ v^1\\ v^2\end{array}\right]\end{array}$$例如欧氏空间里的 geometric vector 即为 contravariant vector，证明如下：
  $$\begin{gathered} \tilde{\mathbf{v}}\tilde{\mathbf{c}}=\mathbf{vc} \\ \Rightarrow \mathbf{v}T\tilde{\mathbf{c}}=\mathbf{vc} \\ \Rightarrow \tilde{\mathbf{c}}=T^{-1}\mathbf{c} \end{gathered}$$
• Covariant vector 是指在进行基变换时，向量坐标对应的变换即为基变换 $T$. 通常将其写为行向量的形式并用下标表示
  $$\begin{array}{c}{u}_{\alpha }=[v_0,v_1,v_2]\end{array}$$例如线性函数 $f(\mathbf{x})=\mathbf{cx}$ 即为 covariant vector ($\mathbf{c}$ 为行向量，$\mathbf{x}$ 为列向量)，证明如下：
  $$\begin{array}{rlrl}\tilde{\mathbf{c}}\tilde{\mathbf{x}}& =\mathbf{cx}& & \\ \tilde{\mathbf{c}}(T^{-1}\mathbf{x})& =\mathbf{cx}& & \text{基变换后}，\tilde{\mathbf{x}}=T^{-1}\mathbf{x}\\ \Rightarrow \tilde{\mathbf{c}}& =\mathbf{c}T& & \end{array}$$
• We’ll usually write of a $(n,m)$-tensor where $n$ is the number of contravariant components and $m$ is the number of covariant components. The rank is then the sum of $m+n$. Therefore a contravariant vector is a $(1,0)$-tensor and a covector is a $(0,1)$-tensor.

---

Linear Transformations as Tensors

• 下面考虑进行基变换后 $\tilde{L}$ 的形式。假设 $\mathbf{v}$ 为 geometric vector，$\mathbf{u}=L\mathbf{v}$ 为线性变换后的向量坐标，有
  $$\begin{array}{rl}& L\mathbf{v}=\mathbf{u}\\ \Rightarrow & L(T\tilde{\mathbf{v}})=T\tilde{\mathbf{u}}\\ \Rightarrow & \tilde{\mathbf{u}}=T^{-1}LT\tilde{\mathbf{v}}\end{array}$$由 $\tilde{\mathbf{u}}=\tilde{L}\tilde{\mathbf{v}}$ 可知，
  $$\tilde{L}=T^{-1}LT$$因此线性变换为 $(1,1)$-tensor

---

Bilinear Forms

• Bilinear Forms 可以写为矩阵形式:
  $$\begin{array}{c}B(\mathbf{u},\mathbf{v})={\mathbf{u}}^{\mathbf{T}}A\mathbf{v}\end{array}$$
• 下面考虑进行基变换后 $\tilde{A}$ 的形式。假设 $\mathbf{u},\mathbf{v}$ 为 geometric vector，有
  $$\begin{array}{rl}& {\mathbf{u}}^{\mathbf{T}}A\mathbf{v}={\tilde{\mathbf{u}}}^{\mathbf{T}}\tilde{A}\tilde{\mathbf{v}}\\ \Rightarrow & {\mathbf{u}}^{\mathbf{T}}A\mathbf{v}=({\mathbf{u}}^{\mathbf{T}}{T}^{-T})\tilde{A}(T^{-1}\mathbf{v})\\ \Rightarrow & A=T^{-T}\tilde{A}{T}^{-1}\\ \Rightarrow & \tilde{A}=T^{T}AT\end{array}$$因此 Bilinear Forms 为 $(0,2)$-tensor

---

The Metric Tensor

• Metric Tensor 是一种特殊形式的 Bilinear Form，可以用来定义向量间的距离，例如欧氏空间 ${\mathbb{R}}^n$ 中的向量点积对应的 metric tensor 为 $I_n$，i.e.，$\mathbf{u}\cdot \mathbf{v}={\mathbf{u}}^{\mathbf{T}}{I}_n\mathbf{v}$. 利用 Bilinear Form 基变换的结论，我们可以在基变换的情况下求出向量间的距离，i.e.，${\mathbf{u}}^{\mathbf{T}}(T^T{I}_{n}T)\mathbf{v}$
• 当对任何非零向量 $v$ 都有 $g_p(v,v)>0$ 时，metric tensor 为正定矩阵

Riemannian Manifolds (黎曼流形)

Riemannian metric (tensor) (黎曼度量)

• 对于 real manifold $M$，定义 metric tensor $g_p$ 将任意点 $p$ 处的两个切向量映射为实数，如果该映射满足 bilinear、symmetric 以及 positive-definete，则 metric tensor $g_p$ 为黎曼度量 (a family of inner products)
  $$\begin{array}{c}{g}_p:T_{p}M\times T_{p}M\to \mathbb{R},p\in M\end{array}$$

---

Riemannian Manifolds (黎曼流形)

• 对于 real manifold $M$，若 (1) 在 $M$ 的每个点上都定义一个黎曼度量 $g$ (we have a different tensor for $every$ point on the manifold)，且 (2) 对于流形上的点 $p$，$p\to g_p(X(p),Y(p))$ 为 $p$ 的光滑函数，其中 $X(p),Y(p)$ 为两个切向量，则称 $(M,g)$ 为一个黎曼流形 (A manifold $M$ equipped with a Riemannian metric $g$)
• 可见对于黎曼流形而言，虽然相邻点的 tangent space 可能不同 (manifold curves 不同，因此 tangent space 也不同)，但切空间的内积值在相邻点间光滑地改变。Intuitively, Riemannian manifolds have all the nice “smoothness” properties we would want and makes our lives a lot easier.
• 通过 Riemannian metric $g$，我们能定义两点间的距离
  其中 geodesic $\gamma \in {\mathcal{C}}^{\infty }([0,1],M)$ 且 $\gamma (0)=x,\gamma (1)=y$，$\dot{\gamma }(t)$ 为 $\gamma$ 对 $t$ 的一阶导

---

Induced Metric Tensors

• A natural way to define the metric tensor is to take our $n$ dimensional manifold $M$ embedded in $n+k$ dimensional Euclidean space, and use the standard Euclidean metric tensor in $n+k$ space but transformed to a local coordinate system on $M$. That is, we’re going to define our family of Riemannian metric tensors using the metric tensor from the embedded Euclidean space. This guarantees that we’ll have this nice smoothness property because we’re inducing it from the standard Euclidean metric in the embedded space.
• 对于 $p$ 点处的 tangent vector $\mathbf{v}$，$x$ 为 $n+k$ 维 embedded space，$y$ 为 chart $\phi$ 对应的 $n$ 维 local coordinate system，对应 tangent space 的 basis 为 $(\frac{\partial }{\partial y^i}{)}_p$，坐标为 $v(y^i)$. 我们可以进行基变换，找到 $\mathbf{v}$ 在恒等变换 chart $\vartheta$ 对应 tangent space 的坐标，该坐标即为 $\mathbf{v}$ 在 embedded space 中的坐标：
  $$\begin{array}{rl}\mathbf{v}& =\sum _{i}v(y^i)\cdot (\frac{\partial }{\partial y^i}{)}_p\\ & =\sum _{i}v(y^i)\cdot \frac{\partial (\square \circ {\phi }^{-1})}{\partial y_i}{|}_{y=\phi (p)}\\ & =\sum _{i}v(y^i)\cdot \sum _j\frac{\partial x^j}{\partial y^i}{|}_{y=\phi (p)}\frac{\partial \square }{\partial x^j}{|}_{x=p}\\ & =\sum _{i}v(y^i)\cdot \sum _j\frac{\partial x^j}{\partial y^i}{|}_{y=\phi (p)}(\frac{\partial }{\partial x^j}{)}_p\\ & =\sum _j\left(\sum _{i}v(y^i)\cdot \frac{\partial x^j}{\partial y^i}{|}_{y=\phi (p)}\right)(\frac{\partial }{\partial x^j}{)}_p\end{array}$$由此可知，$\mathbf{v}$ 在 embedded space 中的坐标 $x^j$ 为 ${\sum }_{i}v(y^i)\cdot \frac{\partial x^j}{\partial y^i}{|}_{y=\phi (p)}$
• 假设 ${\mathbf{v}}_{\mathbf{M}},{\mathbf{w}}_{\mathbf{M}}$ 为 embedded Euclidean space 中的 tangent vectors，${\mathbf{v}}_{\mathbf{U}},{\mathbf{w}}_{\mathbf{U}}$ 为同样的 tangent vectors，但表示在 local coordinate system 中。可得下式：
  $$\begin{array}{rlrl}{g}_M({\mathbf{v}}_{\mathbf{M}},{\mathbf{w}}_{\mathbf{M}})& ={\mathbf{v}}_{\mathbf{M}}\cdot {\mathbf{w}}_{\mathbf{M}}& & \text{Euclidean inner product}\\ & =\left[\begin{array}{ccc}{\sum }_{i=1}^{d}v(y^i)\cdot \frac{\partial x^1}{\partial y^i}{|}_{y=\phi (p)}& \dots & {\sum }_{i=1}^{d}v(y^i)\cdot \frac{\partial x^n}{\partial y^i}{|}_{y=\phi (p)}\end{array}\right]\left[\begin{array}{c}{\sum }_{i=1}^{d}w(y^i)\cdot \frac{\partial x^1}{\partial y^i}{|}_{y=\phi (p)}\\ \dots \\ {\sum }_{i=1}^{d}w(y^i)\cdot \frac{\partial x^n}{\partial y^i}{|}_{y=\phi (p)}\end{array}\right]& & \\ & =\left[\begin{array}{ccc}v(y^1)& \dots & v(y^d)\end{array}\right]\left[\begin{array}{ccc}\frac{\partial x^1}{\partial y^1}{|}_{y=\phi (p)}& \dots & \frac{\partial x^n}{\partial y^1}{|}_{y=\phi (p)}\\ \cdots & \ddots & \cdots \\ \frac{\partial x^1}{\partial y^d}{|}_{y=\phi (p)}& \dots & \frac{\partial x^n}{\partial y^d}{|}_{y=\phi (p)}\end{array}\right]\left[\begin{array}{ccc}\frac{\partial x^1}{\partial y^1}{|}_{y=\phi (p)}& \dots & \frac{\partial x^1}{\partial y^d}{|}_{y=\phi (p)}\\ \cdots & \ddots & \cdots \\ \frac{\partial x^n}{\partial y^1}{|}_{y=\phi (p)}& \dots & \frac{\partial x^n}{\partial y^d}{|}_{y=\phi (p)}\end{array}\right]\left[\begin{array}{c}w(y^1)\\ \dots \\ w(y^d)\end{array}\right]& & \\ & ={{\mathbf{v}}_{\mathbf{U}}}^T{{\mathbf{J}}_{\mathbf{x}}}^T{\mathbf{J}}_{\mathbf{x}}{\mathbf{w}}_{\mathbf{U}}& & \\ & =g({\mathbf{v}}_{\mathbf{U}},{\mathbf{w}}_{\mathbf{U}})& & \end{array}$$因此 Induced Metric Tensor 为
  $$g={{\mathbf{J}}_{\mathbf{x}}}^T{\mathbf{J}}_{\mathbf{x}}$$
• 有了 Metric Tensor 之后，我们可以计算 tangent vector 的长度 $\Vert \mathbf{u}\Vert =\sqrt{g(\mathbf{u},\mathbf{u})}$，两个 tangent vector 之间的角度 $\cos \theta =\frac{g(\mathbf{u},\mathbf{v})}{\Vert \mathbf{u}\Vert \Vert \mathbf{v}\Vert }$，或是流体的表面积

Hyperbolic Geometry and Poincaré Embeddings

Curvature (曲率)

• Curvature 可以用于衡量曲面 (surfaces) 或曲线 (curves) 偏离平面 (flat plane) 或直线 (straight line) 的程度，曲率越大，偏离程度越大。以曲线的曲率为例，圆的曲率为 $\kappa =\frac{1}{r}$，其中 $r$ 为圆的半径，而曲线上的三点可以确定一个圆，当这三点无限接近于重合时得到的圆即为密切圆，是对重合点 $x_0$ 附近曲线的最佳圆近似，密切圆的曲率即为曲线在该点的曲率
  

---

Gaussian Curvature

• Gaussian Curvature. a measure of curvature for surfaces (2D manifolds). 某点处的高斯曲率为该点主曲率 ${\kappa }_{max}$ 和 ${\kappa }_{min}$ 的乘积，其中主曲率是指所有过该点曲线的极大曲率 ${\kappa }_{max}$ 以及垂直于极大曲率面的极小曲率 ${\kappa }_{min}$. 下图分别对应负曲率、零曲率和正曲率

• Gaussian curvature 仅适用于 2D surfaces，对于更高维度的流形，我们可以通过观察 surface 上 certain geometric objects 的特性相对于 flat space 的变化来感受曲率。下图展示了三角形在不同曲面上的内角和

---

Parallel Transport, Riemannian Curvature Tensor and Sectional Curvature

• Parallel Transport. 如下图所示，假设我们现在位于地球的 $A$ 点并准备一直向北走到 $N$ 点，图中给出了 $A$ 点处的切向量，在移动过程中，我们始终保持切向量的方向不变。之后，我们继续保持切向量方向不变，从 $N$ 点移动到 $B$ 点，最后再回到 $A$ 点，此时我们发现切向量的方向改变了。 这是因为虽然我们感觉自己在平面上移动，但实际上是在曲面上移动，这就会使得经过一个 loop 后切向量的方向改变了。如果我们在平面上 (zero curvature) 移动就不会发生上述现象
  
• Riemannian Curvature Tensor. 我们可以用 Riemannian Curvature Tensor 来度量 deviation of parallel transport. 对于光滑流形上的每个点，它都可以给出一个 $(1,3)$-tensor，返回该点处 tangent space 内两个线性无关向量的线性变换，用于衡量 parallel transport 使 tangent space 中向量方向发生偏移的程度
• Sectional Curvature. 我们可以通过 Riemannian Curvature Tensor 来衡量 $P$ 点处的曲率
  $$\begin{array}{c}K(u,v)=\frac{⟨R(u,v)v,u⟩}{⟨u,u⟩⟨v,v⟩-⟨u,v{⟩}^2}\end{array}$$其中 $u,v$ 是 $P$ 点处 tangent space 中的两个线性无关向量，$R$ 为 Riemannian Curvature Tensor

Euclidean and Non-Euclidean Geometries

Manifolds with Constant Sectional Curvature

• 在每个点处具有恒定截面曲率的黎曼流形是曲面的特殊形式，它具有以下三种情况：
  • Constant Positive Curvature: Elliptic geometry (e.g. hypersphere)
  • Constant Zero Curvature: Euclidean geometry (e.g. Euclidean space)
  • Constant Negative Curvature: Hyperbolic geometry

---

Euclidean geometry

• 欧氏几何满足以下 5 条定理，这些定理定义了一个公理系统 (axiomatic system)：(1) A straight line segment can be drawn joining any two points. (2) Any straight line segment can be extended indefinitely in a straight line. (3) Given any straight line segment, a circle can be drawn having the segment as radius and one endpoint as center. (4) All right angles are congruent. (5) If two lines are drawn which intersect a third in such a way that the sum of the inner angles on one side is less than two right angles, then the two lines inevitably must intersect each other on that side if extended far enough. (aka the parallel postulate)
  
• 欧氏空间是欧氏几何的一种模型 (model). A model for an axiomatic system is a well-defined set, which assigns meaning for the undefined terms presented in the system, in a manner that is correct with the relations defined in the system. 例如欧氏几何定义了点，但没有定义点的具体含义，而欧氏空间则将点定义为了 $x\in {\mathbb{R}}^n$. 将欧氏几何中的抽象概念进行具体定义后，我们就得到了欧氏几何的一个模型

---

Elliptic Geometry

• Elliptic Geometry 将第五条定理 parallel postulate 更改为了 “Two lines perpendicular to a given line must intersect.”
  
• Elliptic Geometry 的一种 model 为 manifold defined by the surface of a sphere

---

Hyperbolic Geometry

• Hyperbolic Geometry 将第五条定理 parallel postulate 更改为了 “For any given line $R$ and point $P$ not on $R$, in the plane containing both line $R$ and point $P$ there are at least two distinct lines through $P$ that do not intersect $R$..”
  Figure: Lines $x$ and $y$ intersecting at $P$ never pass through line $R$, although it is possible that they can asymptotically approach it.

This figure is not a great visualization because, as we’ll mention below, you can’t really intuitively represent 2D hyperbolic geometry in 2D Euclidean space. The real tough part is that even for the 2D hyperbolic plane, we cannot embed it into 3D Euclidean space (Hilbert’s theorem). This makes it hard to visualize, and results in a more complex model than the other two geometries.

Hyperbolic Space

• 如前所述，我们无法将 2D hyperbolic plane 嵌入到 3D 欧氏空间里，但我们能把它嵌入到 3D Minkowski space (a pseudo-Euclidean space and a type of pseudo-Riemannian manifold)

Minkowski Space

• Even though it’s defined as a vector space, we can regard it as basically having $n$ real dimensions (just like ${\mathbb{R}}^n$) but with a special type of metric tensor: the Minkowski metric.
  $$\begin{array}{rlrl}{g}_E(\mathbf{u},\mathbf{v})& =u_1{v}_1+u_2{v}_2+\dots +u_n{v}_n& & \text{Euclidean Metric}\\ g_M(\mathbf{u},\mathbf{v})& =\pm [u_1{v}_1-u_2{v}_2-\dots -u_n{v}_n]& & \text{Minkowski Metric}\end{array}$$Minkowski Metric 前取正负号均可，下文将选取正号。注意到 Minkowski Space 中有一个维度是和其余维度是不一样的，在狭义相对论中，这一维度代表时间，其余维度则代表空间

Hyperboloid (双曲面)

• 双曲线绕其对称轴旋转而生成的曲面即为双曲面。下图主要关注 two sheet hyperboloid
  $$\begin{array}{c}\frac{x^2}{a^2}+\frac{y^2}{b^2}-\frac{z^2}{c^2}=-1\end{array}$$其中 $z>0$ 的部分为 “forward sheet”
  
• two sheet hyperboloid 对应的参数化表示为 (关于双曲函数：可能是最好的讲解双曲函数的文章)
  $$\begin{array}{rl}x& =a\sinh t\cos \theta \\ y& =b\sinh t\sin \theta \\ z& =\pm c\cosh t\end{array}$$

Models for the Hyperbolic Geometry

• 双曲几何有 4 种常见模型：Klein model, Poincaré disk model, Lorentz (hyperboloid/Minkowski) model, 和 Poincaré half-plane model，下面主要介绍 Lorentz model 和 Poincaré disk model

Lorentz Model

Hyperboloid / Minkowski / Lorentz Model

• Hyperboloid Model 是属于 $n$ 维 Hyperbolic Geometry 的模型，模型中的点被表示在 $n+1$ 维 Minkowski space 中的 two-sheeted hyperboloid 的 forward sheet 上
  $$\begin{array}{c}{x}^2+y^2-z^2=-1\text{where }z>0\end{array}$$
• 现在我们可以对双曲面中的点、线、圆等几何概念进行可视化。测地线 (geodesic) 是 curved space 中对直线的推广，即两点间的最短路径，它被定义为 “a curve where you can parallel transport a tangent vector without deformation”. 在双曲面模型中，测地线被定义为由两点 (that define the line) 和原点构成的平面与双曲面相交得到的曲线，因此两点间的最短路径并不符合欧式几何中的直观认知，它是先向下走再向上走，如下图的棕色曲线所示
  
• 我们可以进一步定义两点间的距离，这可以基于 Minkowski metric 对 tangent vectors 的 arc length 积分得到。$\mathbf{u},\mathbf{v}$ 间的距离为
  $$\begin{array}{c}d(\mathbf{u},\mathbf{v})=\text{arcosh}(g_M(\mathbf{u},\mathbf{v}))\end{array}$$其中 $g_M(\mathbf{u},\mathbf{v})=u_1{v}_1-u_2{v}_2-\dots -u_n{v}_n$ 为 Minkowski metric
• 有了距离定义后我们可以继续定义 Hyperboloid Model 中的圆。最简单的圆就是以 $(0,0,1)$ 为圆心的圆，该圆所处的平面平行于 $z=0$ 平面，这种圆和欧氏空间里的圆形状相同。而对于圆心不在 $(0,0,1)$ 的圆，它的圆心就不在中心，而是在偏离中心的某点上，使得圆上各点到圆心的距离相等
  Figure: Visualization of a hyperboloid circle as a “slice” of the forward sheet of a hyperboloid

---

Example: Calculating the Arc Length of a Geodesic In Hyperbolic Space

• 下面计算 hyperbolic plane embedded in 3D Minkowski space 上两点间的弧长。假设两点间的 curve 为
  $$\begin{array}{rl}x& =\sinh t\\ y& =0\\ z& =\cosh t\end{array}$$其中 $t\in [t_a,t_b]$
• 可以直接积分去求弧长
  $$\begin{array}{rl}d(A,B)& ={\int }_{t_a}^{t_b}\sqrt{g_M((\frac{dx(t)}{dt},\frac{dy(t)}{dt},\frac{dz(t)}{dt}),(\frac{dx(t)}{dt},\frac{dy(t)}{dt},\frac{dz(t)}{dt}))}dt\\ & ={\int }_{t_a}^{t_b}\sqrt{(\frac{dz(t)}{dt}{)}^2-(\frac{dx(t)}{dt}{)}^2-(\frac{dy(t)}{dt}{)}^2}dt\\ & ={\int }_{t_a}^{t_b}\sqrt{(\frac{d\cosh t}{dt}{)}^2-(\frac{d\sinh t}{dt}{)}^2}dt\\ & ={\int }_{t_a}^{t_b}\sqrt{{\sinh }^{2}t-{\cosh }^{2}t}dt\\ & ={\int }_{t_a}^{t_b}\sqrt{1}dt\\ & =t_b-t_a\end{array}$$
• 也可以直接用距离公式去求弧长，得到的结果和直接积分相同
  $$\begin{array}{rlrl}d(A,B)& =arcosh\left(g_M\right((\sinh t_a,0,\cosh t_a),(\sinh t_b,0,\cosh t_b),\left)\right)& & \\ & =arcosh(\cosh t_a\cosh t_b-\sinh t_a\sinh t_b)& & \\ & =arcosh(cosh(t_b-t_a))& & \text{hyperbolic identity}\\ & =t_b-t_a& & \end{array}$$

Poincaré Ball Model

• Poincaré Ball Model 是 $n$ 维双曲几何的一种模型，它的点嵌入在 $n$ 维球体中 (or in a circle in the 2D case which is called the Poincaré disk model)
• Poincaré Ball Model 可以直接由 hyperboloid model 通过 stereoscopic projection 导出，就是将 hyperboloid model 上的点投影到 $z=0$ 平面的单位圆上
  投影时，对于 hyperboloid 上的点 $P=(x_1,\dots ,x_n,z)$ 和单位圆上的投影点 $Q=(y_1,\dots ,y_n)$，它们满足如下关系：
  $$\begin{array}{rl}{y}_i& =\frac{x_i}{z+1}\\ (z,x_i)& =\frac{(1+\sum y_i^2,2y_i)}{1-\sum y_i^2}\end{array}$$
• Poincaré disk model (庞加莱圆盘模型). 如果将 hyperboloid 上的测地线投影到单位圆上，我们就得到了 Poincaré disk model 里的直线，它是一条曲线
  注意到，(1) 该曲线无限逼近，但永远不可能到达单位圆的边界。由投影的过程可知，当曲线接近边界时，它就在接近双曲面上的无穷远处；(2) 上一点表明 “distances at the edge of the circle grow exponentially as you move toward the edge of the circle (compared to their Euclidean distances).”；(3) 曲线以 90 度角接近边界；(4) 上图所示的三条曲线虽然看起来彼此发散，但实际上是平行线，这就是更改欧氏几何第五条定理的结果。下图更清晰地展示了这一过程，我们可以找到无数条过一点的线 (black) 与另一条线平行 (blue)
  Figure: Hyperbolic parallel lines that do not intersect on the Poincaré disk.
• Metric tensor of the Poincaré disk model
  注意到，该 metric tensor 随 $y$ 平滑变化，并且 $y$ 离原点越远，计算得到的距离就越大
• Distance between two points $x,y$ on the Poincaré ball (可以根据 Poincaré ball model 和 Hyperboloid model 之间的坐标转换关系，从 Hyperboloid model 的距离计算公式中推出来)
  $$\begin{array}{r}{d}_{\mathbb{D}}(x,y)=\text{arcosh}\left(1+2\frac{\Vert x-y{\Vert }^2}{(1-\Vert x{\Vert }^2)(1-\Vert y{\Vert }^2)}\right)\end{array}$$Poincaré norm 为
  $$\Vert x{\Vert }_{\mathbb{D}}:=d_{\mathbb{D}}(0,x)=2{\tanh }^{-1}(\Vert x\Vert )$$
• 任何单位圆里的 Euclidean circle 都是 Hyperbolic circle，但反直觉的是圆心并不是 Euclidean center，而是位于一个非对称的位置 (除了以 $(0,0)$ 为圆心的圆)

[NeurIPS 2017] Poincaré Embeddings for Learning Hierarchical Representations

• paper: Nickel, Maximillian, and Douwe Kiela. “Poincaré embeddings for learning hierarchical representations.” Advances in neural information processing systems 30 (2017).
• code: https://github.com/facebookresearch/poincare-embeddings
• Implementation by Gensim: https://radimrehurek.com/gensim/models/poincare.html and a jupyter notebook tutorial: https://nbviewer.org/github/RaRe-Technologies/gensim/blob/develop/docs/notebooks/Poincare%20Tutorial.ipynb
• Implemented by “Hyperbolic Entailment Cones for Learning Hierarchical Embeddings”: https://github.com/dalab/hyperbolic_cones
• blog: [NeurIPS 2017] Poincaré Embeddings for Learning Hierarchical Representations

[NeurIPS 2018] Hyperbolic neural networks

• Ganea, Octavian, Gary Bécigneul, and Thomas Hofmann. “Hyperbolic neural networks.” Advances in neural information processing systems 31 (2018).
• code: https://github.com/dalab/hyperbolic_nn
• blog: [NeurIPS 2018] Hyperbolic neural networks

[PMLR 2018] Hyperbolic entailment cones for learning hierarchical embeddings

• Ganea, Octavian, Gary Bécigneul, and Thomas Hofmann. “Hyperbolic entailment cones for learning hierarchical embeddings.” International Conference on Machine Learning. PMLR, 2018.
• code: https://github.com/dalab/hyperbolic_cones
• blog: [PMLR 2018] Hyperbolic entailment cones for learning hierarchical embeddings

References

• Tensors, Tensors, Tensors
• Manifolds: A Gentle Introduction
• Manifold Hypothesis
• 黎曼联络
• Hyperbolic Geometry and Poincaré Embeddings
• 如何简明地解释曲率（curvature）？

---

Further Reading

• Cannon, James W., et al. “Hyperbolic geometry.” Flavors of geometry 31.59-115 (1997): 2.
• Andrews, Ben, and Christopher Hopper. The Ricci flow in Riemannian geometry: a complete proof of the differentiable 1/4-pinching sphere theorem. springer, 2010.
• Alexander, Stephanie. “Michael Spivak, A comprehensive introduction to differential geometry.” (1978): 27-32.