摘要音频音标：MotionBERT: A Unified Perspective on Learning Human Motion Representations

音频
We present a unified perspective on tackling various human-centric video tasks by learning human motion representations from large-scale and heterogeneous data resources.
【wi: 'pri:zent ə ju:nə’faɪd 'pɜ:spəktɪv ɑn 'tæklɪŋ 'veəriəs 'hju:mən 'sentrɪk 'vɪdiəʊ tæsks baɪ 'lɜ:nɪŋ 'hju:mən 'məʊʃn reprɪzen’teɪʃnz frɑm lɑ:ʤ’skeɪl ænd he’tərəʤi:niəs 'deɪtə rɪ’sɔ:sɪz.】

Specifically, we propose a pretraining stage in which a motion encoder is trained to recover the underlying 3D motion from noisy partial 2D observations.
【spə’sɪfɪkli, wi: prə’pəʊz ə pri:'treɪnɪŋ steɪʤ ɪn wɪtʃ ə 'məʊʃn 'enkəʊdər ɪz treɪnd tʊ rɪ’kʌvə ðə ʌn’dɜ:laɪɪŋ θri: 'di: 'məʊʃn frɑm 'nɔɪzi 'pɑ:ʃl tu: 'di: ɒbzə’veɪʃnz.】

The motion representations acquired in this way incorporate geometric, kinematic, and physical knowledge about human motion, which can be easily transferred to multiple downstream tasks.
【ðə 'məʊʃn reprɪzen’teɪʃnz ə’kwaɪəd ɪn ðɪs weɪ 'ɪnkɔ:pəreɪt ʤi:ə’metrɪk, kɪ’nemətɪk, ænd 'fɪzɪkl 'nɒlɪʤ əbaʊt 'hju:mən 'məʊʃn, wɪtʃ kən bi: 'i:zɪli træns’fɜ:d tʊ 'mʌltɪpl daʊn’stri:m tæsks. 】

We implement the motion encoder with a Dual-stream Spatio-temporal Transformer (DSTformer) neural network.
【wi: 'ɪmplɪment ðə 'məʊʃn 'enkəʊdər wɪð ə 'dju:əl stri:m 'speɪʃəʊ’tempərəl trænz’fɔ:mə ('di:ɛs’ti:fɔ:mə) 'njʊərəl 'netwɜ:k.】

It could capture long-range spatio-temporal relationships among the skeletal joints comprehensively and adaptively, exemplified by the lowest 3D pose estimation error so far when trained from scratch.
【ɪt kʊd kæptʃə lɒŋ’reɪnʤ 'speɪʃəʊ’tempərəl rɪ’leɪʃnʃɪps ə’mʌŋ ðə 'skelɪtəl ʤɔɪnts kəm’prihensɪvli ænd ə’dæptɪvli, ɪg’zempləfaɪd baɪ ðə 'ləʊɪst θri: 'di: pəʊz estɪ’meɪʃn 'erə səʊ fɑ: wen treɪnd frɑm skrætʃ. 】

Furthermore, our proposed framework achieves state-of-the-art performance on all three downstream tasks by simply finetuning the pretrained motion encoder with a simple regression head (1-2 layers), which demonstrates the versatility of the learned motion representations.
【fɜ:ðɜ’mɔ:, ɑ: prə’pəʊzd 'freɪmwɜ:k ə’ʧi:vz steɪt əv ði 'ɑ:t pɜ:fɔ:məns ɒn ɔ:l θri: 'daʊn’stri:m tæsks baɪ 'sɪmpli faɪn’tju:nɪŋ ðə pri:'treɪnd 'məʊʃn 'enkəʊdər wɪð ə 'sɪmpl rɪ’greʃn hed (wʌn tu: 'leə】

Code and models are available at this URL.