【瞎鼓捣】web前端全景直播

SusiiaRomon

已于 2022-06-23 10:53:52 修改

阅读量1.6k

点赞数

文章标签： typescript javascript webrtc 传媒实时音视频

于 2022-06-23 10:52:27 首次发布

本文链接：https://blog.csdn.net/SusiiaRomon/article/details/125419979

版权

最近做了一个全景直播的小项目，走了一下大概的流程，特此记录一下。

1.直播服务器

首先利用srs搭建直播服务器，这部分我是跟着官方wiki走的，详细内容请移步

v4_CN_Home · ossrs/srs Wiki (github.com)https://github.com/ossrs/srs/wiki/v4_CN_Home#getting-started

2.全景相机与推流

全景相机我是用的insta360 x2，推流需要配合一台安卓设备，下载官网的应用软件，

全景相机，运动相机 - Insta360影石官网,360度全景运动相机https://www.insta360.com/cn/download/insta360-onex2安装好后将全景相机连接到手机上，有两种连接方法，一种是通过wifi连接，相机开一个热点，手机连接这个热点，然后推流的话走的是手机的流量。另一种方法是通过otg线连接，插上线后手机会弹出个提示框记得确认。

软件打开并连接上相机后按中间黄色的按钮然后右下角三个杠点开，选择直播模式,输入rtmp开头的推流地址，点击红色的点点就开始推流。

3.全景播放与拉流

*本项目基于vue3+ts

创建vue项目，安装所需依赖：

yarn add video.js videojs-contrib-hls @babylonjs/core @babylonjs/inspector @types/video.js

由于视频自动播放现在受到限制，要么静音自动播放，要么需要用户主动触发播放事件，并且在safari上陀螺仪需要用户主动去获取权限，所以我们需要一个加载组件LoadPage.vue，

<template>
  <div>
        <div class="permissionBg" v-show="showPerm">
      <div class="permissionContent">
        <div class="permissionText">为达到最佳用户体验<br>请给予陀螺仪权限</div>
        <div class="permissionConfirm" @click="getGyroscope">好</div>
      </div>
    </div>
    <div class="loadBg">
        <div class="loadButton" @click="logTest">点击开始</div>
    </div>
  </div>
</template>
<script setup lang="ts">
import { defineEmits, ref } from 'vue'
// emit
const emit = defineEmits(['start'])

// 点击开始
const logTest = () => {
  emit('start') // 告诉父组件用户开始了
}
// 获取safari陀螺仪权限
const showPerm = ref(true)
const getGyroscope = () => {
  console.log(1)
  if (window.DeviceOrientationEvent) { // 如果存在DeviceOrientationEvent继续下面
    // eslint-disable-next-line @typescript-eslint/ban-ts-comment
    // @ts-ignore
    if (typeof window.DeviceOrientationEvent.requestPermission === 'function') { // 如果DeviceOrientationEvent上存在requestPermission方法继续
      // eslint-disable-next-line @typescript-eslint/ban-ts-comment
      // @ts-ignore
      window.DeviceOrientationEvent.requestPermission() // 这玩应是safari独有的方法
        .then(permissionState => {
          if (permissionState === 'granted') {
          // handle data
            showPerm.value = false
          } else {
          // handle denied
          }
        })
        .catch((err) => {
          console.log(err)
        })
    } else {
      showPerm.value = false
    }
  } else {
    showPerm.value = false
  }
}
</script>

根组件app.vue中引入LoadPage.vue，

<template>
<LoadPage @start="startLive" v-show="LoadPageShow"></LoadPage>
<canvas ref="babylon" class="canvas"></canvas>
</template>

<script setup lang="ts">
import { onMounted, ref } from 'vue'
import VrLive from './utils/vr-live'
import LoadPage from './components/LoadPage.vue'
// babylonCanvas
const babylonCanvas = ref()
// show LoadPage
const LoadPageShow = ref(true)
let live:VrLive
onMounted(() => {
  live = new VrLive(babylonCanvas.value, 'https://www.zhiyongw.com:5443/hls/abc.m3u8')
})
const startLive = () => {
  live.initBabylon()
  live.video.muted = false // 解除静音
  live.video.play()
  LoadPageShow.value = false // 隐藏loading页面
}
</script>

app.vue中导入的VrLive对象是我们核心功能类，构造函数需要接收一个canvas、一个直播拉流地址，然后需要公开生成的videoElement，（当然你也可以事先写好然后传进来）

项目src目录中新建utils文件夹，创建VrLive.ts到utils中

import videojs from 'video.js'
import 'videojs-contrib-hls'
import { Color4, Engine, Scene, DeviceOrientationCamera, Vector3, Mesh, CreateSphere, VideoTexture, StandardMaterial } from '@babylonjs/core'
import '@babylonjs/inspector'
/**
 *vr直播组件，
 * @param canvas
 *        babylonjs运行所需要的canvas
 *
 * @param videoSrc
 *        视频直播接流地址
 *
 * @return vr直播组件的实例
 */
export default class VrLive {
    private canvas!:HTMLCanvasElement
    private _video!: HTMLVideoElement
    private videoSrc!:string
    private engine!:Engine // Engine
    private scene!:Scene // Scene
    private camera!:DeviceOrientationCamera // camera
    private videoBall!:Mesh
    constructor (canvas:HTMLCanvasElement, videoSrc:string) {
      if (canvas && videoSrc) {
        this.canvas = canvas
        this.video = document.createElement('video')
        this.videoSrc = videoSrc
        this.initVideo()
      }
    }

    // initBabylon
    public initBabylon () {
      this.initEngine(this.canvas)
      this.initVideoBall()
      this.initCamera()
      this.debuggerLayer()
    }

    // 初始化video
    private initVideo () {
      this.creatVideoElement() // 创建视频标签
      if (!this.isIPhone()) {
        const vrVideo = videojs(
          this.video, // id或者videoElement
          { }, // 配置项
          () => {
            vrVideo.play()
          } // 加载完成后的回调
        )
      }
    }

    // 创建视频标签
    private creatVideoElement () {
      // video标签
      this.video.id = 'vrVideo'
      this.video.muted = true
      this.video.autoplay = true
      this.video.src = this.videoSrc
      // videoSorce标签
      const videoSorce = document.createElement('source')
      videoSorce.type = 'application/x-mpegURL'
      videoSorce.src = this.videoSrc
      this.video.appendChild(videoSorce)
      this.video.playsInline = true
      this.video.style.width = '100vw'
      this.video.style.height = '10vh'
      this.video.style.position = 'absolute'
      this.video.style.top = '0'
      this.video.style.left = '0'
      this.video.style.zIndex = '20'
        // 在safari上，虚拟dom是没办法播放出声音的，所以我们需要吧这玩应怼进真实dom中然后用css给它藏起来
      document.getElementsByTagName('body')[0].appendChild(this.video)// 添加进dom
      this.video.style.display = 'none'
    }

    // isIPhone
    private isIPhone ():boolean {
      const u = navigator.userAgent
      return !!u.match(/\(i[^;]+;( U;)? CPU.+Mac OS X/) || u.indexOf('iPhone') > -1 || u.indexOf('iPad') > -1 // ios终端
    }

    // 初始化引擎
    private async initEngine (canvas: HTMLCanvasElement): Promise<void> {
      this.canvas = canvas // 初始化canvas
      this.engine = new Engine(
        this.canvas,
        true,
        {},
        true
      )
      this.scene = new Scene(this.engine)// 初始化场景
      //   this.scene.clearColor = new Color4(0.784, 0.878, 1, 1)
      this.scene.clearColor = new Color4(1, 1, 1, 1)
      const assumedFramesPerSecond = 60
      const earthGravity = -9.34
      this.scene.gravity = new Vector3(0, earthGravity / assumedFramesPerSecond, 0) // 设置场景重力(模拟地球)
      this.scene.collisionsEnabled = true // 开启碰撞检测
      this.RenderLoop()// 执行渲染循环
    }

    // 渲染循环
    private RenderLoop ():void {
    // 添加窗口变化事件监听
      window.addEventListener('resize', () => {
        this.engine.resize()
      })
      // 执行循环
      this.engine.runRenderLoop(() => {
        this.scene.render()
      })
    }

    // debuggerLayer（Shift+Ctrl+Alt+I）
    private debuggerLayer ():void {
      window.addEventListener('keydown', (ev) => {
        if (ev.shiftKey && ev.ctrlKey && ev.altKey && ev.keyCode === 73) {
          if (this.scene.debugLayer.isVisible()) {
            this.scene.debugLayer.hide()
          } else {
            this.scene.debugLayer.show(
              { embedMode: true }
            )
          }
        }
      })
    }

    // 初始化相机
    private initCamera ():void {
      this.camera = new DeviceOrientationCamera(
        'vrLiveCamera',
        new Vector3(0, 0, 0),
        this.scene
      )
      // eslint-disable-next-line dot-notation
      this.camera.inputs.remove(this.camera.inputs.attached['keyboard'])
      this.camera.attachControl(this.canvas, true)
      this.camera.fov = 1.5
      this.camera.fovMode = DeviceOrientationCamera.FOVMODE_HORIZONTAL_FIXED
    }

    // 初始化视频球
    private initVideoBall ():void {
      // mesh
      this.videoBall = CreateSphere(
        'VideoBall',
        {
          diameter: 5,
          segments: 32,
          sideOrientation: Mesh.DOUBLESIDE
        },
        this.scene
      )
      // texture
      const videoTexture = new VideoTexture(
        'video',
        this.video,
        this.scene,
        false,
        true,
        VideoTexture.TRILINEAR_SAMPLINGMODE,
        {
          autoPlay: true,
          autoUpdateTexture: true
        }
      )
      // material
      const videoMat = new StandardMaterial('videoMat', this.scene)
      videoMat.diffuseTexture = videoTexture
      videoMat.emissiveTexture = videoTexture
      this.videoBall.material = videoMat
    }

    /** setter&getter**/

    // videoElement
    public get video (): HTMLVideoElement {
      return this._video
    }

    public set video (value: HTMLVideoElement) {
      this._video = value
    }
}

我编译了一份es5版本的js文件，有其他版本需要的话去ts官网playground中自行编译

var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {
    function adopt(value) { return value instanceof P ? value : new P(function (resolve) { resolve(value); }); }
    return new (P || (P = Promise))(function (resolve, reject) {
        function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
        function rejected(value) { try { step(generator["throw"](value)); } catch (e) { reject(e); } }
        function step(result) { result.done ? resolve(result.value) : adopt(result.value).then(fulfilled, rejected); }
        step((generator = generator.apply(thisArg, _arguments || [])).next());
    });
};
var __generator = (this && this.__generator) || function (thisArg, body) {
    var _ = { label: 0, sent: function() { if (t[0] & 1) throw t[1]; return t[1]; }, trys: [], ops: [] }, f, y, t, g;
    return g = { next: verb(0), "throw": verb(1), "return": verb(2) }, typeof Symbol === "function" && (g[Symbol.iterator] = function() { return this; }), g;
    function verb(n) { return function (v) { return step([n, v]); }; }
    function step(op) {
        if (f) throw new TypeError("Generator is already executing.");
        while (_) try {
            if (f = 1, y && (t = op[0] & 2 ? y["return"] : op[0] ? y["throw"] || ((t = y["return"]) && t.call(y), 0) : y.next) && !(t = t.call(y, op[1])).done) return t;
            if (y = 0, t) op = [op[0] & 2, t.value];
            switch (op[0]) {
                case 0: case 1: t = op; break;
                case 4: _.label++; return { value: op[1], done: false };
                case 5: _.label++; y = op[1]; op = [0]; continue;
                case 7: op = _.ops.pop(); _.trys.pop(); continue;
                default:
                    if (!(t = _.trys, t = t.length > 0 && t[t.length - 1]) && (op[0] === 6 || op[0] === 2)) { _ = 0; continue; }
                    if (op[0] === 3 && (!t || (op[1] > t[0] && op[1] < t[3]))) { _.label = op[1]; break; }
                    if (op[0] === 6 && _.label < t[1]) { _.label = t[1]; t = op; break; }
                    if (t && _.label < t[2]) { _.label = t[2]; _.ops.push(op); break; }
                    if (t[2]) _.ops.pop();
                    _.trys.pop(); continue;
            }
            op = body.call(thisArg, _);
        } catch (e) { op = [6, e]; y = 0; } finally { f = t = 0; }
        if (op[0] & 5) throw op[1]; return { value: op[0] ? op[1] : void 0, done: true };
    }
};
import videojs from 'video.js';
import 'videojs-contrib-hls';
import { Color4, Engine, Scene, DeviceOrientationCamera, Vector3, Mesh, CreateSphere, VideoTexture, StandardMaterial } from '@babylonjs/core';
import '@babylonjs/inspector';
/**
 *vr直播组件，
 * @param canvas
 *        babylonjs运行所需要的canvas
 *
 * @param videoSrc
 *        视频直播接流地址
 *
 * @return vr直播组件的实例
 */
var VrLive = /** @class */ (function () {
    function VrLive(canvas, videoSrc) {
        if (canvas && videoSrc) {
            this.canvas = canvas;
            this.video = document.createElement('video');
            this.videoSrc = videoSrc;
            this.initVideo();
        }
    }
    // initBabylon
    VrLive.prototype.initBabylon = function () {
        this.initEngine(this.canvas);
        this.initVideoBall();
        this.initCamera();
        this.debuggerLayer();
    };
    // 初始化video
    VrLive.prototype.initVideo = function () {
        this.creatVideoElement(); // 创建视频标签
        if (!this.isIPhone()) {
            var vrVideo_1 = videojs(this.video, // id或者videoElement
            {}, // 配置项
            function () {
                vrVideo_1.play();
            } // 加载完成后的回调
            );
        }
    };
    // 创建视频标签
    VrLive.prototype.creatVideoElement = function () {
        // video标签
        this.video.id = 'vrVideo';
        this.video.muted = true;
        this.video.autoplay = true;
        this.video.src = this.videoSrc;
        // videoSorce标签
        var videoSorce = document.createElement('source');
        videoSorce.type = 'application/x-mpegURL';
        videoSorce.src = this.videoSrc;
        this.video.appendChild(videoSorce);
        this.video.playsInline = true;
        this.video.style.width = '100vw';
        this.video.style.height = '10vh';
        this.video.style.position = 'absolute';
        this.video.style.top = '0';
        this.video.style.left = '0';
        this.video.style.zIndex = '20';
        this.video.controls = true;
        document.getElementsByTagName('body')[0].appendChild(this.video); // 添加进dom
        this.video.style.display = 'none';
    };
    // isIPhone
    VrLive.prototype.isIPhone = function () {
        var u = navigator.userAgent;
        return !!u.match(/\(i[^;]+;( U;)? CPU.+Mac OS X/) || u.indexOf('iPhone') > -1 || u.indexOf('iPad') > -1; // ios终端
    };
    // 初始化引擎
    VrLive.prototype.initEngine = function (canvas) {
        return __awaiter(this, void 0, void 0, function () {
            var assumedFramesPerSecond, earthGravity;
            return __generator(this, function (_a) {
                this.canvas = canvas; // 初始化canvas
                this.engine = new Engine(this.canvas, true, {}, true);
                this.scene = new Scene(this.engine); // 初始化场景
                //   this.scene.clearColor = new Color4(0.784, 0.878, 1, 1)
                this.scene.clearColor = new Color4(1, 1, 1, 1);
                assumedFramesPerSecond = 60;
                earthGravity = -9.34;
                this.scene.gravity = new Vector3(0, earthGravity / assumedFramesPerSecond, 0); // 设置场景重力(模拟地球)
                this.scene.collisionsEnabled = true; // 开启碰撞检测
                this.RenderLoop(); // 执行渲染循环
                return [2 /*return*/];
            });
        });
    };
    // 渲染循环
    VrLive.prototype.RenderLoop = function () {
        var _this = this;
        // 添加窗口变化事件监听
        window.addEventListener('resize', function () {
            _this.engine.resize();
        });
        // 执行循环
        this.engine.runRenderLoop(function () {
            _this.scene.render();
        });
    };
    // debuggerLayer（Shift+Ctrl+Alt+I）
    VrLive.prototype.debuggerLayer = function () {
        var _this = this;
        window.addEventListener('keydown', function (ev) {
            if (ev.shiftKey && ev.ctrlKey && ev.altKey && ev.keyCode === 73) {
                if (_this.scene.debugLayer.isVisible()) {
                    _this.scene.debugLayer.hide();
                }
                else {
                    _this.scene.debugLayer.show({ embedMode: true });
                }
            }
        });
    };
    // 初始化相机
    VrLive.prototype.initCamera = function () {
        this.camera = new DeviceOrientationCamera('vrLiveCamera', new Vector3(0, 0, 0), this.scene);
        // eslint-disable-next-line dot-notation
        this.camera.inputs.remove(this.camera.inputs.attached['keyboard']);
        this.camera.attachControl(this.canvas, true);
        this.camera.fov = 1.5;
        this.camera.fovMode = DeviceOrientationCamera.FOVMODE_HORIZONTAL_FIXED;
    };
    // 初始化视频球
    VrLive.prototype.initVideoBall = function () {
        // mesh
        this.videoBall = CreateSphere('VideoBall', {
            diameter: 5,
            segments: 32,
            sideOrientation: Mesh.DOUBLESIDE
        }, this.scene);
        // texture
        var videoTexture = new VideoTexture('video', this.video, this.scene, false, true, VideoTexture.TRILINEAR_SAMPLINGMODE, {
            autoPlay: true,
            autoUpdateTexture: true
        });
        // material
        var videoMat = new StandardMaterial('videoMat', this.scene);
        videoMat.diffuseTexture = videoTexture;
        videoMat.emissiveTexture = videoTexture;
        this.videoBall.material = videoMat;
    };
    Object.defineProperty(VrLive.prototype, "video", {
        /** setter&getter**/
        // videoElement
        get: function () {
            return this._video;
        },
        set: function (value) {
            this._video = value;
        },
        enumerable: false,
        configurable: true
    });
    return VrLive;
}());
export default VrLive;

4.开发调试

调试过程中如果服务器和前端项目不在同一个域名下，拉流会产生跨域问题，所以开发过程中我在浏览器上安装了Allow CORS: Access-Control-Allow-Origin插件，可以用来解除浏览器跨域限制，手机预览可用fiddler处理跨域后开代理，手机wifi连接代理进行访问。具体参考

Fiddler解决跨域问题https://blog.csdn.net/weixin_43409011/article/details/114121360

Fiddler APP抓包手机代理设置https://blog.csdn.net/weixin_43145997/article/details/123594342

5.发布

将项目build好的dist文件夹内容放入之前搭建的nginx服务器下html目录中即可访问，也可以另外搭建服务器后通过nginx配置转发，需要保证客户端访问页面与拉流地址在同一域名端口下即可。