基于webassembly的H265视频播放器前端Angular学习笔记7:在WebWorker中解码H265数据

基于webassembly的H265视频播放器前端Angular学习笔记7:在WebWorker中解码H265数据

本文基于开源项目decoder_wasm使用Angular开发的前端页面项目，对项目整理结构进行的改动，并使用Typescript重写大部分js源码，便于维护。
着重学习Angular框架下Worker，Wasm的使用，本人非前端开发从业人员，页面简陋请见谅。

创建组件与路由

创建组件

$ ng generate component DecoderWorkerTest --skipTests=true
CREATE src/app/decoder-worker-test/decoder-worker-test.component.scss (0 bytes)
CREATE src/app/decoder-worker-test/decoder-worker-test.component.html (34 bytes)
CREATE src/app/decoder-worker-test/decoder-worker-test.component.ts (326 bytes)
UPDATE src/app/app.module.ts (1134 bytes)

设置路由

{ path: 'decoder-worker', component: DecoderWorkerTestComponent },

创建Worker并加载Wasm

详见的之前篇笔记
基于 webassembly 的 H265 视频播放器前端 Angular 学习笔记 6:使用 WebWorker 加载 Wasm
拷贝代码相关逻辑到新的测试组件中

拷贝两个文件
decoder-worker-test.worker.ts
WasmModule.ts

界面拷贝过来

decoder-worker-test.component.html

<div style="margin: 20px; display: inline;">
  H265文件:
  <input type="file" #H265FileInput (change)="play(H265FileInput.files);">
</div>
<div style="margin-top: 10px; display: block; width: 540px; height: 360px; background-color: black;">
  <canvas #playCanvas width="540px" height="360px"></canvas>
</div>

读文件拷贝过来

decoder-worker-test.component.ts

  play(files: Array<File>) {
    console.log(files);
    const file = files[0];
    let filePos = 0;
    let streamSize = 0;
    do {
      const reader = new FileReader();
      reader.onload = (ev: ProgressEvent<FileReader>) => {
        const typedArray: Uint8Array = new Uint8Array(ev.target.result as ArrayBuffer);
        const size = typedArray.byteLength;
		// TODO
      };
      streamSize = this.readFileSlice(reader, file, filePos, CHUNK_SIZE);
      filePos += streamSize;
    } while (streamSize > 0);
  }

  private readFileSlice(reader: FileReader, file: File, startAddr: number, size: number) {
    const fileSize = file.size;
    let fileSlice: Blob;

    if (startAddr > fileSize - 1) {
      return 0;
    }
    else if (startAddr + size > fileSize - 1) {
      // fileSlice = this.blobSlice(file, startAddr, fileSize);
      fileSlice = file.slice(startAddr, fileSize);
      reader.readAsArrayBuffer(fileSlice);
      return fileSize - startAddr;
    }
    else {
      fileSlice = file.slice(startAddr, startAddr + size);
      // fileSlice = this.blobSlice(file, startAddr, startAddr + size);
      reader.readAsArrayBuffer(fileSlice);
      return size;
    }
  }

渲染逻辑拷贝过来

拷贝两个文件
webgl-player.ts
webgl-texture.ts

其实直接import也行，但保证每个测试的代码独立，就拷贝一份方便理解。

  @ViewChild('playCanvas', { static: true }) playCanvas: ElementRef<HTMLCanvasElement>;

ngAfterViewInit(): void {
    console.log(this.playCanvas);
    if (this.playCanvas) {
      this.webglPlayer = new WebGLPlayer(
        this.playCanvas.nativeElement, {}
      );
      console.log(this.webglPlayer);
    }else{
      console.error('canvas error');
    }
  }

准备工作目录结构

$ tree
.
├── Decoder.ts
├── decoder-worker-test.component.html
├── decoder-worker-test.component.scss
├── decoder-worker-test.component.ts
├── decoder-worker-test.worker.ts
├── WasmModule.ts
├── webgl-player.ts
└── webgl-texture.ts

实现Decoder类处理Worker数据交互接口

Common.ts

export const CHUNK_SIZE = 4096;
export const DECODER_H264 = 0;
export const DECODER_H265 = 1;
export const LOG_LEVEL_JS = 0;
export const LOG_LEVEL_WASM = 1;
export const LOG_LEVEL_FFMPEG = 2;

export enum DecoderWorkerTestMessageType {
    ON_INIT, // 组件初始化
    ON_DATA, // H265数据
    ON_DESTROY, // 组件注销
    ON_WASM_LOADED, // wasm加载完成
    ON_DECODE_DATA // 解码数据
}

export interface DecoderWorkerTestMessage {
    type: DecoderWorkerTestMessageType;
    rect?: { width: number, height: number };
    data?: Uint8Array;
}

向worker中发送数据

  play(files: Array<File>) {
    console.log(files);
    const file = files[0];
    let filePos = 0;
    let streamSize = 0;

    const reader = new FileReader();
    reader.onload = (ev: ProgressEvent<FileReader>) => {
      const typedArray: Uint8Array = new Uint8Array(ev.target.result as ArrayBuffer);
      this.postMessage(this.worker, { type: DecoderWorkerTestMessageType.ON_DATA, data: typedArray });
    };
    const interval = window.setInterval(() => {
      streamSize = this.readFileSlice(reader, file, filePos, CHUNK_SIZE);
      filePos += streamSize;
      if (streamSize <= 0) {
        window.clearInterval(interval);
      }
    }, 200);

  }
  private postMessage(worker: Worker, message: DecoderWorkerTestMessage) {
    if (worker) {
      if (message.data) {
        console.log(message);
        worker.postMessage(message, [message.data.buffer]);
      } else {
        worker.postMessage(message);
      }
    }
  }

从worker中接收解码数据

  private handleMessage(message: DecoderWorkerTestMessage) {
    console.log(message);
    switch (message.type) {
      case DecoderWorkerTestMessageType.ON_DECODE_DATA:
        const width = message.rect.width;
        const height = message.rect.height;
        const yLength = width * height;
        const uvLength = (width / 2) * (height / 2);
        if (this.webglPlayer) {
          this.webglPlayer.renderSrcFrame(message.data, width, height, yLength, uvLength);
        } else {
          console.error('webgl init error');
        }
        break;

      default:
        break;
    }

  }

  public runWorker(): void {
    if (typeof Worker !== 'undefined') {
      this.worker = new Worker('./decoder-worker-test.worker', { type: 'module' });
      this.worker.addEventListener<'message'>('message', ({ data }) => {
        const message = data as DecoderWorkerTestMessage;
        this.handleMessage(message);
      });
    }
  }

worker中接收H265数据

  private handleMessage(message: DecoderWorkerTestMessage) {
    console.log(message);
    switch (message.type) {
      case DecoderWorkerTestMessageType.ON_DECODE_DATA:
        const width = message.rect.width;
        const height = message.rect.height;
        const yLength = width * height;
        const uvLength = (width / 2) * (height / 2);
        if (this.webglPlayer) {
          this.webglPlayer.renderSrcFrame(message.data, width, height, yLength, uvLength);
        } else {
          console.error('webgl init error');
        }
        break;

      default:
        break;
    }

  }

  public runWorker(): void {
    if (typeof Worker !== 'undefined') {
      this.worker = new Worker('./decoder-worker-test.worker', { type: 'module' });
      this.worker.addEventListener<'message'>('message', ({ data }) => {
        const message = data as DecoderWorkerTestMessage;
        this.handleMessage(message);
      });
    }
  }

worker中发送H265解码数据

private postMessage(message: DecoderWorkerTestMessage) {
        if (message.data) {
            this.worker.postMessage(message, [message.data.buffer]);
        } else {
            this.worker.postMessage(message);
        }
    }
    
 this.postMessage({ type: DecoderWorkerTestMessageType.ON_DECODE_DATA, rect: { width, height }, data });

测试结果

正常播放视频文件，有不理解的地方可以微信联系我。

微信号：yjkthddx