UnityRenderStreaming发送端

看了一下官方的demo,一个基本的发送端需要有RenderStreaming、Broadcast、CameraStreamer、AudioStreaming、InputSystemChannelReceiver(或者WebBrowserInputChannelReceiver)几个基本的脚本,下面搭建一个发送端测试一下.

新建一个场景Custom,由于项目中较多用到UGUI,新建一个UGUI的button,用来响应用户端点击btn,在主相机上挂在这些脚本:

 可以根据需要选择websocket或者http通信,然后把对应的脚本拖拽赋值.

注意,如果接收端是web页面,就使用WebBrowserInputChannelReceiver脚本接收web页面的交互操作.这里先测试exe接收端,使用了InputSystemChannelReceiver.

注意要设置Canvas的渲染模式为ScreenSpace-Camera,并指定推送视频流的相机,这样才能把UI也转换到视频流中.

UGUI的交互,需要用到一个新的EventSystem,新建一个CustomEventSystem脚本,代码如下:

using UnityEngine.EventSystems;
using UnityEngine.InputSystem;
using UnityEngine.InputSystem.LowLevel;

public class CustomEventSystem : EventSystem
{
    protected override void Awake()
    {
        base.Awake();
        unsafe
        {
            InputSystem.onDeviceCommand += InputSystemOnDeviceCommand;
        }
    }

    private static unsafe long? InputSystemOnDeviceCommand(InputDevice device, InputDeviceCommand* command)
    {
        if (command->type != QueryCanRunInBackground.Type)
        {
            // return null is skip this evaluation
            return null;
        }

        ((QueryCanRunInBackground*)command)->canRunInBackground = true;
        return InputDeviceCommand.GenericSuccess;
    }

    protected override void OnApplicationFocus(bool hasFocus)
    {
        //Do not change focus flag on eventsystem
    }
}

 把unity自带的EventSystem移除,挂载CustomEventSystem:

新建一个Custom脚本处理一下用户端的交互:

using Unity.RenderStreaming;
using UnityEngine;
using UnityEngine.InputSystem;
using UnityEngine.UI;

public class Custom : MonoBehaviour
{
    //ugui按钮点击测试
    public Button btn;

    //获取用户端交互 --exe
    private InputSystemChannelReceiver receiver;
    //获取用户端交互 --web
    //private WebBrowserInputChannelReceiver receiver;
    //鼠标交互
    private Mouse m_mouse;
    //触屏交互
    private Touchscreen m_screen;
    //键盘交互
    private Keyboard m_keyboard;
    //鼠标位置
    private Vector2 mousePos;

    private void Awake()
    {
        receiver = GetComponent<InputSystemChannelReceiver>();
        //绑定设备响应事件
        receiver.onDeviceChange += OnDeviceChange;
        btn.onClick.AddListener(OnClickBtn);
    }

    /// <summary>
    /// 点击按钮
    /// </summary>
    private void OnClickBtn ()
    {
        Debug.Log("----------->>>>>>>>>OnClickBtn");
    }

    /// <summary>
    /// 网页端有交互变动
    /// </summary>
    /// <param name="device">交互设备</param>
    /// <param name="change">变化类型</param>
    private void OnDeviceChange (InputDevice device, InputDeviceChange change)
    {
        switch (change)
        {
            //添加设备
            case InputDeviceChange.Added:
                SetDevice(device, true);
                return;
            //移除设备
            case InputDeviceChange.Removed:
                SetDevice(device, false);
                return;
        }
    }
    

    private void SetDevice (InputDevice device, bool add)
    {
        switch (device)
        {
            case Mouse mouse:
                m_mouse = add ? mouse : null;
                break;
            case Keyboard keyboard:
                m_keyboard = add ? keyboard : null;
                break;
            case Touchscreen screen:
                m_screen = screen;
                break;
        }
    }
    /// <summary>
    /// 按下鼠标左键
    /// </summary>
    private void OnLeftMouseBtnPress ()
    {
        Debug.Log("----------->>>>>>>>>OnLeftMouseBtnPress");
    }
    /// <summary>
    /// 按下鼠标左键并拖拽
    /// </summary>
    private void OnLeftMouseBtnPressDrag()
    {
        Debug.Log("----------->>>>>>>>>OnLeftMouseBtnPressDrag");
    }
    /// <summary>
    /// 松开鼠标左键
    /// </summary>
    private void OnLeftMouseBtnRelease()
    {
        Debug.Log("----------->>>>>>>>>OnLeftMouseBtnRelease");
    }

    private void FixedUpdate()
    {
        if (m_mouse != null)
        {
            if (m_mouse.leftButton.wasPressedThisFrame)
            {
                mousePos = m_mouse.position.ReadValue();
                OnLeftMouseBtnPress();
            }
            else if (m_mouse.leftButton.isPressed && mousePos != m_mouse.position.ReadValue())
            {
                mousePos = m_mouse.position.ReadValue();
                OnLeftMouseBtnPressDrag();
            }
            else if (m_mouse.leftButton.wasReleasedThisFrame)
            {
                mousePos = Vector2.zero;
                OnLeftMouseBtnRelease();
            }
        }

        if (m_screen != null)
        {
            Debug.Log("----------->>>>>>>>>m_screen.touches.Count: " + m_screen.touches.Count);
        }
    }
}

这里先写几个比较常用的鼠标点击,拖拽和touch打印一下,发送端搭建完毕,后面再搭建一下用户端,看看能不能跑通.