unity 原型_使用Unity和React快速进行原型制作

unity 原型

使用Unity和React快速进行原型制作 (Rapid Prototyping with Unity and React)

Web applications are great way to quickly reach a lot of users without the hassle of native client installs and play store downloads. Web frameworks such as React allow page components to communicate and respond to user interaction much like traditional applications. Being web-focused, React also offers exceptionally strong support for adaptive and responsive layout, e.g. smoothly re-flowing the page content for a vertically-oriented mobile device or browser window.

Web应用程序是快速吸引大量用户的好方法，而无需麻烦地进行本机客户端安装和播放商店下载。 像React这样的Web框架允许页面组件像传统应用程序一样进行通信并响应用户交互。 以网络为中心，React还为自适应和响应式布局提供了异常强大的支持，例如，平滑地重排页面内容以用于垂直定向的移动设备或浏览器窗口。

However, applications with complex UI and internal state — like a 3D game or simulation — are largely out of scope for React. Instead, such tasks are the domain of game engines like Unity3D. Unity can export WebGL applications which run self-contained game worlds with 3D rendering, physics interactions, and so on. Unity WebGL can also, in principle, communicate with the browser via jslib plug in and react-unity-webgl.

但是，具有复杂UI和内部状态的应用程序(例如3D游戏或模拟)在很大程度上超出了React的范围。 相反，此类任务是Unity3D等游戏引擎的领域。 Unity可以导出WebGL应用程序，这些应用程序可以通过3D渲染，物理交互等运行独立的游戏世界。 原则上，Unity WebGL还可以通过jslib插件和react-unity-webgl与浏览器进行通信。

In theory, then, we can get the best of both worlds for rich interactive web applications by implementing one or more complex worlds in Unity WebGL, then allowing them to communicate with each other via the browser.

从理论上讲，通过在Unity WebGL中实现一个或多个复杂世界，然后允许它们通过浏览器彼此通信，我们可以充分利用丰富的交互式Web应用程序的两个世界。

Unity和React示例项目 (Unity and React Example Project)

Using this technique, we were able to implement a Machine Vision training simulator composed of three distinct panels — two have unique Unity3D WebGL instances — in a React-based page. Each page has two way communication between the instances of Unity and React’s front end.

使用这种技术，我们能够在基于React的页面中实现一个机器视觉训练模拟器，该模拟器由三个不同的面板组成-两个面板具有唯一的Unity3D WebGL实例。 每个页面在Unity实例和React的前端实例之间都有两种通信方式。

Panel #1 displays an assembly line with 3D assets coming down a conveyer belt. Panel #2 handles dataset generation by letting users take snapshots of parts that had been incorrectly classified as they came down the belt. Panel #3 allows players to define ML classes and populate them with snapshots from #2 (a form of Transfer Learning), in order to train the Machine Vision model to better classify parts coming down the belt. The results of this model training are visible in Panel #1, as parts in the next batch are classified under the retrained model.

＃1面板显示一条装配线，其中3D资产从传送带上下来。 第2面板通过让用户拍摄在皮带上被错误分类的零件的快照来处理数据集的生成。 面板＃3允许玩家定义ML类，并使用来自＃2的快照(传递学习的一种形式)填充它们，以便训练机器视觉模型以更好地对沿皮带输送的零件进行分类。 该模型训练的结果在面板＃1中可见，因为下一批零件在重新训练的模型下进行了分类。

Again, “under the hood”, this is all accomplished by embedding two separate instances of Unity WebGL in a single React page, and passing key data (3D asset identifiers, bitmap snapshots, ML model output) back and forth between them in messages.

再次，在“幕后”中，这全部通过将两个单独的Unity WebGL实例嵌入到单个React页面中，并在它们之间在消息中来回传递关键数据(3D资产标识符，位图快照，ML模型输出)来完成。

See a live example of this project here.Also, check out the open source project here.For more information on what inspired this template, check out our article here.

看到这个项目的一个活生生的例子在这里 。还有，看看开源项目在这里 。对于更多的信息是什么激发了模板，看看我们的文章在这里 。

消息集成模板 (A Template for Messaging Integration)

In practice, combining these two frameworks is a bit tricky. Unity provides great documentation on how to communicate with the browser, but no out of the box solution. That’s where cra-template-unity-rect (CTUR) template comes in.’

在实践中，将这两个框架结合起来会有些棘手。 Unity提供了有关如何与浏览器通信的出色文档，但没有现成的解决方案。 这就是cra-template-unity-rect (CTUR)模板的来源。

CTUR is a Create React App template, installed with a single CLI command, that sets up a messaging architecture for Unity WebGL games to send and receive messages to and from the browser. These messages are reusable and customizable.

CTUR是一个Create React App模板，安装了一个CLI命令，该模板为Unity WebGL游戏设置了消息传递体系结构，以在浏览器之间收发消息。 这些消息是可重用和可定制的。

Ultimately, we hope that CTUR will significantly speed up development of web applications that can take advantage of both Unity’s beautiful 3D environments and React’s flexibility and familiarity.

最终，我们希望CTUR将大大加快可利用Unity精美的3D环境以及React的灵活性和熟悉度的Web应用程序的开发。

CTUR is completely open source and can be found here.

CTUR是完全开源的，可以在这里找到。

正在安装 (Installing)

1. Install npx and node

   安装npx和节点

2. Open a terminal and go to a folder where you want to download the template.
   打开终端，然后转到要在其中下载模板的文件夹。

3. Enter the command: npx create-react-app my-unity-react-app --template @carnegie-mellon-robotics-academy/cra-template-unity-react

   输入命令： npx create-react-app my-unity-react-app --template @carnegie-mellon-robotics-academy/cra-template-unity-react

4. See Readme for how to build and develop.

   有关如何构建和开发的信息，请参见自述文件 。

这个怎么运作 (How it works)

We send messages back and forth as serialized objects. Our project follows a basic structure on both sides. Each message has a topic. This topic is always a string. Components can subscribe to these topic in order to receive messages. Messages also have a message object. This object contains a method string and a parameters object.

我们将消息作为序列化对象来回发送。 我们的项目遵循双方的基本结构。 每个消息都有一个主题。 该主题始终是一个字符串。 组件可以订阅这些主题以接收消息。 消息也有一个message对象。 该对象包含method字符串和parameters对象。

{
  topic: "test",
  message:
  {
    method: "startTest",
    parameters:
    {
      i_testTime: 3,
      resolver: "endTest"
    }
  }
}

Unity communicates to the browser using a JS lib plugin. Unity has great documentation on how to do it here. This plugin allows us to communicate to the browser window. You can add other functions as needed. ReactUnityWebGL is an open source package that we used included in the project. It does a nice job of wrapping unity content in a react element.

Unity使用JS lib插件与浏览器通信。 Unity 此处提供有关如何执行此操作的出色文档。 这个插件使我们能够与浏览器窗口进行通信。 您可以根据需要添加其他功能。 ReactUnityWebGL是我们在项目中使用的开源软件包。 它很好地将unity内容包装在react元素中。

MessageUnityOutgoing: function (str) {
  // Send string to ReactUnityWebGL…    
  ReactUnityWebGL.MessageFromUnity(Pointer_stringify(str));
}

Using our unity_api methods, React components can subscribe to topics when it mounts to the DOM. Each method can be imported as needed. This examples shows how to call the method handleTestMessage when Unity sends a message with a topic of “unity_awake”.

使用我们的unity_api方法，React组件可以在安装到DOM时订阅主题。 每种方法都可以根据需要导入。 这个例子展示了当Unity发送主题为“ unity_awake”的消息时如何调用handleTestMessage方法。

const unityAwakeSubscriber = { topic: "unity_awake", callback: handleTestMessage }useEffect(() => {
  addTopicListener(unityAwakeSubscriber);
}, [])

We also provide synchronous and asynchronous methods to Unity. You can uses asynchronous methods to wait for a response from Unity. In this example, a react component starts a test. By passing the resolver as the parameter. Unity will know that this promise is waiting on a message with a topic of “endTest”.

我们还为Unity提供同步和异步方法。 您可以使用异步方法来等待来自Unity的响应。 在此示例中，react组件开始测试。 通过传递解析器作为参数。 Unity将知道此承诺正在等待主题为“ endTest”的消息。

const handleTest = async () => {const msg = { topic: "test", message: { method: "startTest",     parameters: { i_testTime: 3, resolver: "endTest" } } }setTesting(true);await sendUnityMessageAsync(msg, "endTest").then((message) => {
  console.log("Test Ended ", message);
  setTesting(false);
});
}

Unity has a similar way of sending and receiving these messages. All messages from react get passed from MessageDispatcher.cs (MD). MD provides methods similar to unity_api. This example shows how to subscribe to a topic and method from another mono-behavior.

Unity具有发送和接收这些消息的类似方式。 来自react的所有消息均从MessageDispatcher.cs(MD)传递。 MD提供的方法类似于unity_api。 本示例说明了如何从另一个单一行为订阅主题和方法。

messageDispatcher.addMessageListener(new Subscriber("test", startTest));

A subscriber consists of a topic and a callback. The callback startTest must match the method that it expects. Callbacks must receive an object as its parameters. Once you have the parameters, you will need to deserialize them with into the structure it expects.

订户由主题和回调组成。 回调startTest必须与期望的方法匹配。 回调必须接收一个对象作为其参数。 一旦有了参数，就需要将它们反序列化为期望的结构。

private void startTest(object parameters) {
 TestParam testParameters = JsonConvert.DeserializeObject<TestParam>(parameters.ToString());
 
 // Start the test with the commands. Pass the resolver to end the }

Sending messages is also easy. MD has a method to send data through the js lib plugin. It is passed an object which can be null, and a topic. The object will get serialized and sent as a message with the topic.

发送消息也很容易。 MD有一种通过js lib插件发送数据的方法。 它传递了一个可以为null的对象和一个主题。 该对象将被序列化并作为带有主题的消息发送。

messageDispatcher.SendData(null, "endTest");
// message output
// {topic: endTest, message: null}

如何贡献 (How to Contribute)

The main goal of the template is to help Unity and Web developers integrate unity with modern front end frameworks like React. React offers rich GUI affordances and is a convenient middleware layer to the server.

模板的主要目标是帮助Unity和Web开发人员将统一性与诸如React这样的现代前端框架集成在一起。 React提供了丰富的GUI功能，并且是服务器的便捷中间件层。

This template is in its early stages. There are a lot of additional features that we would like to add to the project. These features all come in handy when working with React and Unity.

该模板尚处于初期阶段。 我们要添加到项目中的许多其他功能。 这些功能在使用React和Unity时都非常方便。

These include:

这些包括：

• Support for managing redux.
  支持管理redux。
• Automatic focus for unity windows and keyboard capture
  自动对焦以实现统一的窗口和键盘捕获
• Unity build script to build to “dist” automatically.
  Unity构建脚本可自动构建为“ dist”。
• Easy setup for multiple instances of unity.
  轻松设置多个统一实例。
• Unity editor interface where you can build your own messages.
  Unity编辑器界面，您可以在其中构建自己的消息。
• Support for other frontend frameworks.
  支持其他前端框架。

If you think there are any other great frameworks or want to improve the template please become a contributor. https://github.com/cmroboticsacademy/cra-template-unity-react/

如果您认为还有其他出色的框架或想要改进模板，请成为贡献者。 https://github.com/cmroboticsacademy/cra-template-unity-react/

卡内基梅隆机器人学院 (Carnegie Mellon Robotics Academy)

Carnegie Mellon’s Robotics Academy studies how teachers use robots in classrooms to teach Computer Science, Science, Technology Engineering, and Mathematics (CS-STEM). Our mission is to use the motivational effects of robotics to excite students about science and technology. The Robotics Academy fulfills its mission by developing research-based solutions for teachers that are classroom-tested and foreground CS-STEM concepts.

卡内基·梅隆(Carnegie Mellon)的机器人学院研究教师如何在教室中使用机器人教授计算机科学，科学，技术工程和数学(CS-STEM)。 我们的任务是利用机器人的激励作用激发学生对科学和技术的兴趣。 机器人学院通过为教师开发基于研究的解决方案来完成其任务，这些解决方案经过了课堂测试和CS-STEM的概念验证。

This material is based upon work supported by the National Science Foundation under Grant Number 1937063.

本材料基于美国国家科学基金会在授权号1937063下的支持。

翻译自: https://medium.com/carnegie-mellon-robotics-academy/rapid-prototyping-with-unity-and-react-f3f12b5d7707

unity 原型