shoebox_Shoebox-我的虚拟手绘，手工编码现场乐队

shoebox

by Michael Forrest

通过迈克尔·福雷斯特(Michael Forrest)

Shoebox-我的虚拟手绘，手工编码现场乐队 (Shoebox — my virtual hand-drawn, hand-coded live band)

I wrote this song and then made this realtime animation engine for “virtual live performances” so my song could be played by some funny stylized characters. I hand-coded and hand-drew just about every element during this 12 month project. The video above is the first render of the results, but I have designed the system so that different songs and animations can be used without too much trouble. It’s mostly CoffeeScript and three.js.

我写了这首歌，然后制作了用于“虚拟现场表演”的实时动画引擎，以便可以用一些有趣的程式化角色来播放我的歌。 在这个12个月的项目中，我几乎对每个元素都进行了手工编码和绘制。 上面的视频是结果的第一个渲染，但我已经设计了该系统，以便可以使用不同的歌曲和动画而不会带来太多麻烦。 主要是CoffeeScript和three.js。

研究+布置舞台 (Research + laying out the stage)

First looked at a few live performances on YouTube to see how they were lit and shot.

首先查看YouTube上的一些现场表演，看看它们是如何被照亮和拍摄的。

I listened to all the songs on my (not yet released) album to work out what instrumentation I’d need if a band had played them, and laid out a stage with four areas — a singer (me), drummer, bass player and synth/keys. There was an organ in the back.

我听了我专辑(尚未发行)中的所有歌曲，以弄清楚如果一支乐队演奏了它们，我需要什么乐器，并安排了一个具有四个区域的舞台-歌手(我)，鼓手，贝斯手和合成器/键。 后面有个器官。

This stage started feeling a bit too big so I decided to make a smaller one for my first videos.

这个阶段开始感觉有点太大，所以我决定为我的第一个视频制作一个较小的阶段。

仪器建模 (Modelling the instruments)

I defined most of my models as a single folded sheet with two transparent “cheeks” on each side.

我将大多数模型定义为一个折叠的表，每侧有两个透明的“颊板”。

This meant I could describe an instrument with a small amount of code.

这意味着我可以用少量代码来描述一种仪器。

I wanted these things to be printable so I created detailed vector artwork for every instrument. Here’s a close up of the modular synth artwork.

我希望这些东西是可打印的，所以我为每种乐器创建了详细的矢量插图。 这是模块化合成器艺术品的特写。

Here’s a guitar.

这是吉他。

As you can see everything is hand-drawn and hand-coded. I’m not mucking about with .dae files or binary file formats (well — apart from the images…).

如您所见，一切都是手工绘制和手工编码的。 我不是在嘲笑.dae文件或二进制文件格式(嗯-除了图像…)。

All these textures went into a big hand-made texture atlas.

所有这些纹理都变成了大型的手工纹理图集。

I described the coordinate offset for each element in the atlas and used the same data to map the textures onto the 3d models.

我描述了地图集中每个元素的坐标偏移，并使用相同的数据将纹理映射到3d模型上。

乐队建模 (Modelling the band)

My sister made me a little cardboard cutout a few years ago so I started with her artwork.

几年前，姐姐给我做了一个小纸板切口，所以我从她的作品开始。

I decided to make my band up of three other characters: a frog on synths, a mandrill called Barry on bass and a Minecraft cow on drums.

我决定组成另外三个角色：合成器上的青蛙，低音上的叫Barry的山rill和鼓上的Minecraft牛。

I think Barry’s feet gave me the most trouble

我认为巴里的脚给我带来了最大的麻烦

I constructed these characters with a minimum of code specifying their layout — they’re just box primitives really.

我用最少的代码来指定这些字符的布局来构造这些字符-实际上，它们只是盒形图元。

I attached face layers afterwards — note the transparent areas above and below to allow protruding beards and suchlike.

之后，我附加了面部图层-注意上方和下方的透明区域，以留出胡须等。

The four band member textures are in one atlas and the only thing different between each band member is its vertical offset in the atlas.

四个带状部件纹理位于一个图集中，并且每个带状部件之间唯一的不同是其在图集中的垂直偏移。

灯火 (Lights)

To make a compelling stage show I first needed to learn how to light a stage. I did some research and picked a style and some colors.

为了制作引人入胜的舞台表演，我首先需要学习如何点亮舞台。 我做了一些研究，选择了款式和一些颜色。

I modelled the lighting enclosures and rigging. Wasn’t too much work when I based it on a lathe primitive.

我对照明设备的外壳和装配进行了建模。 当我基于车床图元时，它工作并不多。

绘制一首歌 (Mapping out a song)

I needed to map out a song structure to define all the camera movements, lighting and animations I wanted.

我需要绘制一个歌曲结构来定义我想要的所有相机运动，灯光和动画。

I wrote scripts to convert the data from this spreadsheet into something I could work with in code. Here’s the final data file for the arrangement. There’s not much to it considering how much work it’s doing.

我编写了脚本，将来自此电子表格的数据转换为可以在代码中使用的东西。 这是安排的最终数据文件。 考虑到正在执行的工作量并没有太多。

This is essentially all the data required to render a performance of one song.

本质上，这是表现一首歌的表演所需的全部数据。

动画乐队 (Animating the band)

I wanted minimal data but natural movement. I thought I’d need a multi-touch tool to capture animations but once I started defining them I realized I could pose each part individually to bring the animations together step-by-step. Here’s a video of me quickly demonstrating my animation tool.

我想要最少的数据，但要自然移动。 我以为我需要一个多点触控工具来捕获动画，但是一旦开始定义它们，我意识到我可以单独摆放每个零件，以逐步将动画组合在一起。 这是我快速演示动画工具的视频。

I saved these animations to a database and generated a JSON file that could be referenced by the front-end code.

我将这些动画保存到数据库中，并生成了一个前端文件可以引用的JSON文件。

Controllers

控制器

There are three main controllers responsible for a performance — Camera, Animation and Lighting.

负责表演的三个主要控制器是：相机，动画和灯光。

The camera controller interprets the different “shot” types into spatial coordinates and animates the camera based on the minimal info provided.

摄像机控制器将不同的“镜头”类型解释为空间坐标，并根据提供的最少信息为摄像机设置动画。

For example, here’s how a camera ‘orbit’ movement is described.

例如，这是描述相机“轨道”运动的方式。

So we start from bar 69, and over the duration of 8 bars we orbit from michael's front-left to front/right/above, staying 1000 units away. (I picked my units based on pixel measurements to make certain things easier but it would have been better with hindsight to use metres).

因此，我们从第bar 69开始，在8 bars的持续时间内，我们从michael的front-left到front/right/above旋转，并保持1000单位的距离。 (我根据像素测量值选择了单位，以使某些事情变得容易，但事后考虑使用米会更好)。

The lighting controller had a “track” for each light (or logical grouping of lights) and then each of these was defined separately in the configuration file. Here’s what the configuration looked like for the backwash light — I just wanted do define the bar numbers and colors or color transitions.

照明控制器为每个灯光(或灯光的逻辑分组)创建一个“轨道”，然后在配置文件中分别定义每个灯光。 这是反洗灯的配置外观-我只是想定义条码编号和颜色或颜色过渡。

My animation tool let me create named animation loops. I referenced these in my configuration file by name and then used this to pick the right animation for each character during the song.

我的动画工具允许我创建命名的动画循环。 我按名称在配置文件中引用了这些名称，然后使用它为歌曲中的每个角色选择了正确的动画。

Here’s Barry’s animations for the whole song:

这是巴里整首歌的动画：

So, taking the second item as an example, we use the animation named “barry chorus” for 16 bars starting from bar 25. We specify that he will hold the bassGuitar for these animations. Being played by somebody is a function of the instrument so the guitar knows where it needs to move to be played by Barry.

因此，以第二个项目为例，我们使用名为“barry chorus”的动画从第25小节开始的16小节。我们指定他将为这些动画保存bassGuitar 。 有人弹奏是乐器的功能，因此吉他知道巴里弹奏的位置。

GL着色器 (GL Shaders)

I wanted the video to have some decent cinematography and the most important camera effect to me was to have some depth of field effects in the shot (i.e. things in the background and foreground should be out of focus).

我希望视频具有不错的摄影效果，而对我来说，最重要的相机效果是在镜头中具有一定的景深效果(即，背景和前景中的物体应不清晰)。

My shaders are pretty hacky but the gist of it is that we first generate a depth map so we know how far everything in the scene is from the camera. Then we use this information to blur areas of the image accordingly.

我的着色器非常笨拙，但要点是我们首先生成了一个深度图，因此我们知道场景中的所有物体离相机有多远。 然后，我们使用此信息来相应地模糊图像的区域。

The other important effect was the volumetric lighting. The lights needed to look like they were shining through smoke for it to feel like a gig. Here’s the rough extent of the shader technology in my scene. (The names are a bit off — and note that I ended up with multiple “fake suns” so that each light had its own).

另一个重要的效果是体积照明。 灯光需要看起来像它们在烟雾中闪耀，才能像演出一样。 这是我场景中的着色器技术的大致范围。 (名称有些偏离-请注意，我最终得到了多个“假太阳”，因此每个灯光都有自己的灯光)。

听众 (The audience)

I’d originally created a system where my Twitch followers would be my audience (with face detection to put their faces on bodies) but this didn’t feel quite right for a YouTube video so I went to Cubeecraft.com, the original inspiration for this whole style. I picked out anything that had resonance for me including Spencer and Watchmen characters. These are all hand-made so I filtered them down to ones with similar layouts and found I could copy and paste slices between characters in Sketch to quickly generate files.

我最初创建的系统中，我的Twitch追随者将是我的听众(通过面部检测将他们的脸贴在身体上)，但是对于YouTube视频而言，这感觉不太正确，所以我去了Cubeecraft.com，这是最初的灵感来源这整个风格。 我挑选出任何对我有共鸣的东西，包括斯宾塞和守望者角色。 这些都是手工制作的，因此我将它们过滤为具有相似布局的布局，发现可以在Sketch中的角色之间复制和粘贴切片以快速生成文件。

I used TexturePacker to generate the textures. Probably could have saved myself a lot of time earlier by doing the same instead of hand-coding each vertex mapping.

我使用TexturePacker生成纹理。 通过执行相同的操作而不是手动编码每个顶点映射，可能可以节省很多时间。

渲染图 (Rendering)

I used a couple of things and settled on CCapture to make 4K (non-realtime) renders — it took about half an hour to render the final video. I synced this up with the audio again in Apple Motion and used Final Cut Pro to add the intro and outtro.

我用了几件事，决定使用CCapture制作4K(非实时)渲染-渲染最终视频花了大约半个小时。 我在Apple Motion中再次将其与音频同步，并使用Final Cut Pro添加了前奏和后奏。

结果(再次) (The result (again))

视频演示之外的功能 (Features beyond the video demo)

This video render is just one possible output. I rendered out a 360 video (although I rushed it and didn’t managed to get the volumetric lighting to work so it doesn’t really look right).

此视频渲染只是一种可能的输出。 我渲染了一个360度视频(尽管我匆匆忙忙，但未能设法使体积照明正常工作，因此看起来效果不佳)。

I have this thing hosted on a server but I’m not quite ready to release it until I figure out a good way to make it interactive. I can plug my realtime performances into it so that it syncs up to my semi-improvised live shows. It runs on my iPhone so it works nicely with Google Cardboard style headsets.

我把这个东西托管在服务器上，但是直到我想出一种使它具有交互性的好方法之前，我还没有准备好发布它。 我可以将我的实时表演插入其中，以便将其同步到半即席现场表演。 它可以在我的iPhone上运行，因此可以很好地与Google Cardboard风格的耳机配合使用。

I modeled other instruments beyond the ones you see here and I have a lot of sound-reactive stuff that I haven’t shown yet (which, incredibly, also seems to work on mobile). One of the hardest things to do was keep scope under control as I realized more and more possibilities for this system.

除了您在此处看到的那些乐器以外，我还对其他乐器进行了建模，并且我还有很多声音React性的东西尚未显示(令人难以置信的是，它似乎也可以在移动设备上使用)。 当我意识到该系统越来越多的可能性时，最困难的事情之一就是控制范围。

有什么问题吗 (Questions?)

I’ve really rushed through this explanation — I wanted to start by pointing out the major elements before going into any more detail about each thing.

我确实急于完成这个解释-我想先指出主要要素，然后再详细介绍每件事。

What would you like to hear more about? I’m planning on going into more depth in a video series in future so it would be good to understand what people want to learn more about.

您想进一步了解什么？ 我计划在将来的视频系列中更深入地介绍，所以最好了解人们想了解的更多信息。

If you’re inspired by this project’s potential as a product. Personally I can imagine it being an accessible type of music publishing medium living somewhere between recordings and live shows, especially if you plugged it into services like Twitch.

如果您受此项目作为产品潜力的启发。 我个人可以想象它是一种可访问的音乐发布媒体，介于唱片和现场表演之间，特别是如果您将其插入Twitch之类的服务中。

喜欢这首歌吗？ (Liked the song?)

If you liked the music, come and find me on Facebook, Twitter and/or SoundCloud.

如果您喜欢音乐，请在Facebook ， Twitter和/或SoundCloud上找到我。

翻译自: https://www.freecodecamp.org/news/shoebox-my-virtual-hand-drawn-hand-coded-live-band-454368d0e66f/

shoebox