How to Make a VR Game With Unity and Google Cardboard

From Here

Update note: This tutorial was updated in May 2016 for the new Google Cardboard SDK, which is now called theGoogle VR SDK. If you completed this tutorial previously, and want to migrate over to the new SDK, refer to the Google VRrelease notes.

Welcome… to the THIRD DIMENSION!

Transform a boring ol’ top-down game (top) to an exciting VR game (bottom)!

It’s probably safe to say that everybody has been waiting for virtual reality experiences since that classic of modern cinema, “Lawnmower Man”, promised us a delightful future of, um, virtual reality where we could all have telepathic powers. Or something like that.

In this tutorial, you’ll help humanity prepare for its future of telepathic abilities and virtual reality by learning to take a simple Unity game and make it work as a VR experience using Google Cardboard. You’ll find out how to:

• Integrate a VR camera into your projects
• Modify UI elements so that they work in VR mode
• Make buttons selectable in VR mode
• Switch programmatically between regular and VR mode at run time

Note: At the time of writing, mind control and becoming a super human through VR is not a viable technology, in spite of what “Lawnmower Man” showed us. So, until Apple has a mind control API, just stick with mastering its precursors.

What is Google Cardboard?

In theory, creating a VR experience is straightforward. Instead of displaying your world as a single image on screen, you display two images.

They come from two cameras placed a few inches apart, and the user views the image from the left camera with the left eye and vice versa, creating the appearance of depth.

Additionally, with some judicious use of motion sensors, you can detect the direction the user is facing. Combine that with the 3D world you’ve created, and you’ve got yourself an immersive experience.

In practice, it takes some pretty sophisticated hardware to display two images on a high-definition screen, along with tracking the user’s head, and putting that all into a device small and light enough that it doesn’t hurt your neck.

Well, it turns out that such highly advanced technology is probably in your pocket or sitting within arm’s reach. It’s in almost every smartphone, and that’s the idea behind Cardboard; it can take the display and sensors in your phone and turn them into a respectable VR device, using just some cardboard and plastic lenses.

Getting Started

Note: You’re going to be working with the Unity GUI quite a bit. If you’ve never worked with it before, you might want to take a look at ourUnity GUI tutorial.

To get started making your own VR game with Google Cardboard, you’re going to need the following:

• Unity Personal Edition, version 5.x
• A smartphone, either an iPhone 5 or later or an Android phone running Jelly Bean or later on which to test. By the way, this tutorial will assume you’re working on an iPhone.
• Oh, yeah, and a Cardboard unit.

Note: If you are new Unity and you are unfamiliar with the interface, you can get up to speed by reading the Introduction to Unity tutorial found on this site.

Wait… How Do I Get a Cardboard Unit?

Don’t have a Cardboard unit? Your best bet is to order it from any of the vendors listedhere. It should cost you around $20 or $30, plus shipping. If you’re feeling like a DIY superstar, you could even make your own.

When buying a Cardboard device, look for ones that mention “V2” or “Cardboard 2.0”, because they fit a larger variety of phones, including big phones like the iPhone 6+. They also support user input with a button that’s covered in conductive metal tape that taps the phone’s screen when you press down on it.

Can I Do this Tutorial Without a Cardboard Unit?

Well, kinda. You’ll be able to run your game on your phone, and it’ll look like this.

You can approximate a 3D experience if you stare past the screen just right while playing. If you move your phone around, you’ll be able to control it just as though your phone were strapped to your head, and you can simulate clicks by tapping the screen.

Although you could play the game and get a sense of what it would be like if you were viewing it through a VR headset, it doesn’t mean youshould. It’s kind of like the difference between eating chocolate and listening to a friend describe to you how chocolate tastes. Sure, you’ll get the general idea, but it just won’t be the same.

Long story short: if you’re too impatient to wait for that Cardboard unit to arrive, you can still learn from this tutorial, but you’ll get a lot more out of it with the right equipment.

The Sample Game — Ninja Attack is Back!

Take a moment to try out your sample game. Download and unzip the Unity starter project.

Next, start up Unity. From the welcome screen, select Open and then theStarterNinja folder to open the NinjaAttack project.

In the Project Browser, double-click MainScene in Assets. Then give the game a try by pressingPlay.

You’re the ninja on the left. As monsters make their way across the screen, click anywhere on the screen to launch a ninja star and take ’em out! Take out 20 monsters and you win, but if a monster reaches the red zone on the left, you lose.

The blobs are at it again. Will they ever learn?

Does this game look familiar? For those of you who are long-time readers of raywenderlich.com, you might recognize this as the same ninja game from yourSpriteKit and Cocos2D tutorials. But this time it’s rendered in glorious 3D!

Not that you’ll really notice most of that glorious dimension though. The entire game is top-down. It almost feels like a waste rendering all these polygons. Now you’re seeing why this game is the perfect candidate for VR.

Getting Started with The Google VR SDK

The first thing you need to do is download the SDK. Head on over to the Google VR Downloads page, click on the “Agree” button if you see one, and download the SDK. You can do this either by cloning the repo (if you’re intogit), or clicking the “download the repo directly” link (if you’re not).

Next, import it into your project. From Unity’s menu, select Assets\Import Package\Custom Package… and then select theGoogleVRForUnity.unitypackage from within the repo you just downloaded.

Make sure everything is selected, uncheck the DemoScenes folder, and then click theImport button.

Hack it Like It’s Hot

To get your game working as a VR experience, you need to perform a few quick ‘n dirty hacks.

From the GoogleVR\Prefabs folder in the Project Browser, drag the GvrMain Prefab into your scene. In the inspector, give it almost the same Position as your ninja main character — (-5.53, 1.13, 0.122) — and aY Rotation of90.

You’ll notice it’s a little bit higher than the ninja’s center to represent the fact that you’re looking through his eyes.

Next, select the Main Camera in the hierarchy and uncheck it in the inspector to disable it. Do the same thing for theraccoon-ninja object.

Now, run your game in the Unity editor again. You should see something resembling a 3D scene! And if you hold down the option key while moving your mouse around, your camera will move as if you were turning your head.

Running Your Scene on an iOS Device

It’s great to be able to run your game in the Unity editor, but last time I checked, there was no way to fit a computer monitor inside a VR headset without some serious pain, hence why you’re also working on an iPhone.

• Select File\Build Settings — iOS should already be selected as your default platform.
• Click Player Settings and switch to the inspector
• Under Resolution and Presentation, change the Default Orientation toLandscape Left.
• In Other Settings, change your Bundle Identifier to be something appropriate for your organization. (Likecom.<your_company>.NinjaAttackVR)
• Change your Target Device to iPhone Only

Attach your iPhone to your computer, select Build and Run and give your export folder a name; it can be anything you’d like.

Unity will export your project, and then it should automatically open up in Xcode. If it doesn’t, start up Xcode and manually open the generated project. Run it, and try it out on your phone!

The first time you run your game, it takes you through a setup process where you can scan a QR code on your Cardboard unit. This lets the Google VR SDK make a few minor graphical adjustments based on how far apart your lenses are, their distance from the phone, and so on.

Note: If the setup process displays the error message Problem in parsing the URL after you scanned the QR code of your Cardboard unit, you’ll have to modify theinfo.plist of your Xcode project as described here, in theApple developer forums.

Now go ahead and insert your phone into your Cardboard unit. Turn your head to adjust the camera’s view, and enjoy some fairly convincing 3D graphics.

Make it a Game Again!

Being able to view your game world is great and all, but you need to bring the gameplay back. Specifically, you want to be able to shoot ninja stars in the direction you’re facing. That’s the first piece of gameplay you’ll work with.

For UI, Cardboard supports a single button. It might seem limiting, but if you combine it with the motion tracking you get from moving your head, it allows for interactions that are more complex.

In Ninja Attack, you detect if your user is in VR mode with the GvrViewer.Instance.VRModeEnabled property. You’ll check if the button is pressed with theGvrViewer.Instance.Triggered property. If both of those come out to be true, you launch a ninja star in the direction the user is looking.

Open up your NinjaStarLauncher.cs script. You can find it attached to theGameLogic GameObject in the inspector.

Create a new private variable:

private Vector3 _vrShooterOffset;

Initialize it in your Start() method:

_vrShooterOffset = new Vector3(0.0f,-0.4f, 1.0f);

Replace Update() with the following:

void Update () {
  //1 
  if (GvrViewer.Instance.VRModeEnabled && GvrViewer.Instance.Triggered && !_gameController.isGameOver) {  
    GameObject vrLauncher = GvrViewer.Instance.GetComponentInChildren<GvrHead>().gameObject;
    // 2
    LaunchNinjaStarFrom(vrLauncher, _vrShooterOffset);
  } else if (!GvrViewer.Instance.VRModeEnabled && Input.GetButtonDown("Fire1") && 
    !_gameController.isGameOver) {
    // This is the same code as before
    Vector3 mouseLoc = Input.mousePosition;
    Vector3 worldMouseLoc = Camera.main.ScreenToWorldPoint(mouseLoc);
    worldMouseLoc.y = ninja.transform.position.y;
    ninja.transform.LookAt(worldMouseLoc);
    LaunchNinjaStarFrom(ninja, _shooterOffset);
  }
}

That will get things working. Here’s a look at what Update() does:

1. You first check if the game is in VR mode and if the user has pressed the button by examining the properties on theGvrViewer.Instance singleton object.
2. After that, you call LaunchNinjaStarFrom() to instantiate a ninja star. You pass in two parameters:
   1. The first is the head GameObject. The Google VR library moves it around for you, so it should already be pointed in the right direction.
   2. The second is a slight offset, so the method instantiates a ninja star slightly in front of and below the head GameObject, which looks a little more natural — otherwise it would look like you’re shooting ninja stars out of your eyes. Cool, but weird.

Since your Ninja Star GameObject is already designed to fly out from where it’s created, it will fire in the correct direction.

Give it another try! At this point, you can turn your head around and shoot bad guys. The win or lose logic still applies.

Take that, blobbies!

Fixing the Game Over Menu

As you might have observed, when the game is over you’re still left with the old Game Over menu. It not only shows up improperly in 3D, but you have no way of clicking on it.

Shockingly, this button renders poorly in 3D.

The game currently uses a Display Canvas — as seen in the Unity New Gui Tutorial — to display the Game Over screen, which always displays on top of the gamewindow.

This canvas is great for most game GUIs because it automatically scales itself to fit on top of your screen no matter what your game’s camera is doing, and it nicely handles different screen sizes.

But in this case, you need a GUI canvas that exists in the world itself, partly so it renders properly in 3D, but also because youdon’t want it to stay locked to the camera.

You want your users to be able to look up and down so they can look at different UI elements and trigger the active one by clicking the button.

Creating a New Canvas

Select the GameOverCanvas in the Hierarchy, right-click on it and selectDuplicate.

Rename the duplicated canvas VRGameOverCanvas, to distinguish it from the original one, and rename itsGameOverTxt child object toVRGameOverTxt.

In the Canvas component of VRGameOverCanvas, change the Render Mode to World Space

In the Rect Transform component, change the Position to (-2.24, 1.1, 0.07), and the Y Rotation to 90.

Finally, change the X and Y Scale to be 0.009. When it’s all done, the VRGameOverCanvas should look like this:

And you should see the two canvases roughly overlap each other in the Game View (when the game’snot running):

Where did these values come from? Truthfully, I just fiddled around with them until I had something that looked good when viewing them through the Cardboard camera.

Sometimes programming is more an art than a science. :]

Supporting Both Canvases

Next up, you’re going to alter GameController.cs so that it’s aware of both canvases. Open up theGameController script that’s also attached to theGameLogic GameObject.

Add the following two public variables to your class:

public Canvas VRGameOverCanvas;
public Text VRGameOverTxt;

Add the following line to the beginning of resetGame():

VRGameOverCanvas.enabled = false;

Replace GameOver() with:

public void GameOver(bool didIWin) {
    isGameOver = true;
    _didIWin = didIWin;
    string finalTxt = (_didIWin) ? "You won!" : "Too bad";
    if (GvrViewer.Instance.VRModeEnabled) {
        VRGameOverCanvas.enabled = true;
        VRGameOverTxt.text = finalTxt;
    } else {
        gameOverCanvas.enabled = true;
        gameOverTxt.text = finalTxt;
    }
}

This displays the proper canvas and text objects, depending on whether or not you’re in VR mode (GvrViewer.Instance.VRModeEnabled).

After you’ve saved your script, you’ll need to assign the correct objects to the new public variables.

Find the GameController script in the inspector. Click on the target next to each of the new variables, and selectVRGameOverCanvas object as yourVR Game Over Canvas variable and theVRGameOverTxt object as yourVR Game Over Txt variable.

Note: Are you wondering why you’re going through the trouble of supporting two canvases instead of just changing the existing one? Because eventually, you’re going to support both top-downand VR mode. Stay tuned!

If you were to run your game now, you would see that it properly shows the end-game screen in VR mode. You can look up and down and see the different parts of your interface; all that’s missing is a way for you to click thatPlay Again button.

Note: At least, this is how it should look in theory.

In practice, there is a bug in Unity (starting with 5.3.4p2) where World Space GUI canvases don’t render onto a RenderTexture, which is what the GoogleVR SDK uses for distortion correction — the fisheye look you see in the two cameras.

As a temporary workaround, you can go to your GvrMain object and in theGvr Viewer component, changeDistortion Correction toNone.

Turning off distortion correction in Google VR

This means that you may see some slight distortion when you try your app in a Cardboard viewer, but at least you’ll be able to see your GUI. :]

Adding Gaze Input

Luckily, Unity has built-in support for “Treating the center point of the camera as a mouse cursor” when using a world-space GUI canvas, but you need to provide an extra script to make it work in VR space.

First, expand GvrMain\Head in the Hierarchy. Find the Main Camera there and rename it toVR Main Camera

Select your VRGameOverCanvas object. You should see an Event Camera entry in theCanvas Component. Click on the target and select theVR Main Camera you just renamed.

Click on the EventSystem object in the Hierarchy. Click Add Component in the inspector and add theGazeInputModule script. This is a script that makes sure Unity’s GUI system understands how Google VR’s camera system works.

Check the VR Mode Only checkbox, because you only want things to work this way when you’re in VR mode — not when you’re in top-down version of your game.

Finally, click the gear of the Gaze Input Module Component you just added and selectMove Up. Do this one more time, so that it appears above theTouch Input and Standalone Input modules. This is required to make sure the Gaze Input Module takes priority over the other types of input that your game can process.

When it’s all done, it should look like this:

Give it a try now! This time, when you center your view on the Play Again button, it should turn green, and pressing down on the Cardboard button will let you “click” the it to start a new game!

Minor Gameplay Tweaking

So perhaps you found this version a bit harder to play in VR mode. This is partly because you have a reduced field of vision and it’s easier for enemies to slip by when you’re looking in the wrong direction. Also, you can’t change the direction you’re aiming nearly as quickly as you could before. You’re physically constrained by the speed at which you can turn your neck.

You don’t want to penalize your players for choosing to play in VR mode! So how can you correct this?

Oh, well, I was going to suggest slowing down the enemies. But… uh… your call, I guess.

Select your EvilSlimeEnemy Prefab in the Prefabs folder and open up the attachedEnemyMover.cs. Add this code toStart(), right after you setthisSpeed:

if (GvrViewer.Instance.VRModeEnabled) { 
  thisSpeed *= 0.85f; 
}

This leads to your enemies bunching up a bit. If you want to keep them spaced out, go toEnemySpawner.cs — it’s attached to yourGameLogic object — and add these lines inSetNextLaunch() right after you setlaunchInterval:

if (GvrViewer.Instance.VRModeEnabled) { 
  launchInterval *= 1.1f;
}

This will make your game a tad easier in VR mode — just enough that your player shouldn’t feel penalized for choosing to play this way.

Fixing the On-Screen Score

The other UI element you need to address is the on-screen score object, and you’re going to try a slightly different approach for this one. While it still needs to appear in VR mode properly, you want it to stay fixed to your camera no matter where you look.

Imagine projecting your score onto a piece of glass that keeps moving around so that it’s always about 2 meters away from where the player is looking. That’s the effect you’re going to create.

You’ll accomplish this by using another world canvas so it renders properly in 3D, but you’ll make it achild of yourGoogleVR Head object.

Select GvrMain\Head in the Hierarchy. Right-click on it and select UI\Canvas. Rename the new canvas to VRScoreCanvas. Change its Render Mode toWorld Space.

Give it the following values:

• Position: (0, 1, 2.5)
• Width: 400, Height: 100
• Rotation: (0,0,0)
• Scale: (0.0115, 0.0115, 1)

When you’re all done, it should look like this:

Right-click on your VRScoreCanvas and select UI\Text to add a text child-object. Rename it toVRScoreTxt. Change itsanchor point to betop-left. Give it aPosition of(150, -65, 0) and aWidth of 60.

In the text component, set Font Size to 18 and change the Alignment to right-aligned. Finally, change the Text to 999.

When it’s all finished, it should look like this:

It might seem like your text is strangely aligned in the center of the screen, but in VR mode, you see much less of your world than you normally would, so this ends up being pretty close to the edge of your vision. Feel free to adjust it so it looks right to you on your phone.

Next up, add the plumbing to display your score using this text object. The process is similar to how you displayed the Game Over canvas.

Open up GameController.cs and add a new public variable:

public Text VRScoreTxt;

Next, you’ll update VRScoreTxt every time you update scoreTxt. In theResetGame() method, add this line right after you updatescoreTxt:

VRScoreTxt.text = "--";

Then add this line to GotOne(), right after you update scoreTxt:

VRScoreTxt.text = "" + _currScore;

Save your script, go back into Unity and you’ll notice the GameController Component ofGameLogic now has an entry for yourVR Score Txt variable. Click the target next to it and select yourVRScoreTxt text object.

Play your game again, and now you’ll see that your score appears up in the upper-left corner of your vision, and that it follows your head movement.

Swapping In and Out of VR Mode

Since your game works in both top-down and VR mode, you should give the user the ability to switch between them.

The UI for this is pretty straightforward. You’ll add a simple button to your top-down game that users can press to switch into VR mode. As a bonus, the Google VR SDK automatically displays a back button you can use to go back into top-down mode.

Give it a try!

First, you’re going to add the code to handle swapping in and out of VR mode. SelectGameLogic in the Hierarchy. ClickAdd Component in the inspector, selectNew Script and name the scriptCardboardSwapper.

Open it, and replace the class code with this:

public class CardboardSwapper : MonoBehaviour {
 
  public GameObject[] cardboardObjects;
  public GameObject[] monoObjects;
 
  // Turn on or off VR mode
  void ActivateVRMode(bool goToVR) {
    foreach (GameObject cardboardThing in cardboardObjects) {
        cardboardThing.SetActive(goToVR);
    }
    foreach (GameObject monoThing in monoObjects) {
        monoThing.SetActive(!goToVR);
    }
    GvrViewer.Instance.VRModeEnabled = goToVR;
 
    // Tell the game over screen to redisplay itself if necessary
    gameObject.GetComponent<GameController>().RefreshGameOver();
  }
 
  public void Switch() {
    ActivateVRMode(!GvrViewer.Instance.VRModeEnabled);
  }
 
  void Update () {
    if (GvrViewer.Instance.BackButtonPressed) {
      Switch();
    }
  }
 
  void Start() {
    ActivateVRMode(false);
  }
}

The most important method of this class is ActivateVRMode where you toggle the value ofGvrViewer.Instance.VRModeEnabled. It’s what activates VR mode.

The rest of the logic disables or enables various GameObjects in your scene, depending on whether or not you’re in VR mode. CallingActivateVRMode(false) inStart() starts your game off in top-down mode.

You’ll also notice that you call Switch() when you detect the back button has been pressed on the Cardboard display. You can simulate thisGvrViewer.Instance.BackButtonPressed in the Unity editor by pressing theesc key, a feature you’ll no doubt find awfully handy for testing.

You do need to add just a bit more logic to your GameController script so that it properly displays or hides the correct canvas if you’re switching modes at the end of the game.

Open up GameController.cs, and add this method:

public void RefreshGameOver() {
    gameOverCanvas.enabled = false;
    VRGameOverCanvas.enabled = false;
    if (isGameOver) {
        GameOver(_didIWin);
    }
}

Save everything and go back to Unity to populate the values of the two GameObject arrays.

Select GameLogic in the Hierarchy and scroll down to the Cardboard Swapper component in the inspector.

For the Cardboard Objects array, give it a size of 1, and then fill in theHead child of theGvrMain GameObject in your scene. Not only does this disable your Google VR Head so you can go back to the top-down camera, but it disablesVRScoreCanvas as well.

For the Mono Objects array, give it a size of 3, and then selectCanvas,Main Camera, andraccoon-ninja from your scene (not from theAssets tab, which Unity seems to like defaulting to).

Finally, you need to add a button on the top-down canvas for the user. To save time, I’ve already made one for you — it’s in the prefabs folder.

Drag CardboardButton from Assets\Prefabs into the Hierarchy so that it’s a child of yourCanvas object. Make sure that itsPosition is set to(-50, 50, 0):

At the bottom of your button object, hook it up such that clicking the button will call theCardboardSwapper.Switch() method attached toGameLogic. You can follow this animation to see how it’s done:

Give your game another try. Click on the button at the bottom-right of the screen to switch to VR mode, then click the back button in the Cardboard display (or press Escape in the Unity editor) to go back to top-down mode.

Congratulations! You’re swapping VR Modes like a boss!

Where to Go From Here?

Download the final Unity project here.

You now have the power to take any 3D game in Unity and make it a VR experience with just a little cardboard and some plastic lenses. It’s VR for everybody!

The build process for Android is almost the same as for iOS.

Google’s Unity Developer Guide provides some additional technical info.

And finally, you could even add Augmented Reality (AR) to your VR projects!

Lastly, take a look at any 3D game you might make in Unity and see if there’s anything that would translate well to a VR experience. Or, maybe this tutorial will inspire you to create something entirely new.

As always, please feel free to leave questions and comments below or in the forums!

Oh, and if you figure out how to control lawn mowers with your mind, let me know — I’m still working on that.