AR SANDBOX

Instructions

This page will hold purchase, construction, setup, and calibration instructions for an Augmented Reality (AR) Sandbox.

Hardware Requirements

An AR Sandbox requires the following hardware components:

• A computer with a good graphics card, running any version of Linux. The AR Sandbox software, in principle, also runs on Mac OS X, but we advise against it.
• A Microsoft Kinect 3D camera. The AR Sandbox software,  supports all three models of the first-generation Kinect (Kinect-for-Xbox 1414 and 1473 and Kinect for Windows). All three are functionally identical, so get the cheapest model you can find. Note: The second-generation Kinect (Kinect for Xbox One or Kinect for Windows v2) is not yet supported by the AR Sandbox software.
• A digital data projector with a digital video interface, such as HDMI, DVI, or DisplayPort.
• A sandbox with a way to mount the Kinect camera and the projector above the sandbox.
• Sand.

Computer

The ideal computer for an AR Sandbox is a dedicated PC running a version of Linux, with a consumer-level, e.g., AMD/ATI Radeon or Nvidia GeForce, graphics card. The PC should have a good CPU, but does not need large RAM (2GB is sufficient to run the AR Sandbox software) or a large hard drive (20GB is sufficient to install Linux and the AR Sandbox software). While the AR Sandbox software runs under Mac OS X, we strongly recommend Linux because Linux-based installations are more stable.

We recommend an Intel Core i5 or Core i7 CPU running at at least 3GHz (as of 11/2016), an Nvidia GeForce GTX 1060 graphics card (as of 11/2016), and the current release of the 64-bit version of Linux Mint with Mate desktop (18 "Sarah" as of 11/2016). The AR Sandbox requires that the vendor-supplied binary drivers for the graphics card be installed.

The AR Sandbox has two main components: the topographic map renderer, and the water flow simulation. The former is comparatively easy on CPU and graphics card, and works on most current laptops or mid-range PCs. The water simulation, on the other hand, requires a high-end graphics card like the recommended GeForce GTX 1060. While the water simulation can be disabled to allow running the AR Sandbox from a lower-end computer, we do not recommend doing so.

The benefit of a dedicated PC is that the AR Sandbox becomes a computational appliance. Since the AR Sandbox software does not require a live Internet connection, it is possible to install the OS and AR Sandbox software, and then disconnect the computer from the Internet and never update the operating system or AR Sandbox sofware. The AR Sandbox software can be set up to start automatically when the computer boots, in which case it does not even require a monitor, mouse, or keyboard. Should newer versions of the AR Sandbox software become available, they can be uploaded via removable media.

Projector

Ideally, the projector should have a short throw length and a native 4:3 aspect ratio to match the field-of-view of the Kinect camera. The projector's native resolution is secondary; XGA (1024x768 pixels) is sufficient, as the sandbox overall's resolution is limited by the Kinect camera's 640x480 pixels. For practical reasons, short-throw projectors generally project above centerline, i.e., the bottom edge of the projected image appears above an imaginary horizontal plane through the projector's lens. The ideal projector for an AR Sandbox would be a centerline projector, so it could be mounted directly next to the Kinect camera. Since centerline short-throw projectors are rare and typically very expensive, the compromise is to mount an above-centerline projector above the sandbox's rear long edge, while the Kinect is mounted above the sandbox's center (see Figure 1).

We recommend the BenQ MX631ST above-centerline short-throw XGA DLP projector with 13000:1 contrast ratio and 3200 ANSI lumens, which generally sells for around $550. At maximum zoom, it matches the Kinect camera's field-of-view and can thus be mounted at the same height as the Kinect camera, simplifying setup.

We strongly recommend that the projector be connected to the PC's graphics card via a digital video connection, i.e., using an HDMI port on the projector, and an HDMI, DVI, or DisplayPort port on the graphics card. An analog connection, such as using a 15-pin VGA port on the projector, leads to degraded image quality and can cause misalignment between the projected image and the sand surface.

Sandbox

The sandbox itself should have a 4:3 aspect ratio, to match the fields-of-view of the Kinect camera and the projector. The size of the sandbox is limited by the Kinect camera's minimum and maximum sensing distances, and the desired sandbox resolution. Due to the Kinect camera's approximately 90° field-of-view, the Kinect camera has to be mounted about as high above the sand surface as the sandbox is wide. The Kinect camera should be mounted directly above the sandbox's center point, looking straight down (see Figure 1).

Figure 1: Typical arrangement of projector and Kinect camera above a 40"x30" (1mx0.75m) AR Sandbox. The short-throw projector is mounted at the same height as the Kinect camera, but above to the rear long edge of the sandbox to account for its above-axis projection.

We recommend a sandbox size of 40"x30" or 1mx0.75m, leading to a Kinect camera mounting height of approximately 40" or 1m. At this height, the camera's nominal horizontal resolution is 1.56mm, and its effective horizontal resolution is high enough to resolve features on the order of 5mm. Vertical resolution at the same height is 2.79mm. Increasing the size of the sandbox increases the required height of the camera/projector mount by the same factor, and not only reduces horizontal resolution, but also vertical resolution. In a 2mx1.5m sandbox, for example, nominal horizontal resolution is 3.12mm, and vertical resolution drops to 11.16mm (vertical resolution is roughly proportional to height squared).

Sand

The sandbox should be filled with sand to a depth of around 4" or 10cm. At 40"x30", this totals 2.77 cubic feet or 75dm³ or 75l of sand, weighing approximately 198lb or 98kg. We recommend Sandtastik White , for between $15 and $25 per 25lb, or $120-$200 total. Sandtastik sand has excellent projection properties, but a shallow angle of repose when dry. We recommend keeping the sand slightly moist to make it moldable. Adding 1 cup or 0.25l of water to 198lb or 98kg of sand and mixing thoroughly is sufficient.

Hardware Construction

Building the sandbox itself, and mounting the Kinect camera and projector above it are left as exercises to the reader (see Figure 1 for a rough layout sketch). Our own prototype AR Sandbox is built from wood and some metal. The sandbox proper is plywood on top of a sturdy wheelbase (for mobility), slathered generously with polyurethane to make the sandbox waterproof and rot-resistant (We are using small amounts of water to make the sand moldable). The projector and camera head assembly is made from aluminum slats, and the entire assembly is suspended above the sandbox from a vertical steel pipe. The head assembly offers limited adjustment for camera and projector position and orientation to allow physical alignment between sand surface, camera field-of-view, and projected image. Most importantly, the projector should be oriented such that it projects on-axis onto the "ideal" level sand surface. This minimizes distortion effects and focus problems.