Advertisement

Making music in a Kinect-powered sandbox (video)

Tucked away under a tent at Expand 2014 was perhaps the coolest exhibition on the show floor: Sand Noise Device, a literal interactive sandbox experience. No, this isn't a new Grand Theft Auto; it's powered by hacked gaming tech, though, including an Xbox 360 Kinect sensor and a PlayStation Eye camera. Watching it in action immediately brought memories of Xbox 360 classic Geometry Wars to mind, actually. A ring radiates out from a center origin point, and when it hits the glowing, multicolored pucks (that are tracked for position by the PS Eye), a series of particles start shooting outward.

From there, they bounce around a 4 x 3-foot sand table with physics-based reactions to changes in the substrate's elevation and topography. Translation? The particles can get stuck in a depression you create -- gaining speed as they reach the bottom -- or follow a path left by raking a set of shallow grooves into the sand. You can even "catch" one in your palm and hold it there. Oh, and every movement produces sound too.

"It's a generative music system, so you program a bunch of rules, feed it data and it creates music based on that data," Jay Van Dyke, one of the project's creators from The Green Cat Collective, told us. When the center origin wave hits the pucks, and the quasi-spaceships make their way outward (depending on how high or low the sand physically is), that's how loud or quiet the sound will be (respectively). "The Kinect is basically the way we read the topography of the sand," Van Dyke said. "It returns every frame of how people have manipulated the sand to us, and that becomes our environment."

Engadget Expand: The Sand Noise Music Device

That environment, according to Van Dyke, could be pretty big and humans could even be a physical part in it, standing in a pile of sand on the floor. That'd have made a mess at the Javits Center, though, so the team opted instead for a wobbly table that was rated to hold some 300 pounds of the pre-glass grains. And even with that precaution, by late Saturday afternoon, the cement floor in and around the tent was already fairly gritty.

The idea stems from the frustrations of wanting to play an instrument, but not exactly knowing how, Van Dyke said. He mentioned that someone might come along a piano and tap a few keys here and there, but the immediate sense of gratification (one of the reasons games like Rock Band and Guitar Hero were so popular) just isn't there. Many want a quick fix in a fleeting moment, not the years of training it takes to traditionally make music.

"You can play around with it and learn how it works in a couple of minutes and have a really good time," Van Dyke said.

Judging by the amount of Expand attendees that crowded around the sand table at any moment throughout the weekend, it's safe to say Van Dyke and his team were pretty successful.