Tutorial: How to play PC games with your Kinect
How to play PC games with your Kinect
Microsoft likes to think it’s changing the world every time Steve Ballmer sneezes. Windows Phone! AMAZING! It’s single-handedly changing the way we look at phones forever. Except it isn’t. It’s actually a phone mixed with a Zune, and that’s not necessarily a great thing.
Never mind: Windows 7! It’s absolutely incredible, a true milestone in operating system technology! Except it isn’t. It’s Windows Vista with the terrible bits sliced off it, and that’s all. Bah.
But wait! Kinect! It’s an astonishing new way to interact with your PC. And in this case, Microsoft is at least half right. It’s a magical black rectangle. It’s about £100. It’s Microsoft’s strange answer to the Wii’s motion-sensitive Wiimotes, or the Playstation Move, which tracks brightly coloured balls on the end of motion-sensitive controllers.
Basically, it’s a bit of a copycat – apparently the button-based controller isn’t good enough for console gaming any more. No, the new controller is meat-based: it’s you.
Connect to PC
MS wasn’t initially keen on Kinect being used with the PC, actually. It was meant to be Xbox-only – some comments were bandied about stating that Microsoft would resolutely not be supporting its use anywhere other than that which was originally intended. But the boys at Redmond have capitulated.
Kinect and the PC are now allowed to be friends, and that means good things for you. It means you can use its magical depth-sensing and skeleton-tracking to interact with your favourite games. Nice!
Please, though, bear this in mind: Microsoft has only just, at the time of writing, released the official SDK. No applications have yet been written that use it, so we’re using the unofficial drivers released by enthusiasts wishing to crowbar the device into working. These are decent, but they miss out a few important bits, and they’re far from refined.
We’re looking at very early days for this device, and results at this stage can be somewhat disappointing. If you already have a Kinect, by all means have a go. But don’t rush out and buy one just to try to use it to control your favourite games, unless you really feel like swaying about like a spanner and replacing pinpoint keyboard/ mouse control for frustrating, almost incoherent fist-waving.
Because that’s what’s going to happen. FAAST, which we look at shortly, can be pretty accurate, but only if you spend a lot of time working out the ideal settings for you.
Full functionality
With that warning out of the way, it’s worth considering the future. Kinect will get better and better. As developers embrace the official tools, Kinect’s full functionality – which includes a noise-cancelling mic array, a person-tracking motorised webcam, an infra-red camera, and a rather natty depth sensor – will become available.
Sensitivity will rise. Games will start officially supporting it. Us PC users already have a jump over the Xbox crowd, in that we can use the camera’s full 640 x 480 resolution whereas they’re stuck at 320 x 240; there’s no telling what bonuses braniac devs will be able to pull out given a bit of time.
Cleverly, most of the Kinect calculations are done by drivers and middleware – not much on the device itself. This means developers only need to employ simple coding techniques to track skeletons, for example, or recognise basic gestures like a waving hand, rather than directly dealing with the raw data the Kinect is putting out.
It should also mean that Kinect-supporting games reach us quite quickly. There’ll almost certainly be dance games, punch-em-ups, and other things which suit such physical monkeyshines.
But we wouldn’t be surprised to see the tech used elsewhere too: Windows 8, for example, will almost certainly feature some kind of direct Kinect support, enabling you to use Minority Report-style hand gestures to flit through the interface with aplomb.
No Kinect? No problem. There are other ways to get yourself noticed. We took a look at Camspace, for example, which does very similar things to Kinect but uses a webcam in place of its fancy array of sensors. Admittedly Camspace is confusing – if you don’t know where to look, you might end up paying money to play its rather rubbish range of webcam games.
But if you do know where to go, and you’re prepared to put in a bit of effort to line up its tech with your favourite games, it’s pretty cool. Check out this video of someone playing with Garry’s Mod, for example, using nothing but a couple of bits of coloured tape on the end of his fingers. Ingenious!
Or how about this fellow who’s using a natty sideways baseball cap to look around in Minecraft? Nifty.
Part 1: Getting it running
In which we awkwardly install a series of necessary drivers
01. The OpenNI framework
Kinect and your PC don’t get along by default. Even after the release of the SDK, Microsoft doesn’t offer proper drivers. And third-party ones need to be installed in a particular order.
Start by downloading the OpenNI framework, developed by the volunteer team dedicated to getting Kinect and other interaction devices running on the PC without Microsoft’s help. It’s powerful middleware that does lots of heavy lifting.
02. Avin2′s driver
During your install of OpenNI you’ll be asked if you want to install a driver from PrimeSense (the third party that actually made the Kinect). But PrimeSense’s driver isn’t Kinect-specific. You’ll need a hacked version, known as ‘sensorkinect’, developed by a chap named Avin2.
Grab it from his github page at github.com/avin2/SensorKinect and install it. But don’t plug your Kinect in to your PC yet.
03. Into the Nite
The final part to install is Nite, a gesture recognition toolkit from the OpenNI group that’s used by a number of apps. Normally this is commercial software, but Kinect tinkerers can install it for free.
Grab the latest stable build from www.primesense.com/?p=515 and install it, using the code 0KOIk2JeIBYClPWVnMoRKn5cdY4= to unlock it. Now just restart your machine and plug in the Kinect.
Playing games with your Kinect
Part 2: Playing games
In which we lurch around like idiots even more than usual
01. FAAST fun
The Flexible Action and Articulated Skeleton Toolkit is the key tool used to translate your movements into PC inputs. It latches on to the drivers we installed in Part 1 to follow your skeleton and your position in the room and turn this into keyboard inputs.
First things first: grab the latest version of the software from projects.ict.usc.edu/mxr/faast/, extract it, and run the excecutable inside.
02. Hook it up
We’ll presume you’re sitting down. Under ‘Skeleton Mode’, select upper body to tell FAAST it only needs to track the highest part of you, and adjust the smoothing value to somewhere around 0.4.
When you click ‘Connect’, you’ll join FAAST to the OpenNI drivers and, hopefully, see an outline of yourself in its main window. Keep this on screen if you can; it’s useful to see what translates into what later on.
03. Do the pose
Put yourself in front of the Kinect and get your silhouette in frame. Stick your elbows out to the side, your fists up, and pose like you’re Hulk Hogan.
Hold the pose for a couple of seconds and you should pass the Kinect calibration procedure, with the unit locking on to the length of your limbs and the quality of your posture. As Microsoft would say, you are now the controller.
04. Map your controls
Click over to the Actions page, and you’ll see the built-in mapping, which relates your leaning from a central position to in-game movements suitable for walking around.
Dive into an appropriate game now – you’ll just need to hit Start Emulator, fire up the game, and waggle yourself around to see how it works. Not perfect, we’ll admit, but it’s a start.
05. Point and shoot
Stop the emulator, go to the mouse tab, and switch mouse control to active. We went for absolute control, the shoulder joint as the centre, 8-inch bounds for each axis, and a movement threshold of 5.
Start the emulator and your flailing arm should translate into mouse movements, meaning you’ll be able to make a gun shape with your fingers and relive playground war games to your heart’s content.
06. Trigger happy
You can’t shoot yet. You’ll need to add a custom command to the Actions tab to turn a specific body movement into a trigger action. Stop FAAST’s emulation (you might need to force-quit it using the Task Manager), go to the Actions tab, and insert something along the lines of right_arm_up 5 mouse_click left_button (tweak as you see fit) to map the movement of your other hand to a firing action.
Part 3: No Kinect needed
In which we realise that we already had a webcam and thus wasted our money
01. Grab the software
You’ll find CamSpace at camspace.com. The idea is that it locks on to objects of a certain colour or size, then follows them around on your webcam, translating their movements into actions in game.
There are a bunch of pre-cooked profiles available for a load of games, but – most irritatingly – you have to save up ‘Campoints’ to get them. You’re given one per day. Er, sorry.
02. Set it up
But you could always just roll one from scratch. Grab a bunch of brightly coloured objects; Camspace needs something that contrasts with the rest of the environment in order to work well.
A Coke can, for example, or a lurid marker pen. Run the software, then suffer through its horrible calibration game. Now right-click the webcam window, choose Developer Tools, and make your own.
03. The easy bit
Alright. We lied. This isn’t the easy bit. But let’s say it’s relatively straightforward. You’ll want to set up the number of objects you want Camspace to track, say what each of them needs to do, and perhaps tweak a few sensitivities to match the specifics of your movement. You should be able to get it working with just about any game – check developers.camspace.com/tutorials/gamecontroller for more.
Conclusion
If you’ve followed the steps in the tutorial, you’ll probably agree with our assertion earlier on that Kinect – and its webcam compatriots – just aren’t very good at controlling games. Sorry about that. We did our best.
But they are pretty good at some other things, which you should definitely check out. The Nocolas Burrus’ RGBdemo, for instance, is a great example of the technology. Get it running by starting rgbd-viewer.exe (you’ll need the drivers we described in Part 1 of the walkthrough on page 83) and check out the 3D view mode. Click and drag the mouse, and you’ll see something very nifty – it’s you (or whatever you’ve pointed your Kinect at) mapped in an actual 3D environment. Cool!
You can even export this mapping into a program, such as Meshlab or Blender for further tweaking.
There are plenty of other cool toys available, too. You can also try your hand at charging up and shooting a Dragon Ball Z-style fireball with Kamehameha. Alternatively, you can attempt to ‘juggle’ virtual balls made out of light with, er, Kinect Juggle.
Heck, you can play classic puzzle games, such as Tetris using utterly impractical kung fu gestures. Lanny Lin has created a configuration file for FAAST that will allow you to do just that. Check out the video of him doing just that at his Lannyland blog.
All of these toys are only the tip of the iceberg of what could potentially emerge from under the many, brilliant minds that make up the homebrew developer community, and it’s definitely the community doing the majority of the development. So now it’s up to you – will you be the one to make Kinect an essential PC device?