Inside Microsoft’s PixelSense

Last week’s launch of Microsoft’s Surface tablet means the previous product that bore the name has been re-branded as the Samsung SUR40 with Microsoft PixelSense. But the larger, 40-inch touch table display that’s so popular in event and exhibit spaces was also recently upgraded. We sat down with a few of Microsoft’s premiere content development partners for Surface/PixelSense to talk about how the new system differs from the original, and what designers should keep in mind when leveraging it in event and exhibit footprints.

Size & sensing

The unit itself is noticeably thinner than the previous version, which was about a foot-and-a-half deep and stood on its own like a piece of furniture. Now the 40-inch display has a four-inch bevel around the outside of the screen, with a depth of about four inches. The all-in-one form factor features a computer built it the unit itself.

The depth of the previous Surface unit was needed for the infrared cameras embedded within to sense where users were touching the screen. There are no cameras in the new version, it embeds the vision-sensing technology inside of a LCD panel.

“What continues to make it unique amongst almost all other touch-sensing, multi-user platforms out there is that it’s vision-based. It can actually see the shadow on your finger, so it can give you the orientation of your finger and a rotation amount. Which is great in a multi-user scenario, because if you can see which way a finger is coming from, you can spawn UI controls so that they automatically face the user who touched them,” says Stimulant ceo Darren David.

The PixelSense OS also got a nice boost, upgrading from Windows Vista to Windows 7. The first version was also not upgradable from an OS perspective, whereas 2.0 is upgradable.

Display

The new LCD screen produces a clearer display, as opposed to the old projection-based model.

“It’s full HD, 1920-by-1080 resolution, where it was 1024-by-768 in the previous version. It’s visually a much cleaner and brighter picture,” says Inhance Digital co-founder and president Maziar Farzam.

Interknowlogy ceo Emilie Hersh agrees that the HD caliber of the screen is definitely an improvement.

“With a lot of our healthcare-based customers it was especially of interest, because the clarity on the screen in V1 was not necessarily up to par with expectations for looking at things like radiology images or anything where they needed clear detail. Our healthcare clients are much more impressed with the clarity and the resolution now that it’s HD,” Hersh says.

Recognition

Both versions of Surface/PixelSense leverage fiducial or object recognition tags, an eight bit tag system with 256 different values that allow objects marked with them to interact directly with the display. In version 2.0 the PixelSense technology actually detects the object at the LCD pixel level, so it’s a little bit more accurate than before.

“The key advantage [with PixelSense] is object recognition, because it is built-in. If you can think about how to engage and tie it into a product and have the physical object interact with the table, it really brings the wow factor out,” Farzam says.

Interknowlogy did just that as part of a recent project for Coca-Cola at the Atlanta Food & Wine Festival. Designers tagged three different smartwater bottles and coded them into the app. When attendees put one of three different bottles down on the table it would spin out information on the event, the product line or Interknowlogy.

“There are all sorts of interactive elements you can use. QR codes are great, because users can snap it with their smartphone and they’re taking the experience with them,” Hersh says.

Mounting

The first version of Surface was a horizontal-only display, but the new thinner unit comes with a VESA mount allowing it to be hung vertically from a wall or mounted on an angle, which gives designers a bit more flexibility in how it can be integrated into an environment.

“When you have it in table format, collaboration is much easier; that’s where that true multi-touch environment comes into play. When you put it on the wall, clearly you’re not going to have too many people touching it at once, so there are some design considerations that need to come into play,” Hersh says.

The unit itself can also be programmed to respond to interactions differently depending on how it is mounted, resulting in richer experiences for the user.

“It has an accelerometer built in, which the software can respond to, so you can build an application that behaves differently automatically. It knows if the table is horizontal or vertical,” David says.

Lighting

Unfortunately, the latest upgrade didn’t tackle the problem of the unit’s sensitivity to environmental lighting.

“We found Surface 1.0 was very sensitive to environmental lighting and we thought the new version would be a major improvement, but unfortunately it’s still about the same. When we deploy it at trade shows, I caution everyone who uses it to be very cognizant of environmental and overhead lighting. It definitely causes interference with the infrared touch technology,” Farzam says.

David agrees, saying “the worst are direct halogens or incandescent lights—or anything that gives off a lot of IR light. It results in phantom touches and it can make a UI seem kind of sketchy.”

Leveraging PixelSense

The current price point for the unit is $8,600, which is way down from the V1 price tag of $15,000. Designers should estimate a total cost between $9,000 and $10,000 shipped, including tax and the mounting legs.

And while the technology is very cool in and of itself, the content is the real key in leveraging PixelSense successfully.

“I think the unit itself is not intimidating to the user; you want to go up and touch it and interact with it. A lot of the skill lies in the design of the application, so that you make it intuitive for the user. The hardware is great, but if the application isn’t designed correctly and with the user in mind, it’s not going to be successful,” Hersh says.