Gesture-Based Computing

Time-to-Adoption: Four to Five Years
Gesture-based computing remains an interesting technology, and the number of gestural devices on the market has grown over the past year. As the underlying technologies evolve, a variety of approaches to gesture-based input are being explored. The screens of the iPhone, the iPad, and the Surface, for instance, react to pressure, motion, and the number of fingers touching the devices. The smaller devices additionally can react to manipulation — shaking, rotating, tilting, or moving the device in space. The Wii and similar gaming systems use a combination of a handheld, accelerometer-based controller and a stationary infrared sensor to determine position, acceleration, and direction. The technology to detect gestural movement and to display its results is improving very rapidly, increasing the opportunities for this kind of interaction. Two new gaming systems — the Sony PlayStation 3 Motion Controller and the Microsoft Kinect system — take a step closer to stripping the gesture-based interface of anything beyond the gesture and the machine, at least in terms of how it is experienced by the user. An interesting area of development is the intersection of haptics — touch-sensitive interfaces that also give physical feedback — and gesture-based computing, where we are beginning to see new kinds of interactions that combine the affordances of both approaches.

Relevance for Teaching, Learning & Creative Inquiry

  • Gestural interfaces allow users to easily perform precise manipulations that can be difficult to manage with a mouse.
  • Gesture-based games like those developed by researchers at Georgia Tech University can help deaf children learn linguistics at a critical time of language development.
  • Large multi-touch displays support collaborative work, allowing multiple users to interact with content simultaneously.

Gesture-Based Computing in Practice

For Further Reading

IDENT Technology’s Near Field Electrical Sensing Interfaces
IDENT Technology uses near field electrical sensing to allow mobiles to respond to grip and proximity sensing. A ringing mobile will put the call through if it is picked up and held, but will send it to voice mail if it is picked up and quickly put down again.

Point, Click: A Review of Gesture Control Technologies
(Damian Rollison,, 9 February 2010.) This article discusses the key developers and platforms working with gesture-based technologies.

Developed at Disney Research, Pittsburgh, TeslaTouch provides physical feedback for touch interfaces. Different sensations convey information about the items being tapped or dragged.