What is Gesture-Based Computing?

It is already common to interact with a new class of devices entirely by using natural gestures. The Microsoft Surface, the iPhone and iPod Touch, the Nintendo Wii, and other gesture-based systems accept input in the form of taps, swipes, and other ways of touching, hand and arm motions, or body movement. These are the first in a growing array of alternative input devices that allow computers to recognize and interpret natural physical gestures as a means of control. We are seeing a gradual shift towards interfaces that adapt to — or are built for — humans and human movements. Gesture-based computing allows users to engage in virtual activities with motion and movement similar to what they would use in the real world, manipulating content intuitively. The idea that natural, comfortable motions can be used to control computers is opening the way to a host of input devices that look and feel very different from the keyboard and mouse — and that enable our devices to infer meaning from the movements and gestures we make.

INSTRUCTIONS: Enter your responses to the questions below. This is most easily done by moving your cursor to the end of the last item and pressing RETURN to create a new bullet point. Please include URLs whenever you can (full URLs will automatically be turned into hyperlinks; please type them out rather than using the linking tools in the toolbar).

Please "sign" your contributions by marking with the code of 4 tildes (~) in a row so that we can follow up with you if we need additional information or leads to examples- this produces a signature when the page is updated, like this: - alan alan Jan 27, 2010

(1) How might this technology be relevant to the educational sector you know best?

  • These kinds of systems will only increase the amount of students that have access to educational resources. - JamieMadden JamieMadden Oct 5, 2010
  • As mentioned in relation to Augmented-Reality, SixthSense out of MIT Media Labs is doing incredible things in relation to gesture-based computing. The boon to education is phenomenal if one considers the ability for users to interface with the world around them, collect multiple forms of content and configure new creative elements, concept representations, and collaborative ideas all through natural interactions that bridge their personal digital space with the surrounding environment. This applies to sciences, art, literature, and more. - Dougdar Dougdar Oct 5, 2010

(2) What themes are missing from the above description that you think are important?

  • This could be a huge help to disabled students and, generally, provide a good balance to classical work at the computer (counteracting carpal tunnel syndrome and the like) - in that way changing the way we work and live - helga helga Sep 30, 2010
  • another response here

(3) What do you see as the potential impact of this technology on teaching, learning, or creative expression?

  • Depending on how it is implemented, it will allow students with disabilities to interact with their environments better - JamieMadden JamieMadden Oct 5, 2010
  • another response here

(4) Do you have or know of a project working in this area?

Please share information about related projects in our Horizon Project sharing form.