Authors: Dávid Lakatos, Matthew Blackshaw, Anthony DeVincenzi, Daniel Leithinger, Hiroshi Ishii
Recompose is a new system for manipulation of an actuated surface. By collectively utilizing the body as a tool for direct manipulation alongside gestural input for functional manipulation, we show how a user is afforded unprecedented control over an actuated surface. We describe a number of interaction techniques exploring the shared space of direct and gestural input, demonstrating how their combined use can greatly enhance creation and manipulation beyond unaided human capability.
Below: Gestural detection and depth recognition as seen by computer vision
Below: A look at implemented gestural interactions. From top left: Selection, Actuation and translation, rotation, scale
Our system builds upon the Relief table, developed by Leithinger. The table consists of an array of 120 individually addressable pins, whose height can be actuated and read back simultaneously, thus allowing the user to utilize them as both input and output. Building upon this system, we have furthered the design by placing a depth camera above the tabletop surface. By gaining access to the depth information we are able to detect basic gestures from the user. In order to provide visual feedback related to user interaction, a projector is mounted above the table and calibrated to be coincident with the depth camera. Computer vision is utilized to determine and recognize the position, orientation, and height of hands and fingers, in order to detect gestural input.
Recompose: Direct and Gestural Interaction with an Actuated Surface will be published as a work in progress at CHI 2011