Evaluation of Modular Actuated Tangible User Interfaces

Research project in the area of Virtual and Augmented Reality.

Keywords: tabletop interaction, widgets, haptics, actuation, prototyping platform, Tangible User Interfaces.

About this Project

"Tangible Computing" is an active area of research in the field of human-computer-interaction. Its intention is the design of intuitive and meaningful user interfaces through a tight coupling of physical and virtual objects. Virtual objects are represented by a physical representation, which enhances interaction by feeling and direct physical manipulation, making the interaction "tangible". A key aspect are so-called "Tangible User Interface Objects" (short "tangibles"). These physical object are typically used on an interactive work surface to allow the manipulation of virtual objects and models. Tangible user interfaces provide unique properties like intuitive use, immediate feedback and minimal necessary visual attention.

Existing systems utilize physical objects in interaction tasks for input only, enabling the user to communicate changes to the system. Changes of the virtual objects on the other hand are not reported to the user in a similar way, since the employed tangibles do not have any output channels. Self-positioning tangibles, so-called "actuated" tangibles, have been researched only sparsely so far; all existing solutions have significant technical limitations and require considerable construction effort.

In the course of this project we developed new "Actuated Tangible User Interface Objects" (short ACTOs) in order to address this situation. A flexible and modular design allows the integration of different input and output devices, ranging from simple buttons over complex sensors to graphical displays or active mechanical components. Typically integral parts like actuation mechanism or tracking technology can be individually customized and interchanged. This makes the ACTO system ideal for research and education, e.g. it provided the basis for our MediCubes projekt: https://www.ims.tuwien.ac.at/projects/medicubes

 ---

„Tangible Computing“ ist ein aktives Forschungsfeld im Bereich Mensch-Computer-Interaktion. Das Ziel ist die Entwicklung intuitiver und ausdrucksstarker Benutzerschnittstellen durch stärkere Kopplung von physischen und virtuellen Objekten. Dabei wird virtuellen Gegenständen eine physische Repräsentation verliehen, wodurch sie leichter erfahrbar und manipulierbar, also „Tangible“ (engl. greifbar, fühlbar, erlebbar), werden. Einen Schwerpunkt der Forschung stellen sogenannte „Tangible User Interface Objects“ (kurz Tangibles) dar. Dabei handelt es sich um physische Objekte welche auf einer interaktiven Arbeitsfläche benutzt werden, um virtuelle Objekte und Modelle zu manipulieren. Angreifbare Benutzerschnittstellen bieten einzigartige Eigenschaften, wie beispielsweise intuitive Verwendbarkeit, unmittelbares Feedback und minimale notwendige visuelle Aufmerksamkeit.

Existierende Systeme verwenden meist physische Interaktionsobjekte ausschließlich zur Eingabe, um Änderungen durch einen Benutzer an das System weiterzuleiten. Änderungen der virtuellen Objekte können aber nicht auf gleiche Weise an den Benutzer zurückgegeben werden, da die eingesetzten Tangibles über keinerlei Ausgabekanäle verfügen. Selbst positionierbare, sogenannte „Actuated“ (engl. in Gang gebracht, ausgelöst, betätigt) Tangibles, wurden bisher nur wenig erforscht. Alle existierenden Lösungen haben große technische Einschränkungen und stellen außerdem einen großen Konstruktionsaufwand dar.

Im Rahmen dieses Projekts wurden daher neue „Actuated Tangible User Interface Objects“ (kurz ACTOs) entwickelt, welche diese Einschränkungen aufheben. Ein flexibles, modulares Design ermöglicht die Integration unterschiedlichster Ein- und Ausgabemodalitäten von einfachen Buttons über komplexe Sensoren bis zu Displays und aktiven mechanischen Komponenten. Üblicherweise unveränderbare Bestandteile wie der Fortbewegungsmechanismus oder die Technik zur Positionsbestimmung können individuell angepasst und ausgetauscht werden. Damit bietet sich unsere ACTO-Plattform besonders für den Einsatz zu Forschungszwecken und in pädagogischem Kontext an, beispielsweise bildete es die Basis für unser MediCubes-Projekt: https://www.ims.tuwien.ac.at/projects/medicubes  

Project Partners

Höhere technische Bundes- Lehr und Versuchsanstalt (Spengergasse 20, A-1050), Österreichische Gesellschaft für innovative Computerwissenschaften (INNOC; Haussteinstraße 4/2, A-1020 Wien)

Funding provided by

Federal Ministry of Science, Research and Economy (bm:wfw)

Additional Information

Our design strategy for ACTOs as a prototyping platform for research and education offers flexibility in almost any aspect. The following image illustrates the technical implementation of the system including an optical tracking system, a server and multiple ACTOs:

The system is easily customizable in almost any way. An Arduino compatible “Base Module” with RF communication capabilities can be fitted with an optional “Motor Module” which incorporates the desired actuation mechanism for positioning and rotating the ACTOs. On top of the Base Module there is a socket for mounting a custom-made “Extension Module” which can be equipped with arbitrary I/O devices.

The ACTOs are reusable for a wide range of scenarios, minimizing possible redesign times. The actuation mechanism for movement is integrated in the ACTO, allowing the use on any flat surface. The integration of new hard- or software is alleviated through provided templates, changing the physical configuration of an ACTO is a matter of minutes. The system is self-sufficient and has no heavy or bulky components. It can be set up very quickly anywhere. Costs scale with the number of ACTOs, not the size of the working area.

A framework has been developed to support the easy integration of Extension Modules for ACTOs as well as the implementation of ACTO based applications. Its key components are the ACTO Server App on the smartphone with an Activity Template for comfortable application development, the ACTO OS on the tangibles which includes a template for easy integration of new Extension Modules, and a communication protocol to interface all components and a possible WLAN client.

In order to evaluate the suitability of the ACTO system as a prototyping platform for researchers and students, we tested how easy it is to learn to use and extend the system.

For that purpose we observed computer science students at a Federal Higher Technical Institute as well as computer science master students at our university during the process of applying our system for the first time. Their assignment was to sketch an application scenario for tabletop TUIs and to design and to construct the appropriate Extension Modules for our system. The collection of available Extension Boards and Motor Units is growing steadily:

A typical ACTO setup:

The young scientists of the HTBLVA Spengergasse together with their teacher Walter Hammerl:

Project Website

For more information, please visit the project website:

• www.sparklingscience.at/de/projects/show.html?--typo3_neos_nodetypes-page%5Bid%5D=526

Downloads

ACTO framework

ACTO framework, SW and templates

32.4 MB

Zip archive

Download

ACTO hardware

ACTO schematics, pcb layouts, BoM

1.7 MB

Zip archive

Download

ACTO OS

ACTO OS template, Arduino Server

15.1 KB

Zip archive

Download