Last update: February 18, 2014 [HOME]

Go Directly To [ADAG] [ALICE] [AMHA] [ENTERTAINMENT] [HX] [PARA] [ROBO] [SEAT] [finished]

Research Projects at Eindhoven University of Technology (2004--)

[alphabetically sorted by project acronym] 

ADAG: Adaptive Gaming


The main objective of this project is to find out the design of optimal challenges in computer games. Based on the theoretical concept of Rauterberg (1995) an adaptive game is developed based on the new adaptation concepts introduced by Spronck (2005) and Spronck et al. (2006). With experiments we will investigate and determine the optimal 'incongruity' level for playing. One of the expected outcomes is a technology going beyond the concept of 'levels' in game design. In a more general perspective, this project can lead to a fundamental new way of understanding human information processing and learning.



Department of Industrial Design, Eindhoven University of Technology, Netherlands.


Pieter Spronck. This project is carried out in cooperation with MICC-IKAT at Maastricht University (Netherlands).


Rauterberg, M. (1995). About a framework for information and information processing of learning systems. In: E. Falkenberg, W. Hesse,  A. Olive (eds.), Information System Concepts--Towards a consolidation of views (IFIP Working Group 8.1, pp. 54-69). London: Chapman&Hall.

Spronck P., Ponsen M., Sprinkhuizen-Kuyper I., Postma E. (2006). Adaptive Game AI with Dynamic Scripting. Machine Learning, Vol. 63, No. 3, pp. 217-248.

Spronck P. (2005). Adaptive Game AI. Ph.D. thesis, Maastricht University Press, Maastricht, The Netherlands.


ALICE:  Cultural Computing


The project ALICE is about cultural computing as an interactive, entertaining experience inspired by Alice in Wonderland.

The main objective of this project is to find a possible answer for the Western culture to the cultural computing project 'ZENetic Computer' from Tosa and Matsuoka designed for the Eastern culture. Cultural computing is a new research area for interactive systems that can address the cultural layer of the user. In the scope of the project ALICE interactive adventures are experiences provided by an Augmented Reality (AR) environment based on selected parts from Lewis Carroll's book "Alice's Adventures in Wonderland". The user assumes the role of Alice and explores an interactive narrative. ALICE is an exploration of interactive story-telling in AR. By exploiting the unique characteristics of AR compared to established media such as film and interactive media, the project uses AR as a new medium for edutainment and entertainment. Innovations include the refashioning of conventions used in film and interactive tools for the development of an AR narrative, and the use of simple artificial virtual and real characters (avatar and robot respectively) to create an immersive interactive experience.


In co-operation with:
Microsoft Research Laboratory, Cambridge, UK.


External relationships:
Marc Cavazza, Ryohei Nakatsu, Ben Salem, Naoko Tosa.


Project team:

Jun Hu (manager), Tijn Kooijmans (assistant), Dirk van de Mortel; Christoph Seyferth, Dima Aliakseyeu, Marco Combetto,  Ben Salem.

Rauterberg, M. (2006). Cultural Computing: the ALICE project / Usability in the Future / Emotional Challenges in Future Computing.



Rauterberg M. (2006). How to assess the user’s experience in cultural computing. In: T. Bosenick, M. Hassenzahl, M. Müller-Prove, M. Peissner (Eds.): Usability Professionals 2006 (pp. 12-17). Fraunhofer Informationszentrum Raum und Bau.

Rauterberg M. (2006). Usability in the future –explicit and implicit effects in cultural computing. In: AM. Heinecke, H. Paul (Eds.): Mensch & Computer 2006: Mensch und Computer im StrukturWandel (pp. 29-36). München, Oldenbourg Verlag.

Rauterberg M. (2006). From personal to cultural computing: how to assess a cultural experience. (pp. 13-21) Pabst Science Publisher.

Nakatsu R., Rauterberg M., Salem B. (2005). Forms and theories of communication: from multimedia to Kansei mediation. Multimedia Systems, Vol. 11(3), pp. 304-312.



AMHA:  Automatic Mental Health Assistant


The AMHA (Automatic Mental Health Assistant) project is part of the Mars-500 experiment to be carried out at the Institute for Biomedical Problems (IBMP) in Moscow. The European Space Agency (ESA) and the Russian Academy of Sciences jointly plan and conduct this experiment in order to simulate a manned mission to Mars. It will provide the unique opportunity to study interactions between crew members while collecting data about their health and performance during experimental isolation. The confinement study will imitate all key peculiarities expected to be present during future missions to Mars (i.e. ultra long duration flight, need for autonomy, complicated communication due to signal delay, and limited stock of expendables).


In co-operation with:
European Space Agency (ESA) and the Russian Academy of Sciences.


External relationships:
Mark Neerincx, Jack van Loon, Karl Tuyls.


Project team:

Rene Ahn, Emilia Barakova, Roman Gorbunov.




Gorbunov R., Barakova E., Rauterberg M. (2013). Design of social agents. Neurocomputing, vol. 114, pp. 92-97 [IF=1.634].


ENTERTAINMENT: Theory of Entertainment Computing


The main objective of this project is to find out what the actual state-of-the-art can say about the different effects of entertainment technology on human behavior. A first literature research study was done to investigate in particular the positive effects (compared to the whole body of literature about negative effects).



Department of Industrial Design, Eindhoven University of Technology, Netherlands.


Ben Salem. This project is carried out in the scope of the IFIP Working Group 14.3 "Theoretical Basis of Entertainment".


Nakatsu R., Rauterberg M., Vorderer P. (2005). A new framework for entertainment computing: from passive to active experience. In: F. Kishino et al. (Eds.): ICEC 2005, Lecture Notes in Computer Science, vol. 3711, pp. 1 – 12.


Rauterberg M. (2004). Positive effects of entertainment technology on human behaviour. In: R. Jacquart (ed.), Building the Information Society (pp. 51-58). IFIP, Kluwer Academic Press.

Rauterberg M. (2004).
Positive effects of VR technology on human behavior. In: Proceedings of The 14th International Conference on Artificial Reality and Telexistence - ICAT 2004 (pp. 85-88). KAIST & VRSJ [ISSN 1345-1278].


Rauterberg, M. (2004). Entertainment technology and human behaviour: literature search. Technical Note, No. TUE-ID-04-07-1, Technical University Eindhoven, The Netherlands.
All documents listed in Technical Note, No. TUE-ID-04-07-1 are zipped for download in alphabetically separate files by the first letter of first author's family name:
[A] [B] [C] [D] [EF] [G] [H] [IJ] [K] [L] [M] [NO] [PQ] [R] [S] [T] [UV] [W] [XYZ]


Salem B., Rauterberg M. (2005). Aesthetics as a key dimension for designing ubiquitous entertainment systems. In: M. Minoh & N. Tosa (eds.) The 2nd International Workshop on Ubiquitous Home—ubiquitous society and entertainment (pp. 85-94). NICT Keihanna and Kyoto.


HX.research: Exploring Human Experience through Physiological Measurements


Design of interactive products and systems started several decades ago. At the begin cognition was emphasized. Recently emotional aspects are added. This project investigates user expereinces beyond these two relevant aspects of cognition and emotion. We are going into so far unexplored unconscious dimensions.

[For more background information feel free to look at HXresearch]



Erasmus Mundus Doctorate Programme in Interactive and Cognitive Environments.


Andreu CATALA and Marta DIAZ (UPC, Barcelona, Spain).


Project team:

Jun Hu (co-supervisor), Leonid IVONIN (PhD), Huang Ming CHANG (PhD)


Rauterberg, M. (2013). User experience beyond cognition and emotion.


Ivonin L., Chang H.M., Chen W., Rauterberg M. (2013). Measuring archetypal experiences with physiological sensors. SPIE newsroom, URL: [DOI 10.1117/2.1201301.004669]


Chang HM., Ivonin L., Díaz M., Catalŕ A., Chen W., Rauterberg M. (2013). From mythology to psychology: Identifying archetypal symbols in movies. Technoetic Arts: A Journal of Speculative Research, vol. 11, no. 2, pp. 99–113.


Ivonin L., Chang H.M., Chen W., Rauterberg M. (2012). Unconscious emotions: Quantifying and logging something we are not aware of. Personal and Ubiquitous Computing,  vol. 17, no. 4, pp. 663-673 [IF=1.133].

Chang H.M., Ivonin L., Diaz M., Catala A., Chen W., Rauterberg M. (2013). Experience the world with archetypal symbols: A new form of aesthetics. In: N. Streitz, C. Stephanidis (eds.): Distributed, Ambient, and Pervasive Interactions DAPI - HCI international (Lecture Notes in Computer Science, vol. 8028, pp. 205–214), Berlin Heidelberg: Springer [CpD=0.499].

Chang H.M., Ivonin L., Diaz M., Catala A., Chen W., Rauterberg M. (2013). What do we feel about archetypes: Self-reports and physiological signals. In: Proceedings of 21st European Signal Processing Conference (September 9-13, 2013, Marrakech, Morocco; paper no. 1569742621).

Chang H.M., Ivonin L., Chen W., Rauterberg M. (2013). Feeling something without knowing why: Measuring emotions toward archetypal content. In: M. Mancas, N. d’ Alessandro, X. Siebert, B. Gosselin, C. Valderrama, T. Dutoit (eds.) Proceedings of Intelligent Technologies for Interactive Entertainment - INTETAIN (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol. 124, pp. 22–31). © Institute for Computer Sciences, Social Informatics and Telecommunications Engineering: Springer.


PARA: Thinker versus Tinker


The main objective of this project is to find out what are the main characteristics of the relationship between science ("thinker") and engineering ("tinker"). These two different approaches can be described as different paradigms. Alfred Nobel's last will and testament is clear and concise: "one part to the person who shall have made the most important discovery or invention within the field of physics". This distinction in discovery and invention gives us the opportunity to analyse the data of all Nobel Prize laureates in Physics from begin on. In the category 'invention' we subsume 'invention' and 'technical development'. First results show very interesting outcomes.



Department of Industrial Design, Eindhoven University of Technology, Netherlands.


Christoph Bartneck. We are in corresponding contact with Claus D. Hillebrand.


Rauterberg, M. (2005). Thinker versus Tinker.


Press Coverage:

The Economist (2010). Daily Chart. [PDF]


Bartneck C., Rauterberg M. (2007). Physics Nobels should favour inventions. Nature, vol. 448, no. 7154, p. 644.

Bartneck C., Rauterberg M. (2007). HCI reality - an unreal tournament?. International Journal on Human-Computer Studies, vol. 65, no. 8, pp. 737-743.

Rauterberg M. (2006). HCI as an engineering discipline: to be or not to be!?. African Journal of Information and Communication Technology, vol. 2, no. 4, pp. 163-184.

Rauterberg M. (2003). Human computer interaction research--a paradigm clash?. In: A. Kameas & N. Streitz (eds), Tales of the Disappearing Computer (Proceedings of the Disappearing Computer Conference, 1-4 June Santorini, Greece). Patras, Greece: CTI Press.

Rauterberg M. (2000). How to characterize a research line for user-system interaction. IPO Annual Progress Report, No. 35, pp. 66-86.


ROBOT: Emotional Humanoid Robot Behaviour


The main objective of this project is to find out what are the main features for designing a humanoid robot which can show emotional behaviour. Based on the state of the art in cognitive science, we are looking for a new kind of architecture to control robot behaviour: from "sensor-control-actuator" to "perception-emotion-action".


Nirvana Technology Inc, Japan.


Ben Salem.


Rauterberg M., Salem B., Mortel van de D. (2005). From passive to active forms. In: L. Feijs, S. Kyffin & B. Young (eds.) Design and semantics of form and movements (pp. 110117), Koninklijke Philips Electronics N.V.


Salem B., Rauterberg M. (2005). Power, Death and Love: a trilogy for entertainment. In: F. Kishino et al. (eds.), Entertainment Computing--ICEC 2005. Lecture Notes in Computer Science, vol. 3711, pp. 279–290.


Rauterberg M. (2004). Enjoyment and entertainment in East and West. In: M. Rauterberg (ed.), Entertainment Computing--ICEC 2004. Lecture Notes in Computer Science, vol. 3166, pp. 176181.

SEAT: Smart Technologies for Stress Free Air Travel


Overall Summary: Passenger comfort is clearly a main factor in user’s acceptance of transportation systems. An individual's reaction to a vehicle environment depends not only on the physical inputs but also on the characteristics of the individual. The findings of a number of passengers’ surveys and comfort related research indicate that there is not a universal optimal setting for comfort related parameters in a plane. Hence individual passengers are always likely to have certain conflicting requirements as perception for comfort is affected by a variety of factors - gender and ethnicity among the most important ones. SEAT promotes a radically new concept where passenger comfort is taken to a new level. The SEAT system will develop smart responsive seats and interior environment with the capability of detecting physiological and psychological changes of passenger’s condition in real time. This in turn will be analyzed and appropriate adjustments such as temperature control, air ventilation, seat parameters etc. put in place. Furthermore each passenger will be able to create its own retrieve with personal entertainment and office characteristics in place. The entire approach is to create an environment that responds to the individual requirements and desires and is not centrally controlled or manually adjusted. The system is based on advanced technologies and systems developed by the partners as breakthrough research developments or other advanced technologies. A prototype of this new system is to be demonstrated to the public at the end of the project.
WP4 "Entertainment and Office" (leader M. Rauterberg). The objectives of WP4 are:
• To develop a novel system that allows interactive entertainment at different levels – individual, group and central.
• To implement physical activities in terms of desired locomotion patterns as part of the entertainment.
• To integrate office characteristics in the entertainment environment to allow effortless switch to different modes of activities.
• To develop a concept for “subliminal entertainment”, where intelligent selection of sound images etc facilitate balanced psychological status.

Sponsored by:
Sixth Framework Programme of the European Commission; Priority [4] [Aeronautics and Space]; Proposal/Contract no.: 30958/ AST5-CT-2006-030958.

In co-operation with:
Imperial College London (project leader: Prof Ferri M.H.Aliabadi); Acústica y Telecomunicaciones, S.L; Asociación de Investigación de la Industria Textil; Antecuir S.L.; Czech Technical University; Design Hosting Software Ltd.; Eidgenoessische Technische Hochschule Zuerich; Insitituto Tecnologico del Calzado y Conexas; Queen Mary and Westfield College; StarLab; Thales.

WP4 Project member:

Hao LIU; Chee Fai TAN.

see project website.


see project website.

Finished Research Projects at Eindhoven University of Technology (1998-2006)

eC: e-commerce and e-payment


The e-commerce research group at IPO is adopting a Human-Computer Interaction (HCI) approach to tackle the issues of trust, perceived security, usability and acceptability in e-commerce system design. These research activities are carried out in conjunction with the research initiative entitled "Enabling Electronic Commerce" that has been set up between the University of Tilburg and the Eindhoven University of Technology.

A description of the research conducted by this research group is given as follows: 

Dennis Abrazhevich: user-related factors in online payment systems;

Florian N. Egger: meeting consumer trust requirements in e-commerce system design;

Fulko van Westrenen: modeling risk-taking behaviour.



Dennis Abrazhevich (2004). Electronic Payment Systems: a User-Centered Perspective and Interaction Design. PhD Thesis, Technical University Eindhoven.

LivingLab: a smart home environment for user studies



- To study interaction within a ‘smart home’.

- Focus on elderly as a target group although none of the technologies studied are exclusively for this group.

- Focus on ‘long-term interaction’: how the technologies experimented with, fit the every-day life of users.

- Accrue knowledge on software architectures and user-centered design techniques issues arising.


Project Description:

This project is moving from its initial definition phase to what you could call the set-up phase. The ‘Living Lab’ will provide the facilities for hosting experimental subjects for long periods (e.g., two weeks) and to evaluate their experience with: - Stationery purpose-specific information and communication appliances (see related ‘Tele-visit service and Co-presence’ and ‘Infant Interaction’ projects). - Position and activity sensitive browsing: a wireless in-home service, available everywhere in the house through a single physical device, which interprets the activity of the inhabitant. -'Smart home' technologies for supporting the well being of residence: e.g., air quality control, mobility and nutritional recommender agents.



Markopoulos P., Rauterberg M. (2000). LivingLab: a white paper. IPO Annual Progress Report, No. 35, pp. 53-65.

SMARTEX: Smart  Exhibition Booth


The main objective of this project is to find out if an adaptive exhibition booth yields a more effective and efficient exhibition. We measure efficiency by the amount of time required to perform an information search task. It is proposed to do so with a comparative, experimental study between a conventional booth (CB) and an adaptive booth (AB). We aim to establish some guidelines to be integrated in the specifications of an exhibition booth. Such guidelines would include both the necessary infrastructure and the required information (both content and media).


Gielissen BV, Eindhoven, Netherlands.


Ben Salem.


Salem B., Rauterberg M. (2004). Multiple User Profile Merging (MUPE): Key Challenges for Environment Awareness. Lecture Notes in Computer Science, Vol. 3295, pp. 196–206.

VIP: Visual interaction platform


Human Computer Interaction through command languages, menu selections and drag and drop techniques, as is prevalent in desktop computers, is far removed from the usual way in which we interact with our environment. Existing interaction devices, moreover, can be very bothersome in situations where several people have to co-operate, together with the computer, on one problem. These problems have stimulated the development of a new generation of computer interfaces in which the virtual world of the computer is better integrated with the real world of the user(s). Such interfaces are referred to as Augmented Reality (AR) or as Natural User Interfaces (NUIs). The expected success of this approach lies in its ability to build on fundamental human skills: namely to interact with real world subjects and objects. NUIs are multimodal(i.e., they integrate different forms of interaction) and therefore allows the users to choose the appropriate and/or individually preferred interaction style for every (inter)action.

The VIP is one instance of such a NUI that is being developed on the BUILD-IT system at IPO. Pattern recognition is the key technology for such next generation user interfaces and therefore determines to a large extent how feasible different interaction modalities (such as spoken language, hand writing, gesture, etc) are. Some of the proposed projects aim at extending pattern recognition capabilities within the existing VIP.


Aliakseyeu D., Champoux B., Martens J-B., Rauterberg M., Subramanian S. (2003). The Visual Interaction Platform. In: M. Rauterberg, M. Menozzi & J. Wesson (eds.), Human-Computer Interaction--INTERACT '03 (pp. 1101-1102). IOS Press.


Aliakseyeu D. (2002). A computer support tool for the early stages of architectural design. PhD Thesis, Technical University Eindhoven.


Aliakseyeu D., Subramanian S., Martens J-B. & Rauterberg M. (2002). Interaction techniques for navigation through and manipulation of 2D and 3D Data. In: Proceedings of Eighth Eurographics Workshop on Virtual Environments (S. Müller, W. Stürzlinger, Eds.).


Aliakseyeu D., J-B. Martens, S. Subramanian, M. Vroubel and W. Wesselink (2001). Visual Interaction Platform. In: Proceedings of INTERACT 2001, July 2001, Tokyo, Japan, pp. 232-239.


Aliakseyeu D. & Martens J-B. (2001). Physical paper as the user interface for an architectural design tool. In: Proceedings of INTERACT 2001, July 2001, Tokyo, Japan, pp. 680-681.


Subramanian S. & IJsselsteijn W. (2000). Survey and Classification of Spatial Object Manipulation Techniques. In: Proceedings of OZCHI 2000, Interfacing Reality in the New Millennium, December 2000, pp. 330-337.

Finished Research Projects at Swiss Federal  Institute of Technology ETH (1989-1998)

Internet-Study about Internet Addiction




Egger O. & Rauterberg M. (1996). Internet behaviour and addiction. ETH Technical Report AP-01-96, Swiss Federal Institute of Technology, Zurich.


AMME: an Automatic Mental Model Evaluator to Analyse User Behaviour Recorded on Logfiles 


To support the human factors engineer in designing a good user interface of interactive computer programs, a method has been developed to analyse the empirical data of the interactive user behaviour recorded in a logfile. The sequences of keystrokes produced by the user contain a lot of information about the mental model of this user, the individual problem solution strategies for a given task and the hierarchical structure of the task - subtasks relations. The developed tool, AMME, can analyse the keystroke sequences of users and come up automatically with a Petri-Net description of the task dependent mental model of the user. This mental model is the input to a Petri-Net simulator, so the human factor engineer will be able to analyse his interface design without further empirical investigations to detect interactive deadlocks, mis-designed areas, etc. The sequences of decisions and actions produced by users contain much information about the mental model of this user, the individual problem solution strategies for a given task and the underlying decision structure. We distinguish between (1) the logical structure, (2) the sequential goal structure, and (3) the temporal structure. The following results of a experiment of learning are important: Our hypothesis that "learning how to solve a specific task with a given system means that the behavioural complexity decreases and the cognitive complexity increases'' seems to be correct. Furthermore, we can conclude from the empirical results that we must discriminate between the logical structure and the time structure of a task. A new understanding of the relationship between the mental knowledge structure and the decision process for action planning is possible.

Rauterberg, M. (1993) AMME: an Automatic Mental Model Evaluation to analyze user behaviour traced in a finite, discrete state space. Ergonomics 36(11):1369-1380.

Rauterberg, M. (1995).
About a framework for information and information processing of learning systems. In E. Falkenberg, W. Hesse & A. Olive (Eds.), Information System Concepts--Towards a consolidation of views (IFIP Working Group 8.1, pp. 54-69). London: Chapman&Hall.

Rauterberg, M. (1996).
About faults, errors, and other dangerous things. In C. Ntuen & E. Park (Eds.), Human Interaction with Complex Systems: Conceptual Principles and Design Practice (pp. 291-305). Norwell: Kluwer.

Rauterberg, M. (1996).
How to measure cognitive complexity in human-computer interaction. In R. Trappl (Ed.), Cybernetics and Systems '96 (Vol. 2, pp. 815-820). Vienna: Austrian Society for Cybernetic Studies.

Rauterberg, M. (1996).
Why and what can we learn from human errors. In A. Özok & G. Salvendy (Eds.), Advances in Applied Ergonomics (pp. 827-830). West Lafayette: USA Publishing.

Rauterberg, M. (1996).
A Petri net based analyzing and modelling tool kit for logfiles in human-computer interaction. In H. Yoshikawa & E. Hollnagel (Eds.), Proceedings 'Cognitive Systems Engineering in Process Control--CSEPC'96 (pp. 268-275). Kyoto University: Graduate School of Energy Science.

Rauterberg, M. & Aeppli, R. (1995).
Learning in man-machine systems: the measurement of behavioural and cognitive complexity. In Proceedings of IEEE International Conference on Systems, Man and Cybernetics--SMC'95 (Vol. 5, IEEE Catalog Number 95CH3576-7, pp. 4685-4690). Piscataway: Institute of Electrical and Electronics Engineers.

Rauterberg, M. & Aeppli, R. (1996).
How to measure the learning process in man-machine systems. In A. Özok & G. Salvendy (Eds.), Advances in Applied Ergonomics (pp. 312-315). West Lafayette: USA Publishing.

Rauterberg, M. & Aeppli, R. (1996).
How to measure the behavioural and cognitive complexity of learning processes in man-machine systems. In P. Carlson & F. Makedon (Eds.), Proceedings of 'Educational Multimedia and Hypermedia'--ED-MEDIA'96 (pp. 581-586). Charlottesville: AACE.

BUILD-IT: a video-based interaction technique for CAD and other systems


It is time to go beyond the established approaches in human-computer interaction. After a serious critic of command language, menu selection, and desktop interfaces we discuss the two known approaches to overcome the obstacles and limitations: [immersive] Virtual Reality (VR), and Augmented Reality (AR). Both design strategies are diametrically opposed: VR enriches the virtual world with real humans, while AR augments the real world with intelligent features. Only with the AR design strategy humans are able to behave as much as possible in a natural way: behavior of humans in the real world with other humans and/or real world objects. Our interest in human centred design let us follow this idea. Based on the fundamental constraints of natural way of interacting we derive a set of recommendations for the next generation of user interfaces: the Natural User Interface (NUI). A NUI is implemented with a video-based interaction technique for a planning tool as a front end for CAD systems. Two beamers are used as output devices, one video camera is used for the input. The 3D models of objects can be handled with a wooden brick as an universal interaction handler. The physical position of the brick is determined via the video camera. The implementation is carried with MET++ (see Perspectix) on a Silicon Graphics O2 computer.

Genealogical tree of the "Build-It" family [History PDF]:

  the "original" DigitalDesk 

see Pierre Wellner (1991)

see Pierre Wellner (1993)

see Pierre Wellner's Thesis (1994)

[for further references see P. Dourish (2001)]
Bricks in interaction
see Fitzmaurice, Ishii & Buxton (1995)

the ARIEL system
see Wendy Mackay et al (1995)

the DigitalDrawingBoard
Wendy Mackay et al (1996)

our first ideas
see Rauterberg & Steiger (1996)

the DigitalPlayingDesk at ETH Zurich
Rauterberg et al (1996)

the graspable interfaces at artec (1997)
  the Build-It system at ETH Zurich

see Rauterberg et al (1997)

the Intelligent Planning Table in Stuttgart, see
FhG-IPA (2001)

FhG-IPA (2004)
Build-It in architectural design

see ETHZ (1999)

the Visual Interaction Platform

see VIP project (1999) at TU/e

the Intelligent Planning Table, see
TU Clausthal (2000)

TU Chemnitz (2000)
University Magdeburg (2001)
University Stuttgart (2002)
University Bremen (2003)
RWTH Aachen (2003)
the Communicate-It project

see Guttormsen et al (2000)



Patent Application (1996): Interaction area for data representation. EP0859977 [inventors: M. Bichsel & M. Rauterberg]

Patent Application (1997): Interaction area for data representation. PCT/CH97/00336 [inventors: M. Bichsel & M. Rauterberg]

Patent Application (1998): Interaction area for data representation. WO 98/13745 [inventors: M. Bichsel & M. Rauterberg]
BUILD-IT Manual (1998) [pdf]


Fjeld, M., M. Morf, H. Krueger (2004). Activity theory and the practice of design: evaluation of a collaborative tangible user interface. Int. J. Human Resources Development and Management, Vol. 4, No. x,  pp.

Fjeld M. & M. Rauterberg (2001) Designing for Tangible Interaction: The BUILD-IT Project. ERCIM News no. 46, pp. 34-35.


Fjeld, M. (2001) Designing for tangible interaction. ETH PhD Thesis [pdf & movies].


Rauterberg, M. & Steiger, P. (1996) Pattern recognition as a key technology for the next generation of user interfaces. In: Proceedings of IEEE International Conference on Systems, Man and Cybernetics -- SMC'96, October 14 - 17, 1996 in Beijing, China (Vol. 4, IEEE Catalog Number: 96CH35929) Piscataway: Institute of Electrical and Electronics Engineers, pp. 2805-2810.

Rauterberg, M., Bichsel, M., Leonhardt, U. & Meier, M. (1997).
BUILD-IT: a computer vision-based interaction technique of a planning tool for construction and design. In S. Howard, J. Hammond & G. Lindgaard (Eds.), Human-Computer Interaction--INTERACT'97 (pp. 587-588).

Digital Playing Desk: a video-based interaction technique for the game Go-bang 


To compare the advantages and disadvantages of a "Graspable User Interface" a field study was carried out. During five days of the largest computer fair in Switzerland four different computer stations with (1) a command language, (2) a mouse, (3) a touch screen, and (4) a DigitalDesk interface was presented for public use. In this version of the DigitalDesk the user has to play a board game by moving a real chip on a virtual playing field against a virtual player. The task was to win the computer game "Go-bang". The reactions of the virtual player were simulated by "emoticons" as colored comic strip pictures with a corresponding sound pattern. We investigated the effects of these four different interaction techniques with two different methods: (a) an inquiry with a questionnaire, and (b) a field test for public and anonymous use. (a) Results of the inquiry: 304 visitors rated the usability of all four stations on a bipolar scale. The touch screen station was rated as the easiest to use interaction technique, followed by the mouse and DigitalPlayingDesk interface; the "tail-light" was the command language interface. One very important result was a significant correlation between "age" and "DigitalPlayingDesk usability". This correlation means that older people prefer significantly more a graspable user interface in form of the DigitalPlayingDesk than younger people. (b) Results of the field test: The analysis of 9'006 automatically recorded contacts to one of the four stations shows that the highest chance to win the game could be observed for the DigitalDesk interface technique. We conclude that the DigitalPlayingDesk as a graspable user interface with emotional, non-verbal feedback is a promising candidate for the next generation of dialog techniques.

Rauterberg, M. &
Szabo, K. (1995). A design concept for N-dimensional user interfaces. In Proceedings of '4th International Conference INTERFACE to Real & Virtual Worlds' (pp. 467-477). Montpellier: EC2 & Cie.

Rauterberg, M., Mauch, T. & Stebler, R. (1996).
The Digital Playing Desk: a Case Study for Augmented Reality. In Proceedings IEEE International Workshop on Robot and Human Communication--ROMAN'96 (IEEE Catalog Number: 96TH8179, pp. 410-415). Piscataway: Institute of Electrical and Electronics Engineers.

Rauterberg, M., Mauch, T. & Stebler, R. (1996).
What is a promising candidate for the next generation of interface technology. In Proceedings of '5th International Conference INTERFACE to Real & Virtual Worlds' (pp. 95-103). Montpellier: EC2 & Cie.

Rauterberg, M., Mauch, T. & Stebler, R. (1996).
How to improve the quality of human performance with natural user interfaces as a case study for augmented reality. In A. Mital, H. Krueger, S. Kumar, M. Menozzi & J.E. Fernandez (Eds.), Advances in Occupational Ergonomics and Safety I (pp. 150-153). Cincinnati: International Society for Occupational Ergonomics and Safety.

MetUse: Metrics for Usability beyond Usability Testing 


One of the main problems of standards (e.g., DIN 66234, ISO 9241) in the context of usability of software quality is, that they can not be measured in product features. We developed and validated a new approach to measure user-interface quality in a quantitative way. First, we developed a concept to describe user-interfaces on a granularity level, that is detailed enough to preserve important interface characteristics, and is general enough to cover most of known interface types. We distinguish between different types of 'interaction-points'. With these kinds of interaction-points we can describe several types of interfaces (CUI: command, menu, form-fill-in; GUI: desktop, direct manipulation, multimedia, etc.). We carried out two different comparative usability studies to validate our quantitative measures. The results of one other published comparative usability study can be predicted. The empirical outcomes of four comparison studies with six different interfaces are used to validate our quantitative usability metrics.

Rauterberg, M. (1993).
A product oriented approach to quantify usability attributes and the interactive quality of user interfaces. In H. Luczak, A. Cakir & G. Cakir (Eds.), Work With Display Units--WWDU'92 (pp. 324-328). Amsterdam: North-Holland.

Rauterberg, M. (1993).
Quantitative Measures to Evaluate Human-Computer Interfaces. In: M. Smith & G. Salvendy (Eds.) Human-Computer Interaction: Applications and Case Studies. (Advances in Human Factors/Ergonomics, Vol. 19A, pp. 612-617), Amsterdam: Elsevier.

Rauterberg, M. (1995).
Usability evaluation: an empirical validation of different measures to quantify interface attributes. In T. Sheridan (Ed.), Analysis, Design and Evaluation of Man-Machine Systems 1995 (Vol. 2, pp. 467-472). Oxford: Pergamon.

Rauterberg, M. (1995).
Four different measures to quantify three usability attributes: 'feedback', 'interactive directness' and 'flexibility'. In P. Palanque & R. Bastide (Eds.), Design Specification and Verification of Interactive Systems--DSV-IS'95 (pp. 209-223). Wien New York: Springer.

Rauterberg, M. (1996).
An empirical validation of four different measures to quantify user interface characteristics based on a general descriptive concept for interaction points. In Proceedings of IEEE Symposium and Workshop on Engineering of Computer-Based Systems (IEEE Order Number: PR07355, pp. 435-441). Los Alamitos: IEEE Computer Society Press.

Rauterberg, M. (1996).
How to measure and quantify usability attributes of user interfaces. In A. Özok & G. Salvendy (Eds.), Advances in Applied Ergonomics (pp. 429-432). West Lafayette: USA Publishing.

Rauterberg, M. (1996).
How to measure the ergonomic quality of user interfaces in a task independent way. In A. Mital, H. Krueger, S. Kumar, M. Menozzi & J.E. Fernandez (Eds.), Advances in Occupational Ergonomics and Safety I (pp. 154-157). Cincinnati: International Society for Occupational Ergonomics and Safety.

SOUND: Automatic Sound Generation of Everyday Sound 


The objective of this project is the development of concepts, methods and a prototype for an audio frame work. This audio frame work shall describe everyday sounds on a highly abstract semantic level. We describe every sound as the result of one or several interactions between one or several objects at a certain place and in a certain environment. The hearing of sounds in everyday life is based on the perception of events and not on the perception of sounds as such. For this reason, everyday sounds are often described by the events they are based on. In this project, a framework concept for the description of sounds will be developed, in which sounds can be represented as auditory signal patterns along several descriptive dimensions of various objects. We especially look for possibilities to describe the sound class 'solid objects', in particular the class of the primitive sounds 'knock' ('strike', 'hit'), because this class of sounds occurs very frequently in everyday life, the interacting objects can be easily and well described by their material characteristics and the knowledge of solid state physics can be used. As an example the falling of a spherical elastic object onto a linear elastic beam is physically and mathematically modelled and implemented.

Darvishi, A., Munteanu, E., Guggiana, V., Schauer, H., Motavalli, M. & Rauterberg, M. (1995).
Designing environmental sounds based on the results of interaction between objects in the real world. In K. Nordby, P. Helmersen, D. Gilmore & S. Arnesen (Eds.), Human-Computer Interaction -- INTERACT'95 (pp. 38-42). London: Chapman & Hall.

Munteanu, E., Guggiana, V., Darvishi, A., Schauer, H., Rauterberg, M. & Motavalli, M. (1995). Physical modelling of environmental sounds. In F. Pedrielli (Ed.), Proceedings of '2nd International Conference on Acoustic and Musical Research'-- CIARM'95 (pp 107-112). Ferrara: Universita di Ferrara.

Munteanu, E., Darvishi, A., Guggiana, V., Schauer, H., Rauterberg, M. & Motavalli, M. (1995).
Synthesis of environmental sounds in multimedia applications. In H. Maurer (Ed.), Educational Multimedia and Hypermedia-- ED-MEDIA'95' (p. 793). Charlottesville: Association for the Advancement of Computing in Education.

Rauterberg, M., Motavalli, M., Darvishi, A. & Schauer, H. (1994).
Automatic sound generation for spherical objects hitting straight beams based on physical models. In T. Ottmann & I. Tomek (Eds.), Educational Multimedia and Hypermedia--ED-MEDIA'94 (pp. 468-473). Charlottesville: Association for the Advancement of Computing in Education.

BOOM: User-oriented Interface Design of Multi-Media Information Systems 


An eye movement recording experiment has been carried out to test the hypothesis that fixation patterns contain design relevant information. Eight users solved ten tasks with a multi media information system. During this task solving process all eye movements were continually recorded. The following aspects should be considered to come up with design relevant knowledge: (1) pictorial versus textual objects, (2) explicit versus implicit design, (3) task relevant information, and (4) the object area size. We developed empirically proved approaches to solve several design problems for multi media information systems.

Two experiments were carried out to estimate the effect of sound feedback: (1) operating an assembly line simulator with sound feedback of hidden events. The results indicate, that additional sound feedback improves significantly the user performance. And (2) queries in database search with sound feedback of the search quality. The results of this empirical investigation indicate that additional sound feedback does not improve the user performance overall. But, if we differentiate between users, who prefer sound, and those, who do not, we can find significant improvements. We can conclude that sound feedback is necessary, but must be eligible and customizable. An eye movement recording experiment has been carried out to pinpoint the control of the primary attention focus. The user's attention focus is strongly influenced by dynamic processes on the screen. E.g., if a pop up menu of an appropriate size automatically appears on the static screen, then the user's attention focus changes to this new element. The results of our investigation indicate that design of a multi media screen layout can be validated and optimised using eye recording data.

A method for 'metaphor engineering' is introduced in the context of participatory multimedia design. Our hypothesis, that adults talk to children more with a metaphorical language than to other adults, was empirically verified. Especially male adults do not qualify for the job as metaphor engineers. Metaphors are powerful, but not sufficient to come up with a good interface. Single, over-detailed metaphors can be too restrictive and unwieldy. Effective interface metaphors often evolve over time through design, evaluation and redesign. The developed approach for 'metaphor engineering' guarantees that the starting point of the design cycles can be optimised.

Rauterberg, M. (1995).
Mock-up or eye movement recording: Big paybacks from a 'discount' method for multimedia interface design. In H. Maurer (Ed.), Educational Multimedia and Hypermedia--ED-MEDIA'95' (p. 800). Charlottesville: Association for the Advancement of Computing in Education.

Rauterberg, M. & Hof, M. (1994). How to get a fitting metaphor for a multimedia interface?. In A. Grieco, G. Molteni, E. Occhipinti & B. Piccoli (Eds.), Book of Short Papers of 4th International Conference on Work with Display Units--WWDU'94 (Volume 3, pp. D15-D17). University of Milan: Institute of Occupational Health.

Rauterberg, M. & Hof, M. (1995).
Metaphor engineering: a participatory approach. In W. Schuler, J. Hannemann & N. Streitz (Eds.), Designing User Interfaces for Hypermedia (pp. 58-67). Berlin: Springer.

Rauterberg, M. & Styger, E. (1994). Positive effects of sound feedback during the operation of a plant simulator. In B. Blumenthal, J. Gornostaev & C. Unger (Eds.), Human Computer Interaction (Lecture Notes in Computer Science, Vol. 876, pp. 35-44). Berlin: Springer.

Rauterberg, M., Berny, P., Lordong, G., Schmid, A., Zemp, M. & Zürcher, T. (1995).
Designing multi media user interfaces with eye recording data. In A. Grieco, G. Molteni, E. Occhipinti & B. Piccoli (Eds.), Work with Display Units--WWDU'94 (pp. 265-270). Amsterdam: North-Holland.

Rauterberg, M., Styger, E., Baumgartner A., Jenny, A. & de Lange, M. (1995).
Additional Sound Feedback in Man-Computer Interaction: two empirical investigations. In A. Grieco, G. Molteni, E. Occhipinti & B. Piccoli (Eds.), Work with Display Units--WWDU'94 (pp. 259-264). Amsterdam: North-Holland.

RENOIR: Requirements Engineering Network Of International Cooperating Research Groups 


The ultimate measurement for software quality is the degree to which user requirements are fulfilled by a system. Early elicitation and correct definition of requirements prevents costly rework during later development stages and provides the foundation for building high quality systems. Therefore, requirements engineering is considered as a more and more crucial part of the system life cycle.
During requirements engineering the user and engineers have to find a way from an initially opaque and diverse system understanding to exact, reconciled and at least partially formalized system specifications. A multitude of methods from software engineering, ethnology, social sciences, and psychology have been adapted to support this process and to achieve a growing quality of the requirements specification as a foundation of higher system quality. Most of these methods arerelying on adequate specification languages which are expressive and formal enough so that therepresented quality requirements can be verified or validated.
Our main research goal is the development of user centered design methods for the requirement phase.

Rauterberg, M. (1994).
Human factors and modern software development. In B. Theodoulidis (Ed.), Proceedings of the 5th Workshop on the Next Generation of CASE Tools (Department of Computer Science, Memoranda Informatica 94-25, pp. 170-179). Enschede: University of Twente.

Rauterberg, M. (1996).
Moderation instead of modelling: some remarks about formal and informal reengineering methods. In R. Koubek & W. Karwowski (Eds.), Manufacturing Agility and Hybrid Automation I (pp. 167-170). Louisville: IEA Press.

Rauterberg, M. & Strohm, O. (1992).
Work Organization and Software Development. Annual Review of Automatic Programming 16(2):121-128. (, 844 KB)

Rauterberg, M. & Strohm, O. (1994).
About the benefits of user-oriented requirements engineering. In K. Pohl, G. Starke & P. Peters (Eds.), First International Workshop on Requirements Engineering: Foundation of Software Quality--REFSQ'94 (Aachener Beiträge zur Informatik, Band 6, S. 118-128). Aachen: Verlag der Augustinus Buchhandlung.

Rauterberg, M., Strohm, O. & Kirsch, C. (1995).
Benefits of user-oriented software development based on an iterative cyclic process model for simultaneous engineering. International Journal of Industrial Ergonomics 16(4-6):391-410.

Home Page