Augmented Reality: Information at · PDF fileAugmented Reality: Information at Focus Christine Ludwig Siemens Business Services, C-LAB Christian Reimann Universität Paderborn, C-LAB

Augmented Reality: Information at Focus

Christine Ludwig Siemens Business Services, C-LAB

Christian Reimann

Universität Paderborn, C-LAB

C-LAB Report

Vol. 4 (2005) No. 1

Cooperative Computing & Communication Laboratory

ISSN 1619-7879

C-LAB ist eine Kooperation der Universität Paderborn und der Siemens Business Services GmbH & Co OHG

www.c-lab.de [email protected]

C-LAB Report Herausgegeben von Published by

Dr. Wolfgang Kern, Siemens Business Services GmbH & Co OHG

Prof. Dr. Franz-Josef Rammig, Universität Paderborn Das C-LAB - Cooperative Computing & Communication Laboratory - leistet Forschungs- und Entwicklungsarbeiten und gewährleistet deren Transfer an den Markt. Es wurde 1985 von den Partnern Nixdorf Computer AG (nun Siemens Business Services GmbH & Co OHG) und der Universität Paderborn im Einvernehmen mit dem Land Nordrhein-Westfalen gegründet. Die Vision, die dem C-LAB zugrunde liegt, geht davon aus, dass die gewaltigen Herausforderungen beim Übergang in die kommende Informationsgesellschaft nur durch globale Kooperation und in tiefer Verzahnung von Theorie und Praxis gelöst werden können. Im C-LAB arbeiten deshalb Mitarbeiter von Hochschule und Industrie unter einem Dach, in einer gemeinsamen Organisation, an gemeinsamen Projekten mit internationalen Partnern eng zusammen. C-LAB - the Cooperative Computing & Cooperation Laboratory - works in the area of research and development and safeguards its transfer into the market. It was founded in 1985 by Nixdorf Computer AG (now Siemens Business Services GmbH & Co OHG) and the University of Paderborn under the auspices of the State of North-Rhine Westphalia. C-LAB’s vision is based on the fundamental premise that the gargantuan challenges thrown up by the transition to a future information society can only be met through global cooperation and deep interaction of theory and practice. This is why, under one roof, staff from the university and from the industry cooperates closely on joint projects within a common research and development organization together with international partners. In doing so, C-LAB concentrates on those innovative subject areas in which cooperation is expected to bear particular fruit for the partners and their general well-being. ISSN 1619-7879 C-LAB Fürstenallee 11 33102 Paderborn fon: +49 5251 60 60 60 fax: +49 5251 60 60 66 email: [email protected] Internet: www.c-lab.de © Siemens Business Services GmbH & Co. OHG und Universität Paderborn 2005 Alle Rechte sind vorbehalten. Insbesondere ist die Übernahme in maschinenlesbare Form sowie das Speichern in Informationssystemen, auch auszugsweise nur mit schriftlicher Genehmigung der Siemens Business Services GmbH & Co. OHG und der Universität Paderborn gestattet. All rights reserved. In particular transfer of data into machine readable form as well as storage into information systems, (even extracts) is only permitted prior to written consent by Siemens Business Services GmbH & Co. OHG and Universität Paderborn.

C-LAB Report

C-LAB-TR-2005-1 Seite 3 von 12

Table of Content 1 Augmented Reality: Information at Focus....................................................................4

2 What is Augmented Reality? Vision: Seamless Integration of Real Environment and

Digital Information........................................................................................................4

3 Potential Applications of Augmented Reality ...............................................................4

3.1 Presentation & Visualization..................................................................................5

3.1.1 Augmented Reality for Sales Support ................................................................5

3.1.2 Augmented Reality in Geovisualization and Navigation.....................................6

3.1.3 Augmented Reality in Architecture and Interior Design......................................7

3.2 Industry..................................................................................................................7

3.2.1 Augmented Reality in Education and Training as well as Maintenance/Repair .7

3.2.2 Augmented Reality for Remote Support ............................................................8

3.2.3 Augmented Reality for Product Development ....................................................9

3.2.4 Augmented Reality for Production Facilities Construction .................................9

3.3 Edutainment ..........................................................................................................9

3.3.1 Augmented Reality as Museum’s Guide ............................................................9

3.3.2 Augmented Reality in the Tourism Industry .....................................................10

3.3.3 Augmented Reality in Game Applications........................................................11

4 Summary ...................................................................................................................12

C-LAB Report


1 Augmented Reality: Information at Focus Quick access to the right information at the right time and its efficient presentation gains more and more importance i. e. at business fairs, when presenting new products, or in edutainment areas like the tourism sector, due to the amount of digital information sources. Today, such information is communicated mostly through traditional forms of presentation and materials like books, video tapes, tutorials, etc. Augmented Reality technology offers an innovative possibility to present information in a brand new way exactly at the place where the information is needed: in the user‘s field of vision. Despite the technology’s youth its enormous potential can be recognized in different application areas. This White Paper provides an insight into the Augmented Reality technology, shows its basic potentials, and presents the most important fields of application as well as its viable added value.

2 What is Augmented Reality? Vision: Seamless Integration of Real Environment and Digital Information

Augmented Reality (= AR) is a new way of human-computer-interaction, which adds virtual objects to real scenes that are provided by a video camera in real time. Virtual objects complement the real picture by appearing at the correct positions. The digital information merges with the user’s environment, thus offering the possibility to perceive currently important information directly where it is needed. Augmented Reality unfolds its strength when objects cannot be changed physically, e.g. they cannot be turned off (like in production facilities) or they are unique (museum).

The adequate way of presentation can be selected according to the nature of the application (e.g. degree of mobility, infrastructure, hands-free operations). A wide variety of screen devices can be used for displaying the information. Starting with data glasses and clip-on-displays, the range goes up to Tablet PCs, PDAs or mobile phones with photo function. The augmentation happens context-dependent, meaning it suits and is derived from the object of interest, e.g. an assembly component. An

assembler’s real field of vision will then include important additional pieces of information like faded-in assembly directions. In this special case, Augmented Reality can replace the usual assembly manual or complement it.

3 Potential Applications of Augmented Reality In the following, application areas are presented for which the information transfer based on Augmented Reality offers a significant added value. For the purpose of clarity, the scenarios have been divided into three areas:

C-LAB Report


Presentation & Visualization: The appealing and innovative communication of information is particularly important for company divisions like „Sales“ and „Marketing“, i.e. when presenting a new product. With Augmented Reality, it is not only possible to visualize product information that otherwise would be difficult to experience or explain, like the internals of a running motor. The company also gets the chance to present itself as innovative and customers tend to project this perception onto the advertised product. Industry: Augmented Reality shows a considerable potential with industrial applications, too. In the industrial area it is not only possible to save costs and reduce down times through a more time-efficient maintenance, but even to decrease risks by replacing and improving dangerous training situations. Edutainment: In the future, the market demand for an entertaining knowledge transfer will steadily increase. On the one hand, this is based on the trend towards life-long learning, making it necessary for every generation to get involved in further education even beyond their actual job history. On the other hand, the consumer is constantly searching for new and unrivalled experiences due to the increase of more or less spectacular entertainment opportunities. Through the more natural interaction with the computer, enabled by Augmented Reality, novel digital experiences can be created. The consumer gets involved more intensely than usual, and thus identifies himself with the experienced. Using Augmented Reality to transfer knowledge allows for a very innovative and enjoyable form of information visualization and the opportunity to gather knowledge that cannot be experienced with common methods of knowledge transfer. There are numerous possibilities to use Augmented Reality in each of these three application areas. An extract will be described in the following.

3.1 Presentation & Visualization

3.1.1 Augmented Reality for Sales Support A study called “the future of shopping” found out that tomorrow’s customer expects a lot of product information and numerous services at the point of sale. However this must not necessarily happen from human to human. Many customers would also be willing to use technical support or information at the shelf. The nationally funded research project “AR-PDA“ (amongst others in cooperation with the University of Paderborn and Miele & Cie. KG) aimed at the development of a sales support based on Augmented Reality for the point of sale. The customer shall be able to request information about a certain device (e.g. kitchen stove) in the department store that will then be explained to him on the screen directly at the image on the device.

C-LAB Report


The use of Augmented Reality for product presentation promises a considerable added value for the automotive industry, too. Here, customers can receive information about the vehicle in a very innovative and understandable way as it is possible to present technical information more precisely in AR simulations. For example, customers can access a kiosk system at car dealers with Augmented Reality features at weekends when legally no personal information services are allowed to be offered. By this means, potential customers are able to receive all necessary information by themselves. Such measures would enhance the prestige of the brand and that of the single car dealer especially in regard to the innovation of the brand and the car dealer’s technical expertise.

3.1.2 Augmented Reality in Geovisualization and Navigation Different scenarios for territory oriented planning can benefit from the use of Augmented Reality. Most of these are particularly interesting for military purposes, but also for aid organizations during catastrophes. For example, Augmented Reality enables a more efficient planning of military training on a new territory. Different locations and training scenarios can be easily aligned, including territorial conditions like rivers, lakes and forests. A similar situation is given when refugee camps or the like have to be planned. Here, too, the location of tents, buildings, flats, etc., can be faded virtually into an unknown territory. This leads to a simulation that is more realistic than traditional planning methods. In addition, such planning often has to be conducted within very short time periods. Augmented Reality helps to do so. Augmented Reality also offers novel possibilities for individual navigation systems. A small display shows the path directly in the field of vision (e.g. the user’s glasses). Especially in areas without much infrastructure such navigation systems can provide a significant added value, as precise verbal descriptions using landmarks may be very difficult. The following picture shows the advantages and possibilities of Augmented Reality in an undeveloped territory like a desert.

C-LAB Report


(Source: http://www.informationinplace.com/Solutions/CaseStudies/case_RDECOM/case_RDECOM.html)

3.1.3 Augmented Reality in Architecture and Interior Design Augmented Reality presents a new way of decision support in the area of architecture and interior design. Similar to the geographical visualization (see above), special features of the location are grasped more precisely than it would be possible with two-dimensional maps. It is no longer necessary to build an expensive and time consuming model. A virtual one can be faded directly into the real picture. In contrast to the mere VR (virtual reality), which only allows for the presentation of virtual objects or real miniatures, reality can be augmented by virtual information. In this way, disadvantages can be eliminated and advantages of both models (real and virtual) are combined. Regarding interior design, AR enables to run simulations and tests of new furniture or additional elements like a fireplace or new stairs directly in the real image.

3.2 Industry

3.2.1 Augmented Reality in Education and Training as well as Maintenance/Repair

Augmented Reality offers an additional added value especially for applications where traditional training methods like films, workshops or books only insufficiently represent reality. If training under real conditions might be too dangerous or if the implementation requires complex and expensive training areas, Augmented Reality provides new possibilities as well. A training scenario for employees of an oil refinery was developed within the EU-funded research project “AMIRE”, making it possible to locate pipelines which have to be repaired exactly and to complement the exterior real view of the machines and devices by a virtual interior view. The training occurs directly at the place of the future operation, thus

C-LAB Report


improving the applicability of new knowledge. The very complex arrangement of pipelines in a refinery is enhanced by a virtual map and becomes therefore easier to understand by the employee. By using Augmented Reality, complexity can now be handled more efficiently. The respective devices of the schematic description can be identified more easily and more exactly in the real facilities. The user is given additional information about the presented facilities or pipelines and may freely decide how much additional information he wants to have. The following pictures exemplify the training possibilities of a tablet PC:

Maintenance is a similar field of application. Here, Augmented Reality supports technicians during their work. For the maintenance of facilities or complex machines, technicians need to have access to all relevant information. This might include information about which actions have to be executed or what has been already done in the past. By using a camera (e.g. integrated in the PDA or handheld PC), the system recognizes the damaged component automatically and includes all relevant information as well as the exact assembly instructions into the real picture. If the technician works at electronic system components, the connection diagram, for example, might pop up automatically. Thus, it is much easier for the technician to understand the complex system and to check the relevant measured values. With these additional information, mistakes and faults can be avoided. Moreover, the system records all processed work steps automatically and by this means checks their completeness. Instead of reading massive manuals, the technician receives the instructions of the single work steps in his field of vision. In this way, he can use his hands for work and always maintain the overview, even while working on unfamiliar tasks. Similar scenarios have already been realized successfully within the EU-funded research projects “ARVIKA” and “SNOW”.

3.2.2 Augmented Reality for Remote Support With the help of Augmented Reality, it is also possible to have a remote support when solving technical problems or even in the medical sector. A camera records the respective part of the machine or body. The picture is transmitted to an expert, who can advise the team verbally or with the aid of components virtually presented in the picture. This direct interactive support allows for a more accurate help than a mere verbal one.

C-LAB Report


3.2.3 Augmented Reality for Product Development Nowadays, product life-cycles have become quite shortly and new and consumer-oriented products have to be brought into the market quite quickly. Product development times have therefore been hurried up. Additionally, the product development is often done by the work of distributed engineering teams at different locations in different countries. Virtual product development has already been adopted in many industry sectors as a measurement to reduce costs, for example in the automotive sector. Within the research project “VipRoaM” (The future of virtual product creation), Siemens Business Services examined the main research fields of virtual product development for the next ten years in cooperation with various partners in the automotive industry. Simulations reduce the number of expensive prototyping and product tests. During these simulations the customer virtually “sits“in a new car and operates - also virtually - the instruments on the dashboard, the gear change, etc. As a result, the degree of user-friendliness can be measured very early in the development process.

3.2.4 Augmented Reality for Production Facilities Construction Another industrial application field for Augmented Reality is found in the simulation of building production facilities. In most cases, the construction of new production facilities involves tremendous costs and time efforts. Incorrect planning, e.g. an insufficient consideration of structural circumstances or static problems, can cause high additional efforts and should therefore be avoided. Especially when building highly complex facilities (e. g. in an oil refinery or chemical industry), close-to-reality simulations are indispensable. With the help of Augmented Reality the planned machines and systems can be virtually placed in the area where they are supposed to be build-up. The planned machine is virtually shown in the real picture of the production hall. Structural conditions of the hall like windows, lights, emergency exists, etc., can be considered much better compared to conventional planning methods (2D-ground plan or 3D-model).

3.3 Edutainment

3.3.1 Augmented Reality as Museum’s Guide Information related to artwork and objects in museums is currently presented on tape via headset or on information plates next to the artwork. Within the research project “AMIRE”, a brand new way of how to provide information was realized at the Guggenheim Museum: the visitor gets information about the objects via a Tablet PC or PDA. The visitor goes through the museum carrying this device. Marking points at different places in the museum can be recognized by the Tablet PC. On its display, the visitor sees the real picture of the artwork enhanced through virtual objects to provide further information about the exhibit. For example, Painting techniques can be explained directly at the real picture on the screen. In the AR museum’s guide, developed for the Guggenheim Museum Bilbao, the artwork „The Snake“ by Paul Serra is virtually presented in different materials (like gold or wood) in order to demonstrate the relationship between material and artwork, while the original snake is made of black steel.

C-LAB Report


Furthermore, a video is faded into the real image showing how the object was installed in the museum’s fish gallery. Additional applications offer information about the architecture of the building itself as well as its static and floor plan by pointing to special features in the real picture as shown in the following illustration:

The real picture is extended by a three-dimensional picture of the building designed by Frank Gehry and thus enables the visitor to get a complete overview over the extravagant architecture. Further Augmented Reality elements are the map of the museum or direct navigation aids at certain spots, which guide the visitor through the exhibition. The following picture is a screenshot of the virtual museum’s guide on the Tablet PC:

3.3.2 Augmented Reality in the Tourism Industry By the usage of Augmented Reality, tourists are able to experience a multitude of innovative applications. Augmented Reality city guides navigate the tourist through foreign cities. At interesting sights this city guide shows for example a building in former times while presenting additional information at the picture’s screen. Another example is that buildings that were located there in former times, fade into the picture of important places.

C-LAB Report


There are many possibilities. Using a kiosk system, for example, enables a comparatively low-cost presentation of Augmented Reality applications at a permanent location. Several research projects already dealt with this topic. Within the project “ARCHEOGUIDE“, the ancient Olympia was reconstructed using Augmented Reality. The visitor was able to virtually complement the ruins at the original site to get a very realistic impression of the original place in ancient times. The project “CHARISMATIC“aimed at filling cultural and historically interesting places with a form of “virtual life”. Virtual persons were supposed to replay historical scenes like battles, coronations, etc., to provide additional explanations and to converse and discuss with each other. With this application it can be also realized, that a former virtual inhabitant of Pompeii guides visitors through the ancient city.

3.3.3 Augmented Reality in Game Applications Completely new kinds of computer games are also possible to be developed by the usage of Augmented Reality technology. In the mobile game “Kick Real“, realized by Siemens Business Services C-LAB, the user can participate in a virtual penalty shootout with his own foot. The camera of the mobile phone takes a photo of the user’s foot, which then appears in the virtual field on the display. Now, the player just needs to score a goal by slowly kicking the virtual ball with his foot. Direction and speed are determined by the foot’s real movement.

A similar game, preinstalled on the Siemens SX1, is „Mozzies“. Here, the player has to swat virtual mosquitoes that fly in a picture of the real surroundings. The enhanced physical integration of the player through the usage of his own foot, for example, has considerably increased the entertainment value of these games. The highly competitive game industry is characterized by high financial risks due to the necessity of high investments for the realization of new games (similar to film industry). As a consequence, Augmented Reality can be regarded as a source of realizing competitive advantages.

C-LAB Report


4 Summary So Augmented Reality allows for a multitude of new applications that draw their benefit from the more precise integration of reality and virtuality. On the one hand, the fun-factor of entertainment-oriented applications is boosted through the integration of the user (which moreover elevates the effectiveness of industrial trainings). On the other hand, information can be shown exactly where it is needed both in industry and for customer presentations. As a consequence, cost and time savings can be realized and services can be improved. Moreover, Augmented Reality can help to convey a positive and innovative perception of the product, as well as of the whole enterprise.

Cooperation Siemens Business Services GmbH & Co. OHG

Universität Paderborn Prof. Dr. Franz-Josef Rammig

Siemens Business Services GmbH & Co. OHG Dr. Wolfgang Kern

Cooperative Computing & Communication Laboratory Telephone +49-5251-60-6060 Fax +49-5251-60-6066 E-Mail [email protected] URL http://www.c-lab.de

C-LAB Marketing Fürstenallee 11 D-33102 Paderborn

Survey concerning your satisfaction with the report “Augmented Reality: Information at Focus” We kindly ask you to dedicate a short moment of your time to give us your personal assessment of this report. This will help us to improve our understanding of your needs. We would like to increase the alignment of our reports with your interests to offer you a higher surplus value. Thank you for your co-operation. How do you evaluate the subject of this report? applies completely applies not at all Topical Interesting How to you evaluate the content of this report? applies completely applies not at all Topical Interesting Articulate Practical relevance Informative Innovative Further comments: ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Voluntary information: Surname, first name: ___________________________________________________________________ Telephone: ___________________________________________________________________ E-Mail: ___________________________________________________________________ Please send the filled in form via mail, fax or e-mail to the respective address mentioned above.

Universität Paderborn Telephone +49-5251-606062 Telephone +49-5251-606061

Augmented Reality: Information at · PDF fileAugmented Reality: Information at Focus Christine Ludwig Siemens Business Services, C-LAB Christian Reimann Universität Paderborn, C-LAB

Documents