What is SCOPE of the Software?

1AR displays

Although an AR system may incorporate other senses than the sense of sight, vision is generally considered to be the main sense addressed by almost all AR systems. Thus a display which allows presenting some sort of computer generated content to a user is usually mandatory and fundamental for an AR system. Generally it is possible to categorize most systems into visible-see-through (VST) display or optical-see-through (OST) display based systems. Both principles have their specific advantages but also disadvantages.

OST displays allow a direct view of the real world. The superimposition is achieved via a translucent display. The natural view on the world remains mostly intact, which is usually preferable for critical applications like car navigation or surgery aids. With an OST display it is not trivial to cover bright real objects with darker virtual content [3]. OST based displays are more complicated due to the need for calibration, as every user has a different anatomy. The missing possibility to directly check the outcome and quality of the augmentation process, as the system has no direct feedback. It is rather the user that can evaluate the superimposition process and thus is usually required to perform a calibration process in the beginning. As image processing and tracking cannot perform in real-time, delay issues are common for OST displays. Fast head rotations or fast object movements lead to the problem that the virtual superimposed objects are not correctly aligned and are shown not at the correct position but where the real object was shortly before. The illusion of a coexisting real and virtual world quickly falls apart with an incorrect image registration.

VST displays use a video imaging system that captures the real environment, whose images are processed and superimposed before the user perceives them on some kind of a display system. The user is incapable to perceive the real world directly while using a VST display, which could raise possible security issues and hazards with certain applications. VST are better suited for Mediated Reality applications as it is easier to remove unwanted objects from a video image then try to occlude them with an OST display. Delays are also present but don’t represent such a big problem as the video stream and rendered overlay can be synchronized easily. With VST systems the user perceives the reality by the use of a video system, thus the resolution of the camera and the display determines the possible visual impression. It is reasonable to assume that no technology will recreate a natural visual impression in the near future and thus VST systems will lack realism in this field. In comparison to OST systems, where the system has no direct feedback, VST systems can use computer vision methods, independently of the used tracking technique, to analyze the video image directly and to verify the outcome of the superimposition, which gives them the possibility to automatically perform necessary corrections to the image registration process.

AR displays can also be further categorized by the following characteristics:

• Mobile or stationary

• Head-mounted, handheld or permanently mounted

These characteristics generally derive from or rather relate to the type of the application. Mobile devices like smartphones or tablet computers with integrated cameras can be easily used as mobile VST AR systems as they include all necessary components to run AR applications. Those almost ubiquitous devices are fast enough to allow tracking, content rendering and image registration in real-time.

Head-mounted OST or VST devices are becoming smaller and smaller and are thus better suited for everyday use. On the other hand head-mounted devices are still uncommon and attract attention. So a breakthrough cannot be expected until the devices are as unobtrusive as regular glasses. Stationary AR displays are more common in industry projects or tech demos like e.g. Microsoft’s Holodesk, where a user sees through a stationary half transparent OST and can interact with real and virtual objects [4].

2.2Content rendering and image registration

AR applications are usually meant to visualize some kind of data or information that is related to the real scenery. E.g. a car navigation system showing the planned route and highlighting important navigation details. Additionally the car computer system could emphasize important information (augmented vision), like the current speed, outdoor temperature and additional road users. On the other hand medical AR applications are meant to visualize completely different information. Thus it is clear that depending on the aim of the application, not only the setup and implementation but also the augmented content varies a lot.

For our virtual lens experiment application we needed to simulate simple lenses, light sources and a bunch of light rays passing through. Although our simulation was not very sophisticated it was sufficient to allow the students to understand the underlying physical principles. The same applies to the processed and visualized data for any AR system.

This so called image registration process depends heavily on the quality and precision of the tracking. For a realistic looking outcome it might also become necessary to not only consider the positional and orientational tracking. Light conditions, shadows and occlusion of real and virtual elements does matter as well to get a nice and realistic augmentation. Different focus planes of real and virtual elements could interfere with the user’s experience. With video based AR systems the resolution, color temperature and motion blur could become an issue. With optical-see-through displays transparency, occlusions and the contrast of bright and dark elements become a matter. All current stereoscopic head-mounted glasses lack realistic field of depth. Generally the content is presented on one fix depth layer, causing the vergence-accommodation conflict and leading to discomfort [5]. Current research tries to solve this conflict of vergence and accomodation by developing and utilizing volumetric, multi-focal-plane or light-field display technologies. Although single techniques prove to be suitable there are still plenty of unsolved problems.