HUMAN-COMPUTER-INTERFACES FOR AR AND VR APPLICATIONS

 HUMAN-COMPUTER-INTERFACES FOR AR AND VR APPLICATIONS

A user interface that allows for a human-computer-interaction is not strictly mandatory for AR systems. Nevertheless most AR applications offer some kind of interaction. Depending on the aim of the application the interface differs greatly. Medical surgeons, who try to avoid any unnecessary contact with non-sterile items but still need to interact with more and more computerized equipment, possibly prefer adequate controls like gestures and voice commands. Mobile handheld device users usually rely on the ubiquitous and well-known touchscreen, at least as long as they hold their device in the hands. Touchscreen based devices lacks precision and tactical feedback. Nevertheless the users got used to its drawbacks and learned to live with them.

As soon as the mobile device is mounted in a VR viewer [14] the touch interface gets useless and new forms of interaction become necessary. Google introduced a magnetic trigger and, with the newer cardboard v2 design, a simple mechanical button touching the upper corner of the touch display. The user interacts by gazing onto onscreen user interface (UI) elements that gets activated after some visualized delay (Figure 3). This principle is quite straight forward and easy to perform, but also very limited in its functionality as it only allows for triggering. The same applies for the optionally available magnetic trigger. The newer mechanical button on the other hand allows long press actions and more sophisticated interactions like e.g. drag-and-drop functionality. As soon as the cardboard VR-viewer gets equipped with head straps it becomes a very low-cost HMD.

It is uncommon for HMD based applications to rely on a keyboard and mouse interface, as they are not directly visible to the user. Thus HMD based applications are ideally suited for motion controller interaction. Motion sensing controllers allow an interaction by capturing the motions performed by a user and translating them into some information that can be further processed. Particularly computer-games utilize these devices to allow novel forms of interaction. A popular device is the Microsoft Kinect that is able to capture sceneries in three dimensions [16]. The three dimensional imaging allows for a skeletal tracking and gesture recognition and thus a hands-free computer interaction. Another device is the handheld Nintendo Wii Remote controller [17]. Built-in accelerometer and optical sensors allow gesture recognition and pointing for interaction. The HTC Vive VR HMD is sold together with two separately tracked handheld controllers. The positional and orientational tracking along with the buttons allow the user to interact in VR [18]. The designated Google cardboard successor Google Daydream specifies an Android HMD based VR viewer and a separate handheld controller [19]. The controller can track its rotation and orientation with high accuracy and probably allows a gesture and button based interaction, but no positional tracking like the HTC Vive controller. Another affordable consumer input device is the SAGAWERE controller, which is going to be described in detail in the following chapter.

In any case the human-computer-interface needs to be well planned and designed for any specific application. To be accepted, the interface must be easy to learn and to use. For this new standards and guidelines of UIs have to be established. As usual a generalization is difficult. The professional user might be willing to spend more time on learning and mastering a complicated and complex user-interface then the causal user that wants to run a quick test of the newest AR game..