User Experience and Multimodal Usability for Navigation Systems

User experience as a concept of human-computer interaction is crucial for the evaluation of systems and applications. Every new generation of navigational systems provides new features and extended functionality, which has additional functions that can oftentimes confuse the primary information on the system’s functionality. The conducted experiment analyses and observes available versions of navigation systems such as Garmin Drive 50 and/or TomTom Go, which are offered with certain advantages on features. The paper presents the selected aspects regarding the implementation, design, environment, recruitment, tests, and evaluation of the navigation systems, concluding from the results that there is difference between audio and visual mode.


I. INTRODUCTION
New generation of navigational systems provide new features and extended functionality, which can sometimes be source of confusion on the primary information of the system's functionality. This often times lead to poor usability, especially when it comes to operations of the primary functions [1], [5]. In order to design a proper environment with effect on user satisfaction, the usability aspect should be taken into account. This is to ensure successful performance on the primary operations of the system, as well as on the upgraded versions, regardless of the users' experience and ability with it. User experience as a concept of human-computer interaction is a fundamental concept in the evaluation of systems and applications, particularly because it affects issues such as usability, cognitive load, affective experiences, mental demand, efficiency, etc.
The evaluation of usability is quite diverse. As an area, it is under uninterrupted and active development, meaning new approaches and procedures come as continuous expansion of the practice into several contexts [2]. Hence, the testing methodologies can be different based on aim, place, and moderating style. However, in general there are two evaluation methods on usability, namely qualitative and quantitative [2], [6]. The quantitative evaluation is required to generally get numerical values that represent the level of usability, whilst qualitative evaluation is required to identify and examine issues and problems. In our work, we focus mainly on the quantitative method as a primary evaluation, namely surveys designed for specific tasks.
The concept of cognitive load, in the other hand, referring to the amount of effort required while performing an action, is an aspect taken into consideration mainly because of potential interference with the processes in the use of a system [17]. Considering that an ideal application or system ensures an interface that keeps user cognitive load to a minimum, what we have considered as an important indicator [18].
Additionally, mental workload, considered as a demand placed on the user by the system, depends on many parameters, which lead to the fact that the result of a task performed is subjective to the users' experience and the ability [7]. For this reason, it was determined that during the recruitment process the experience, initial capacities, reaction time and fatigue will be taken into account. The conducted experiment analyses and observes available navigation systems such as Garmin Drive 50 and/or TomTom Go, which are available and with certain advantages on features, although not as widely used as the most traditional available Maps. The compactible devices used were Camper/RV 890 and/or TomTom GO 60S Automotive GPS Navigation Device. The main reasons for carrying out this experiment are to find out the satisfaction and efficiency of the applications used and how much of the aimed aspects are satisfactory. This experiment is carried out aiming to analyse and compare user experience on aspects as efficiency, usability, and cognitive load on system's multimodal mode.
The defined null hypothesis assumes that there is slightly no difference between visual or audio mode of feedback in the user experience. Hence, it was decided on three defined dependent variables, namely efficiency, usability, and cognitive load, where the efficiency regards how fast the user finished the task, taking into account its completion time. Furthermore, usability as the second variable is defined as how easy the feedback is learned by the user. Finally, the mental and/or physical load for the tasks being taken by the user and the difference between audio and visual feedback. The independent variable is the feedback type with the given conditions on the audio and visual feedback treatments, shown in table I. The environment of the design is certainly basic, as it contains only one independent variable with two possible conditions based on the mode of the feedback. The design would be affected  [5], [14], [15].
Aspects regarding the implementation, design, environment, recruitment, tests and evaluation are thoroughly discussed in the next sections. The paper is structured as follows: section 2 presents the recruitment process, selected profile and cofounding variables together with the methods, tasks and setup. Section 3 deals with the result, statistics and significance of each test performed. Finally, section 4 gives an overall conclusion and discussion on final findings.

II. METHODOLOGY OF THE STUDY
The recruitment design is selected considering different factors, concentrating especially on the depended variable -the user experience, which should consider that has a wide range of variations. In the view of certain complex tasks that require information retrieval from the system, problem-solving skill of a certain level is necessary for multimodal feedback. Certainly, some individual users might have limitations, disabilities, or accessibility issues, hence within group design would be best to eliminate such differences [3], [4], [16].
Participant's profile is selected deciding in various factors in order to minimize the confounding factors: • Users with similar experience in using such systems. • User of similar age group. • User with/without IT background. • Accessibility or level of disability (visual or audio issues/impairments). Training and overview of the proposed navigation system is given to the user and an orientation session conducted to make participants familiar with the tasks required and how to tackle it. Moderators are not allowed to interfere during any ongoing activity, only in case of error issues, to allow completion of the activity.
In this setting the dependent variable is User Experience for the following categories, i.e., Efficiency, Usability and Cognitive Load. Given it with only one independent variable which is the type of feedback for the chosen systems, taking into consideration two conditions/states: Audio and Visual feedback for the respective system.
Whereas the environment of the carried experiment consisted of several elements. The first session was conducted to assess the navigation system usability, first by being introduced to the use of the system. Secondly, specific tasks for a specific period of time and path. It was followed by a session with the test environment where participants were asked to explore the virtual landscape freely and navigate from one place to another. The participants were instructed to try all of the navigation functions provided.
A short questionnaire was handed out after the road/path completion and the participants answered the questions within d minutes. The moderator followed a set of predefined questions during the later session. Preparation required before the study starts includes the consent form, demographic question-naire, participants coding and preparing all other questionnaires.
The tasks described in table I shall be assigned to the user walking or riding a car. This was not given, and in order to avoid any confounding variables, it could be provided.
Conditions intended are audio and visual feedback. The order of the condition to which the participant was assigned is the key to avoiding other factors or confounding variables. To avoid variables or other factors on the evaluation of dependent variable, it is very important to use randomization in order to lessen the effects. Such variables can be fatigue, or the learning curve while trying the tasks of audio than visual mode in order.
The conditions are counterbalanced among 20 participants using a simple randomization technique. The certain sequencing considering only two options of sequencing: C1ßC2 or C2ßC1, having in mind the counterbalance of the order effect, for n = 20, gives times 10 for each sequence, hence it was assigned which participant gets what sequence in order, having randomly "1" or "0" assigned and counted equal, followed by code of sequences.
After conducting the test, every participant filled in two questionnaires, one for the SUS (System Usability Scale) and second for TLX (Task Load Index). Using the adopted version of the questionnaire for system usability scale to measure the user experience for both audio and visual feedback [6], [7] as presented in the tables II and III. Similarly, some questions were adopted from user satisfaction questionnaire GUESS [9].
The questions defined in table III are required as a post questionnaire for participants who completed the tasks that had been discussed to measure the usability. For the efficiency measurement [8], the evaluation was done taking into consideration two features: the task completion time and the speed of conducting it, taken into units; time in seconds and speed per minute respectively. After the measurement of efficiency for both modes of the system, using T-Test was possible to compare the two means of speed and task completion for audio and visual feedback.

III. SYSTEM USABILITY SCALE (SUS) TESTS OF NAVIGATION SYSTEMS
Usability studies are well suited for gathering qualitative or quantitative behavioural data and for answering designrelated questions. Considering that our setup is rather focused on collecting feedback on the features rather than attitudinal feedback [10], [11], [12], it is well fit for our purpose. Nevertheless, data collected through methods mentioned, was later on described and analysed with Paired Sample T-Test. System usability score was calculated based on the questioners filled by participants [13]. Table IV contains the results from preliminary questionnaire, calculated and elaborated accordingly to the System Usability Scale (SUS) test score. In a general view the Visual mode tends to draw higher values than the Audio one (Fig. 1). Similarly, as seen in the results from boxplot the mean SUS scores of visual values are higher than the audio values.

Scheduled routes
Schedule a route in advance, either by given destination, decided route according to the path and time given ≈ 45min

Specific destination
Type in an address from a location outside of the city center (e.g., Suburban area in proximity to the city) you want to travel and follow a direction on the navigation system using the feedback type assigned.

≈ 90min
Find a nearby medical center Find closest hospital/clinic specialized in surgery and follow feedback from a navigation system ≈ 45min Completion Wrap-up, archiving logs and documents, collection of results. ≈ 45min

Sentence
Score (1-5) 1. I think that I would like to use this system frequently.
Score (1-5) 3. I thought the system was easy to use.
Score (1-5) 4. I think that I would need the support of a technical person to be able to use this system. Score (1-5) 5. I found the various functions in this system were well integrated.
Score (1-5) 6. I thought there was too much inconsistency in this system. Score (1-5) 7. I would imagine that most people would learn to use this system very quickly.
Score (1-5) 8. I found the system very cumbersome to use.
Score (1-5) 9. I felt very confident using the system. Score (1-5) 10. I needed to learn a lot of things before I could get going with this system.

Subject
Question Scale elaboration Mental Demand (MD) How mentally demanding was the task? Very low to Very high Physical Demand (PD) How physically demanding was the task? Very low to Very high Temporal Demand (TD) How hurried or rushed was the pace of the task?
Very low to Very high Performance (P) How successful were you in accomplishing what you were asked to do? Failure to Perfect Effort (E) How hard did you have to work to accomplish your level of performance? Very low to Very high Frustration (F) How insecure, discouraged, irritated, stressed, and annoyed were you? Very low to Very high Considering that only statistics are not enough to shows the significance in the system usability, paired samples T-Test shall be applied. Furthermore, Standard Error of Mean (SEM) and Standard Deviation (SD) are higher for audio over the visual   (Table V)

IV. CONCLUSIONS
The main goal of this paper was to find out whether there is a difference in user experience and system usability, in case of audio and visual feedback of navigation systems. The paper further investigated how using audio or visual mode impacts the variables user experience, usability and cognitive load in given setup. The results of the experiment show that there is a considerable difference in user experience for the System Usability and task cognitive load for the two types of audio and visual feedback for navigation systems, as shown in the results of the tests engaged. As for system usability dependent variable Visual Feedback was preferred among all participants with 98.8% confidence, whereas the degree of freedom showed the result: t(19) = 2.778, p = 0.012. On the other hand, for the Task Cognitive Load as dependent variable, the results showed that only two factors have significant effect in its Task Load, namely Fatigue and Mental Demand where p resulted smaller than 0.05, what leads to confidence of 99% that Audio type feedback requires more mental demand and causes more Tiredness to the users. These data resulted on given scores: t(27) = −3.170, p = 0.004 for Mental Demand and t(27) = −3.663, p = 0.001 for Fatigue. Consequently, the results lead to the rejection of null hypothesis that there is no difference in user experience between audio and visual feedback.