Inﬂuences of Instructions about Operations on the Evaluation of the

Driver Take-Over Task in Cockpits of Highly-Automated Vehicles

Patrick Schn

oll

Institute for Media Technology, University of Technology Ilmenau, Germany

Keywords:

Automated Driving, Automation, Driver Take-Over Task, Driving Simulator, Human-Machine Interaction,

Human-Technology Interaction, Interface Design, Instructions, Operations, Take-Over Request, Test Method,

Test Design.

Abstract:

This paper works towards the development of a technologically independent framework to help render human-

centered examinations of the driver take-over task present in highly-automated vehicles comparable. Based on

available literature, the state-of-the-art and best-practices for driver take-over task examinations are analyzed

and discussed. It turned out, that the scope of the studies’ documentation, their level of detail as well as their

wording differs signiﬁcantly among themselves with respect to the instructions which were given to the test

persons. Besides the stimulus materials made available to the test persons during the examinations, bound-

ary conditions for the solution space of the task execution are deﬁned by the instructions about operations

provided. Therefore, the focus of this paper lies on the structural analysis of such instructions, suitable for a

human-centered examination of the driver take-over task. By deﬁning new demands for their documentation

and enhancing comparability between future studies, this paper aims on holistically improving the robustness

and validity of ﬁndings about human-performance in the ﬁeld of automated vehicles.

1 INTRODUCTION

As long as automated driving systems, which are ca-

pable of redundantly handling an unlimited opera-

tional design domain (SAE International, 2018), are

not available in series, partial automation with re-

gard to the driving task will determine the market of

advanced driver assistance systems. Per deﬁnition,

these systems incorporate at least one transition be-

tween the driver and the automated system and vice-

versa per drive. Therefore, understanding the human-

technology interaction in automated vehicles is a ma-

jor focus of research in the automotive industry.

The driver take-over task represents the procedure

during which the responsibility of controlling the ve-

hicle is transferred between the automated system and

the driver. Various studies exist, which determine

the driver take-over task or the driver’s response to

a driver take-over request (TOR) issued by the auto-

mated system, respectively.

To evaluate human-technology interaction in

cockpits of highly-automated vehicles, driving sim-

ulators are a widely used tool, both in research and

the automotive industry.

In order to obtain more generic, reliable and ro-

bust statements about the human-technology interac-

tion after a take-over request, results obtained from

various driving simulator studies can be compared.

Although the available studies share the same over-

all subject, they differ with respect to their test setups,

methods and designs, leaving the question to what ex-

tent the results of these studies can be rendered com-

parable.

Therefore, the present paper aims at working to-

wards a comprehensive and comparable framework

to support the execution of driver simulator studies

in the ﬁeld of partially automated driving systems by

providing an overview about test methods and designs

for the evaluation of human-technology interaction in

cockpits of highly-automated vehicles.

2 RELATED WORK

This paper builds upon the description of a generic

test setup suitable for the examination of the driver

take-over task following on from (Hackman, 1969;

Bubb et al., 2015; Winner et al., 2015; Schn

oll, 2020).

Its analytical foundation and development have been

described in detail in (Schn

oll, 2020). In the follow-

ing, the building blocks of the framework are pre-

Schnöll, P.

Inﬂuences of Instructions about Operations on the Evaluation of the Driver Take-Over Task in Cockpits of Highly-Automated Vehicles.

DOI: 10.5220/0010260600720080

In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 2: HUCAPP, pages

72-80

ISBN: 978-989-758-488-6

Focus on

Primary /

Secondary

Task(s)

Manual

Driving

Mode

Focus on

Tertiary

Task(s)

Automated

Driving

Mode

Stimulus Materials

Instructions

About

Goals

Instructions

About

Operations

TECHNICAL IMPLEMENTATION

PROCESS

DRIVER TAKE-OVER TASK

Final

Outcome

TEST PERSON

TEST ENVIRONMENT

TEST METHOD / -DESIGN

Figure 1: Illustration of a generic test setup suitable for the

examination of the driver take-over task following on from

(Hackman, 1969; Bubb et al., 2015; Winner et al., 2015;

Schn

oll, 2020).

sented.

Figure 1 shows a generic test setup suitable for

the examination of the driver take-over task (Schn

oll,

2020), following on from (Hackman, 1969; Bubb

et al., 2015; Winner et al., 2015). A subset of terms

used in Figure 1 originate from and are compatible

with the ’framework for analyzing the effect of tasks’

introduced by (Hackman, 1969). The aspect of de-

composing the driving task into primary, secondary

and tertiary tasks, known as the ’hierarchy of the driv-

ing task’, is based on (Winner et al., 2015; Bubb et al.,

2015).

2.1 Test Setup

As illustrated in Figure 1, a generic test setup consists

of a test environment, a test method and design as well

as a ﬁnal outcome.

Test Environment. The test environment itself is

deﬁned by a technical implementation, which pro-

vides stimulus materials to the test person executing

a process. Some stimulus materials provided to the

test person are subject to an active interaction, e.g. a

brake request, while others are at most inﬂuenced im-

plicitly, like e.g. information regarding surrounding

trafﬁc participants or other road users.

Table 1: Categories for test environments being used to ex-

amine driver behavior in a partially automated driving sys-

tem (Schn

oll, 2020).

Test Environments

Exemplary arranged in descending

order based on available stimulus materials

Categories

Vehicle on public roads / test track

Dynamic driving simulator with screen

projection (VR CAVE)

Dynamic driving simulator with VR headset

Static vehicle with screen simulation

Static simulation with screen projection (VR

CAVE)

Static simulation with multiple displays

7 Static simulation with VR headset

Static screen simulation with a minimal input

device set

In the present case, the process corresponds to the

test person performing the driver take-over task, ac-

quiring control over the vehicle from the automated

system, which results in a transition from the auto-

mated to the manual driving mode. Since in full au-

tomation mode, the complete responsibility is consid-

ered to be on system side, test persons can focus on

tertiary tasks, like i.e. the interaction with the vehi-

cle’s entertainment system, which are not connected

to the actual driving task itself.

Process. The process is inﬂuenced by personal fac-

tors (Hackman, 1969), which can be categorized into

stable and variable characteristics (Bundesanstalt f

Straßenwesen, 2012). Variable characteristics are

subject to frequent change, e.g. emotions, whereas

stable characteristics remain unchanged over a longer

period of time, as e.g. the driver’s age (Bundesanstalt

ur Straßenwesen, 2012).

Categorization of Test Environments. Table 1

lists categories for test environments being used to ex-

amine driver behavior in a partially automated driving

system. The categories are exemplary arranged in de-

scending order based on available stimulus material.

In case of a vehicle on public roads or a test track

1 , the whole stimulus material is entirely available

to the test person, whereas with a static screen simu-

lation with a minimal input device set 8 , it is tailored

to a minimum.

Inﬂuences of Instructions about Operations on the Evaluation of the Driver Take-Over Task in Cockpits of Highly-Automated Vehicles

By conducting studies in test environments such

as 2 - 8 , stimulus materials are implicitly and ex-

plicitly tailored by the selection of the test environ-

ment and the provision of certain stimulus materials

during the examination, as deﬁned by the designer of

the study. All stimulus materials, which are tailored

during the design phase of the study, have a more or

less signiﬁcant inﬂuence on the test person’s process

execution.

Therefore, test environments have to be chosen

purposefully in order to avoid unwanted side- and

cross-effects as well as inﬂuences on the overall re-

sult of the study attributed to the test setup. Due to

economic and resource-related boundary conditions,

only a minority of studies can be performed in ve-

hicles (cf. 1 ). When making use of test environ-

ments subject to tailoring of stimulus materials, po-

tential consequences of the tailoring should be criti-

cally examined and discussed along with the results

of the study.

Final Outcome. After having iterated through the

process and having created one or more trial out-

comes, as described by (Hackman, 1969), the ﬁnal

outcome represents the ﬁnal result of the process exe-

cution of the test person.

Test Method / -Design. As illustrated in Figure 1,

besides the test environment, a test setup incorporates

the test method and -design, which is closely related

to the research question of the respective study. The

test method and -design consist of instructions about

operations as well as instructions about goals (Hack-

man, 1969), which are communicated to the test per-

son before the start of the respective test run. This

way, boundary conditions and meta-information are

given to the test person deﬁning the solution space for

the execution of the process.

The instructions about operations contain details

on how the process shall be executed by the test per-

son, whereas the instructions about goals carry infor-

mation about what the test person’s action shall result

in to fulﬁll the work task under examination. Com-

paring the instructions about goals to the ﬁnal out-

come represents the performance of each individual

test person.

3 RESEARCH QUESTION,

DESIGN AND METHOD

Having analytically deﬁned the generic structure (cf.

Figure 1) and categorized the technical implementa-

tion of test setups used in literature, instructions given

to the test persons in studies available in literature

have to be investigated to be able to make a statement

about their comparability.

3.1 Research Question

Independent of the different technical implementa-

tions of test environments used in literature, another

dimension towards comparability of the respective

studies is to examine the instructions about operations

which are given to the test persons about to perform

a driver take-over task. Hence, the research question

in scope is: How can instructions about operations

given to test persons in studies examining the driver

take-over task be compared?

After analyzing and comparing the different in-

struction sets given in literature, a generic set of task

groups for the deﬁnition of instructions about oper-

ations shall be derived in order to give a foundation

for a generic framework supporting comparability of

future studies dealing with human-technology inter-

action in cockpits of highly-automated vehicles.

3.2 Research Design & Method

To answer the research question, various publications

investigating human performance in and interaction

with automated driving systems using driving simu-

lators have been reviewed and compared to obtain an

understanding about their commonalities and differ-

ences as well as best-practices in test methods and

-designs.

Representative statements have been collected and

compiled into an overview which contains examples

about instructions about operations with regard to

the driver take-over task from literature. Based on

the ﬁndings from this analytical research, technolog-

ically independent instructions about operations for

the driver take-over task have been structurally de-

rived based on the methodological approach of the Hi-

erarchical Task Analysis (HTA), as described by (Di-

aper and Stanton, 2003).

4 RESEARCH RESULTS

To obtain an overview about practices in research,

available literature has been analyzed and compared

regarding the instructions about operations which are

communicated to the test persons in the respective

studies. The analysis focused on instructions that re-

late to the overall interaction with the test environ-

ment as well as the driver take-over task itself. Out-

HUCAPP 2021 - 5th International Conference on Human Computer Interaction Theory and Applications

side the scope of this analysis were instructions with

relation to non-driving-related tasks (NDRT).

In the ﬁrst part of this section, ﬁndings from the

literature analysis are presented. These are followed

by a section introducing a methodical approach for

the deﬁnition of instructions about operations which

can act as a baseline for the documentation of future

studies related but not limited to the driver take-over

task. Furthermore, the results obtained from the pre-

sented work are connected and contextualized to the

illustration of a generic test setup suitable for the ex-

amination of the driver take-over task (cf. Figure 1),

which has been introduced before.

The results from the literature analysis are struc-

tured into three sections: engagement and disengage-

ment criteria for automation, the inﬂuence of instruc-

tions given to test persons during the studies and their

potential effects on the considered between-subjects

factors as well as determinant statements which are

related to personal interpretation. The sections men-

tioned above represent comparison criteria for in-

structions about operation given to test persons in

studies examining the driver take-over task.

4.1 Engagement & Disengagement

Criteria

Table 2 gives an overview of engagement and disen-

gagement criteria for automation in driver take-over

task examinations. Whereas it is a best-practice in lit-

erature to engage and disengage automation by press-

ing a button on the steering wheel (cf. Table 2), it is

rather unusual that the simulator itself gives the com-

mand for engaging the automation as with (Mok et al.,

2015).

Gas & Brake Intervention. Typically, the disen-

gagement of the automation system because of a

driver intervention is caused through a steering or

braking action performed by the driver (cf. Table 2).

It is noticeable, that a gas pedal interaction is widely

not taken into account, although it is a driver interac-

tion which could avoid collisions in some scenarios,

i.e. when a vehicle is approaching from the rear with

a high differential velocity.

In contrast to that, disengaging automation based

on the brake pedal actuation is very common in lit-

erature and could be observed in a variety of studies

(Jamson et al., 2013; Louw et al., 2015; Petermeijer

et al., 2017; Capallera et al., 2019). However, not

all studies contained information on the exact pedal

travel threshold which has been utilized. Typical val-

ues for the pedal travel threshold found in literature

range between 10% (Gold et al., 2013; Zeeb et al.,

2016) and 20% (Petermeijer et al., 2017).

Steering Intervention. A steering intervention usu-

ally is triggered when turning the steering wheel by a

certain angle. According values found in literature

range between 2..30 (cf. Table 2).

Other measured quantities triggering a steering in-

tervention are less frequently taken into account in lit-

erature. In the studies analyzed, (Zeeb et al., 2016)

considered the absolute steering wheel angle veloc-

ity and acceleration as a criterion for a driver inter-

vention. Moreover, (Petermeijer et al., 2017) set a

threshold for disengaging automation depending on

the ego vehicle’s deviation from the center of the cur-

rent lane, thus measuring the steering intervention in-

directly while observing the steering behavior over a

certain time period.

4.2 Instructions About Operations

Between-Subjects Factors. The analysis of avail-

able studies examining the driver take-over task has

shown, that the level of detail the overall communi-

cation between the test person and the researcher is

documented with varies signiﬁcantly across the stud-

ies.

Some authors explicitly mention written instruc-

tions used for brieﬁng (Gold et al., 2016; Neuhuber

et al., 2020) whereas typically, the communication

with the test person has been described on a more

generic level. In such cases, a deeper evaluation of

the inﬂuences, the communication between the re-

searcher and the test person has on their task exe-

cution turns out to be difﬁcult. In addition to that,

based on the information gained from the literature

only, it is hard to estimate if the communication with

the test persons has been informal during the studies

or if a potential formalization was not documented in

the publication itself.

However, if a publication lacks a detailed descrip-

tion of the way instructions have been provided to the

test persons, consequential effects on the results of the

respective study cannot be determined. This applies

in particular when utilizing between-subjects factors,

which are commonly used for validation to statisti-

cally increase the robustness of ﬁndings and achieve

a more general validity of the research result.

Personal Interpretation. If the boundary condi-

tions for executing the driver take-over task are not

speciﬁed precisely, the task execution process can be

subject to an unintended and unexpected personal in-

terpretation. However, this may also be implemented

Inﬂuences of Instructions about Operations on the Evaluation of the Driver Take-Over Task in Cockpits of Highly-Automated Vehicles

Table 2: Engagement and disengagement criteria for automation in driver take-over task examinations.

Engagement & Disengagement Criteria

Pressing a button on the steering wheel engages and disengages automation

(Jamson et al., 2013; Louw et al.,

2015; Mok et al., 2015; Zeeb et al.,

2016; Petermeijer et al., 2017)

Automation shall be enabled after a command from the simulator is delivered (Mok et al., 2015)

Steering Intervention, if:

Steering wheel is turned more than 2

(Gold et al., 2013; Louw et al., 2015)

Steering wheel is moved by more than 3

from its current position (Jamson et al., 2013)

Absolute steering wheel angle velocity is larger than 0.075

{s (Zeeb et al., 2016)

Absolute steering wheel angle acceleration is larger than 5

(Zeeb et al., 2016)

Steering, so that the deviation from lane centre is greater than approx. 0.5m (Petermeijer et al., 2017)

Brake Intervention, if:

Brake pedal is pressed

(Jamson et al., 2013; Louw et al.,

2015; Petermeijer et al., 2017;

Capallera et al., 2019)

Brake pedal travel is greater than 10% (Gold et al., 2013; Zeeb et al., 2016)

Brake pedal travel is greater than 25% (Petermeijer et al., 2017)

consciously in studies, i.e. in case such aspects are

coupled to the research question in scope of the ex-

amination. Nonwithstanding this, it is notable, that

some publications dealing with the driver take-over

task leave room for the personal interpretation of the

work tasks by the test person. When describing the

interaction with the automation system, various terms

subject to personal interpretation can be found.

Exemplary, (Jamson et al., 2013; Eriksson et al.,

2017; Banks et al., 2018; Capallera et al., 2019)

report instructions to the test persons which refer

to their subjective perception of comfort. (Jamson

et al., 2013; Banks et al., 2018) mention, that the

test persons could use the automated driving system

on a voluntary basis, whereas (Jamson et al., 2013;

Eriksson et al., 2017; Capallera et al., 2019) relate

to the subjective feeling of safety or the test per-

son’s appropriate reaction to a driver take-over re-

quest or the execution of non-driving-related tasks re-

spectively.

Hierarchical Task Analysis (HTA). As mentioned

before, the instructions about operations provide

boundary conditions under which the driver take-over

task shall be performed by the test person.

In order to converge towards a generic and holistic

instruction set to achieve comparability over different

driver take-over task examinations, the inﬂuence of

instructions subject to personal interpretation must be

minimized. An initial but important step in this di-

rection is the compilation of a technologically inde-

pendent set of work tasks the driver take-over task is

composed of. Therefore, a structured analysis of the

respective tasks is required.

(Diaper and Stanton, 2003) give an overview

about established methods to model tasks and com-

pare their syntactic and semantic differences. Follow-

ing (Diaper and Stanton, 2003), the Hierarchical Task

Analysis (HTA) approach can be used to investigate

human performance ’within complex, goal-directed

control systems’ featuring human-technology inter-

action, which matches a highly-automated vehicle in

trafﬁc. Hence, to generically deﬁne instructions about

operations suitable for the driver take-over task and

the underlying human-technology interaction, a Hi-

erarchical Task Analysis (HTA) has been performed.

The ﬁndings of this analysis are discussed in the fol-

lowing.

Driver Take-Over Task. Table 3 illustrates the re-

sults of the Hierarchical Task Analysis (HTA) as the

generic speciﬁcation for the instructions about opera-

tions suitable for the driver take-over task. The nota-

tion used is based on the one deﬁned for the Hierar-

chical Task Analysis (HTA) by (Diaper and Stanton,

2003). This particularly applies to the execution con-

HUCAPP 2021 - 5th International Conference on Human Computer Interaction Theory and Applications

Table 3: Speciﬁcation for human-centered instructions

about operations for the driver take-over task based on a

Hierarchical Task Analysis (HTA) as described by (Diaper

and Stanton, 2003).

Driver Take-Over Task

Instructions About Operations

Description Condition(s)

Supervise the vehicle status

by monitoring the

human-machine interface

components in the cockpit

IF a take-over

request (TOR),

initiated by the

system, has

been detected:

ą 1

Examine and understand the

situation:

1.1

1.2

1.3

1.4

1.5

1.1

Current position with respect to the next waypoint

1.2

Static environment

1.3

Dynamic environment

1.4

Road and its course

1.5

Trafﬁc situation

Plan the intended driving

maneuver path:

2.1

2.2

2.3

2.4

2.1

Deﬁne drivable space

2.2

Predict dynamics, movements and behavioral reac-

tions of other trafﬁc participants

2.3

Evaluate potential driving strategies based on

2.1

and

2.2

2.4

Deﬁne driving strategy to be performed

Prepare the driver take-over

maneuver:

3.1

3.2

3.1

Bring hands into the position required for manually

controlling the vehicle

3.2

Bring feet into the position required for manually

controlling the vehicle

Perform the driver take-over

maneuver:

4.1

Interact with the vehicle by

applying appropriate control

signals (e.g. gas / brake /

steering) in order to execute

the driving strategy deﬁned

2.4

ditions listed in the second column of Table 3.

A plus-sign indicates, that the respective tasks as-

sociated with it have to be completed before the next

consecutive task group, which is represented by a

one-digit number, can be performed. A special char-

acteristic of this notiﬁcation is that the conjoined tasks

can be completed in an arbitrary order, which reﬂects

the highly-subjective way in which different test per-

sons may fulﬁll a given work task. Once a task group

is completed, the subsequent one follows until the

whole driver take-over task is completed.

The ﬁrst row of Table 3 is an exception to the

above since it describes the ongoing, permanent ac-

tion of the test person supervising the vehicle status

by monitoring the human-machine interface compo-

nents in the cockpit. The actual driver take-over task

is started IF a driver take-over request (TOR), initi-

ated by the automation system, has been detected by

the driver.

In this case, the driver initially has to examine and

understand the trafﬁc situation the vehicle is in 1 . In

order to gain awareness and an understanding about

the surrounding situation, the current position with

respect to the next waypoint, the static and dynamic

environment as well as the road and the trafﬁc situa-

tion have to be understood (cf.

1.1

1.5

). Among

others, the sequence these subordinate tasks are per-

formed in, can depend on stable or variable personal

characteristics, like the individual feeling of safety or

comfort (cf. (Bundesanstalt f

ur Straßenwesen, 2012;

Schn

oll, 2020)).

After having examined and understood the sur-

rounding situation, the intended driving maneuver

path can be planned 2 . To do so, the drivable space

around the ego vehicle has to be deﬁned

2.1

and dy-

namics, movements and behavioral reactions of other

trafﬁc participants have to be predicted

2.2

. After

having gained information about the static and dy-

namic environment as well as the dynamics of the sur-

rounding trafﬁc participants, potential driving strate-

gies can be evaluated

2.3

and deﬁned

2.4

based on

this.

With a driving strategy deﬁned, the driver take-

over maneuver can be prepared

by bringing hands

and feet into the position required for manually con-

trolling the vehicle

3.1

3.2

. Having prepared for

the driver take-over, the maneuver can be performed

4 by interacting with the vehicle

4.1

by applying ap-

propriate control signals in order to execute the driv-

ing strategy deﬁned in

2.4

Figure 2 illustrates the results obtained from per-

forming the Hierarchical Task Analysis (HTA) for the

driver take-over task (cf. Table 3) as an extension of

the generic test setup presented in Figure 1. As illus-

trated in Figure 2, the instructions about operations

for the driver take-over task can be generically de-

scribed by the 6 consecutive task groups supervise,

examine, plan, prepare, perform and interact.

Inﬂuences of Instructions about Operations on the Evaluation of the Driver Take-Over Task in Cockpits of Highly-Automated Vehicles

Instructions

About

Operations

Instructions

About

Goals

DRIVER TAKE-OVER TASK

Supervise

the vehicle

status

Examine

and understand

the situation

Plan

the intended

maneuver path

Prepare

the driver take-

over maneuver

Perform

the driver take-

over maneuver

4.1

Interact

with the

vehicle

SUBJECT TO PERSONAL INTERPRETATION

CONSECUTIVE STEPS

TEST METHOD / -DESIGN

Figure 2: Instructions about operations suitable for the examination of the driver take-over task (cf. Figure 1; Table 3)

following on from (Hackman, 1969; Bubb et al., 2015; Winner et al., 2015; Schn

oll, 2020).

As shown in Table 3, some lower-level tasks in-

side the respective task groups are non-sequenced, as

indicated by the plus-sign associated to them. The or-

der of the task completion is determined by the test

person’s personal factors, whereas the whole process

of performing the driver take-over task itself is subject

to the test person’s personal interpretation.

5 CRITICAL ANALYSIS

The presented approach aims to be suitable for the

description of various driving tasks. However, in gen-

eral, these are inﬂuenced by varying stimulus materi-

als, surrounding conditions and personal factors, lead-

ing to many permutations. Therefore, the presented

tables, ﬁgures, lists and examples do not claim to be

exhaustive.

The performed literature analysis should be re-

garded as an indication for state-of-the-art and best-

practices used in this research ﬁeld and makes no

claim to completeness. It is intended to help draw

conclusions as to which aspects of the documen-

tation of the research results should be given spe-

cial attention. The compilation of the instructions

about operations has been achieved through analyt-

ical research based on a Hierarchical Task Analysis

(HTA). Depending on the research question in scope

of the respective examination, other groupings or lev-

els of detail are possible when following the presented

method. Moreover, the human-machine interface de-

sign as well as the technical implementation of the

test environment’s cockpit are expected to inﬂuence

the overall cooperation between the test person and

the automation system, which should be considered

when planning test runs.

6 CONCLUSIONS

6.1 Summary

This paper works towards the development of a tech-

nologically independent framework to help render

human-centered examinations of the driver take-over

task present in highly-automated vehicles compara-

ble. It introduces and discusses a framework which

pursues the idea of generically describing instructions

about operations suitable for the driver take-over task.

This can be achieved by formalizing human-centered

instructions about operations necessary for the driver

take-over task on one hand, as well as their communi-

cation to the test persons on the other. The presented

framework supports a common baseline for upcom-

ing human-centered studies conducted in the ﬁeld

of highly-automated vehicles and follows on from a

novel description of a test setup for the driver take-

over task.

Based on available literature, the state-of-the-art

and best-practices for examinations dealing with the

driver take-over task have been analyzed and dis-

cussed. It turned out, that the scope of the analyzed

studies’ documentation, their level of detail as well as

their wording differs signiﬁcantly among themselves

with respect to the instructions which were given to

the test persons.

The analysis utilizing the presented framework

has shown, that the instructions about operations

for the driver take-over task consist of subsequent,

HUCAPP 2021 - 5th International Conference on Human Computer Interaction Theory and Applications

generic tasks groups, which can be further assessed

based on the research question in scope. The appli-

cability for the driver take-over task has been exem-

plary demonstrated by having subdivided the generic

task groups further. The presented method can be tai-

lored or extended to meet the speciﬁc requirements of

the respective research questions, thus making it suit-

able for a technologically independent usage in vari-

ous test setups.

6.2 Discussion

When following the presented methodology, re-

searchers should consciously and attentively docu-

ment instructions about operations provided to the test

persons to avoid unintended, implicit side effects on

the study itself. This is especially relevant for test

environments making use of abstracted vehicles, like

driving simulators of various types, since their tech-

nical implementation goes by with an explicit or im-

plicit tailoring of stimulus material provided to the

test person.

To maintain comparability regarding the between-

subjects factors within the own study as well as render

the study comparable with others, researchers should

minimize the potential for subjective interpretation of

their given instructions by the test persons. Therefore,

a structural analysis of the task under examination is

required, which can be performed by following the

framework presented in this paper.

In general, to achieve comparability within the

own study, the test execution should be formalized by

using a reproducible way of providing the instructions

about operations to each test person. In order to en-

able comparability with other studies, as well as gen-

erate transparency over potential unintended side ef-

fects, it is recommended to devote special attention to

the detailed documentation of the instructions about

operations.

In addition to that, the validity and robustness of

the obtained results can be increased by formalizing

the interaction with the test persons, e.g. by utilizing

written instructions and reduce communication to an

absolute necessity. The difﬁculty with this is to ﬁnd

a balance between the formalization of the commu-

nication without making the test persons feeling un-

comfortable or treated impolitely, which can have an

impact on the results of the examination, respectively.

It is recommended to investigate if a gas pedal in-

teraction causing the ego vehicle to accelerate could

be a valid way to solve the driver take-over task in

the respective scenario. Providing access to this ad-

ditional stimulus material can lead to a better under-

standing of driver take-over maneuvers in different

scenarios. Furthermore, inconvenience with the au-

tomation system, i.e. by subjectively driving too slow,

can be qualitatively measured and assessed as a bene-

ﬁcial side effect.

6.3 Outlook

In a subsequent analysis, the test method and -design

of studies investigating the driver take-over task has to

be examined further to gain a holistic understanding

of the research results obtained in this ﬁeld. There-

fore, the introduced methodology should be imple-

mented in existing and upcoming studies to verify its

suitability. Aspects worthwhile to investigate could

be meta-goals and objectives the test persons shall

achieve as well as the correlation between typical

research questions and technical implementations of

test setups the respective studies are conducted in.

Additionally, the applicability of the technologically

independent approach presented could be veriﬁed by

conducting a study based on the methodologies intro-

duced in this work.

Furthermore, deviating performances of test per-

sons within the presented task groups could be

mapped to established models of human error func-

tions. Understanding the origin of the individual be-

havior of each test person contributes to the success-

ful design of future human-machine interfaces for the

driver take-over task.

REFERENCES

Banks, V. A., Eriksson, A., O’Donoghue, J., and Stanton,

N. A. (2018). Is partially automated driving a bad

idea? Observations from an on-road study. Applied

Ergonomics, 68:138–145.

Bubb, H., Bengler, K., Gr

unen, R. E., and Vollrath, M.

(2015). Automobilergonomie. Springer Fachmedien

Wiesbaden, Wiesbaden.

Bundesanstalt f

ur Straßenwesen (2012). Einﬂussfaktoren

auf das Fahrverhalten und das Unfallrisiko junger

Fahrerinnen und Fahrer. Berichte der Bundesanstalt

ur Straßenwesen, Heft M 229(Mensch und Sicher-

heit).

Capallera, M., de Salis, E., Meteier, Q., Angelini, L., Car-

rino, S., Khaled, O. A., and Mugellini, E. (2019). Sec-

ondary task and situation awareness, a mobile applica-

tion for conditionally automated vehicles. In Proceed-

ings of the 11th International Conference on Auto-

motive User Interfaces and Interactive Vehicular Ap-

plications: Adjunct Proceedings, pages 86–92, New

York, NY, USA. ACM.

Diaper, D. and Stanton, N. (2003). The Handbook of Task

Analysis for Human-Computer Interaction. Lawrence

Erlbaum Associates, Mahwah, New Jersey, 1 edition.

Inﬂuences of Instructions about Operations on the Evaluation of the Driver Take-Over Task in Cockpits of Highly-Automated Vehicles

Eriksson, A., Banks, V. A., and Stanton, N. A. (2017). Tran-

sition to manual: Comparing simulator with on-road

control transitions. Accident Analysis & Prevention,

102:227–234.

Gold, C., Damb

ock, D., Lorenz, L., and Bengler, K. (2013).

“Take over!” How long does it take to get the driver

back into the loop? Proceedings of the Human Fac-

tors and Ergonomics Society 57th Annual Meeting,

57(1):1938–1942.

Gold, C., K

orber, M., Lechner, D., and Bengler, K. (2016).

Taking Over Control From Highly Automated Vehi-

cles in Complex Trafﬁc Situations. Human Factors:

The Journal of the Human Factors and Ergonomics

Society, 58(4):642–652.

Hackman, J. R. (1969). Toward understanding the role

of tasks in behavioral research. Acta Psychologica,

31:97–128.

Jamson, A. H., Merat, N., Carsten, O. M., and Lai, F. C.

(2013). Behavioural changes in drivers experienc-

ing highly-automated vehicle control in varying trafﬁc

conditions. Transportation Research Part C: Emerg-

ing Technologies, 30(May):116–125.

Louw, T., Kountouriotis, G., Carsten, O., and Merat, N.

(2015). Driver Inattention During Vehicle Automa-

tion: How Does Driver Engagement Affect Resump-

tion Of Control? 4th International Driver Distraction

and Inattention Conference, (November):1–13.

Mok, B., Johns, M., Lee, K. J., Miller, D., Sirkin, D., Ive,

P., and Ju, W. (2015). Emergency, Automation Off:

Unstructured Transition Timing for Distracted Drivers

of Automated Vehicles. In 2015 IEEE 18th Interna-

tional Conference on Intelligent Transportation Sys-

tems, pages 2458–2464. IEEE.

Neuhuber, N., Lechner, G., Kalayci, T. E., Stocker, A., and

Kubicek, B. (2020). Age-related differences in the in-

teraction with advanced driver assistance systems - a

ﬁeld study. In Kr

omker, H., editor, HCI in Mobility,

Transport, and Automotive Systems. Automated Driv-

ing and In-Vehicle Experience Design, pages 363–

378, Cham. Springer International Publishing.

Petermeijer, S., Bazilinskyy, P., Bengler, K., and de Winter,

J. (2017). Take-over again: Investigating multimodal

and directional TORs to get the driver back into the

loop. Applied Ergonomics, 62(July 2017):204–215.

SAE International (2018). Ground Vehicle Standard (SAE

J3016:JUN2018): Surface Vehicle Recommended

Practice J3016: (R) Taxonomy and Deﬁnitions for

Terms Related to Driving Automation Systems for

On-Road Motor Vehicles. SAE International.

Schn

oll, P. (2020). Development of a Test Environment for

the Evaluation of Human-Technology Interaction in

Cockpits of Highly-Automated Vehicles. In Proceed-

ings of the 4th International Conference on Computer-

Human Interaction Research and Applications, pages

64–73. INSTICC, SCITEPRESS - Science and Tech-

nology Publications.

Winner, H., Hakuli, S., Lotz, F., and Singer, C., editors

(2015). Handbuch Fahrerassistenzsysteme. Springer

Fachmedien Wiesbaden, Wiesbaden.

Zeeb, K., Buchner, A., and Schrauf, M. (2016). Is take-over

time all that matters? The impact of visual-cognitive

load on driver take-over quality after conditionally

automated driving. Accident Analysis & Prevention,

92(January 2017):230–239.

HUCAPP 2021 - 5th International Conference on Human Computer Interaction Theory and Applications