Nourishing Problem Solving Skills by Performing HCI Tasks

Relationships between the Methods of Problem Solving (Retrieval, Discovery, or

Search) and the Kinds of Acquired Problem Solving Skills

Muneo Kitajima

Nagaoka University of Technology, 1603-1 Kamitomioka-machi, Nagaoka, Niigata, Japan

Keywords:

Adaptive Problem Solving, Problem Solving Skill, Well-Deﬁned Problem, Ill-Deﬁned Problem, MHP/RT,

Multi-Dimensional Memory Frame, Routine Expertise, Adaptive Expertise.

Abstract:

There are three methods for deriving a solution for a problem with which a person is facing, which are 1)

retrieval of an existing solution from his/her own memory or from available external resources including

human resources, digital resources, and so on, 2) clarifying the constraints to meet and discovering a solution

that should satisfy them by exploring the problem space, or 3) deriving a solution by applying inference

rules successively until the goal state is achieved. This paper describes the distinctive cognitive processes that

respective methods should use when deriving a solution. On the assumption that the ultimately needed problem

solving skill would be the one which makes a person solve any problem by himself or herself without reliance

on any external resources other than himself/herself, i.e., adaptive problem solving, this paper discusses the

implications of the respective methods of problem solving to acquiring the required problem solving skill.

1 INTRODUCTION

Problems are part of daily life. A problem can be de-

ﬁned as a situation in which people cannot immedi-

ately achieve their goals using routinely performed

procedures. In order for them to solve their prob-

lems, people have to understand what the problem

is, to ﬁnd adequate resources, to undertake adequate

actions, and to monitor the situation until the goal

is reached, getting around impasses and other unex-

pected obstacles and undertaking corrective actions

when necessary.

Following GPS (General Problem Solver)

by Newell and Simon (1972), problem solving is

deﬁned by a sequence of the following processes:

1. Recognize and represent the current state, or gen-

erate a representation of the current state. When

the current state is the state to start a problem solv-

ing activity, it is called “initial state”,

2. Imagine and represent the states to achieve, or

generate a representation including the top-level

goal and intermediate subgoal states,

3. Select an action to move the current state to the

next, which replaces the current state if the move

is done successfully, and

4. Repeat until the top-level goal state is achieved.

In order for a person to carry out a PS (Problem

Solving) activity by applying these PS processes, the

following conditions have to be satisﬁed:

1. Be aware of the existence of goals to achieve and

represent them unambiguously and precisely at an

appropriate grain size,

2. Represent the current state as the initial state un-

ambiguously and precisely at an appropriate grain

size,

3. Have a repertoire of operators for transforming

one state to another, and

4. Have a consistent set of criteria for selecting

an appropriate operator among competing ones,

which can be applied to the current state and cause

transitions to different states.

A PS task requires a problem solver to execute

appropriate PCM (Perceptual-Cognitive-Motor) pro-

cesses that are expected to be effective for accom-

plishing the PS task. Figure 1 illustrates how PS

activities happen at the interface between tasks and

PCM processes. They could become diverse depend-

ing on the nature of tasks (including ICT tasks), and

how people carry out PCM processes. External tasks

impose time constraints on people’s PS activities and

people’s PCM processes are carried out under the

132

Kitajima, M.

Nourishing Problem Solving Skills by Performing HCI Tasks - Relationships between the Methods of Problem Solving (Retrieval, Discovery, or Search) and the Kinds of Acquired Problem

Solving Skills.

DOI: 10.5220/0006637101320139

In Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2018) - Volume 2: HUCAPP, pages

132-139

ISBN: 978-989-758-288-2

Internal (Mental) World

External (Physical, Social) World

Task to be

accomplished

INTERFACE

Perceptual-Cognitive-Motor processes

A variety of PS Tasks

Environmentin

External Definition of PS

Internal Definition of PS

<< Use Memory for PS Activities

Store in Memory Outcome of Performance <<

Carried out in Two Minds: Consciously or Unconsciously

Adaptive Problem Solving Activities

A sequence of consciously selected actions carried

out with the intension of accomplishing a desired

state, i.e., task goal, starting from the current state

represented in a problem space

Taxonomy of PS Tasks

Carried

INTERFACE

A se

sta

ried

ate

Continuous cyclic loop of

perception and movement

Figure 1: Adaptive problem solving activities.

limitations (or possible ranges) of PCM capabilities.

Therefore, PS activities are necessarily adaptiveto the

tasks people have to deal with. Adaptation is carried

out within the range of PCM capabilities. As pointed

out by Greiff et al. (2017), APS (Adaptive Problem

Solving) skill is one of critical competencies people

must have in order to achieve well-being in the cur-

rent networked, diverse, and heterogeneous society.

The interactions between the tasks that people

have encountered and the PCM processes that peo-

ple have actually carried out to accomplish the tasks

have signiﬁcant inﬂuence on people’s development of

PS performances (Kitajima, 2016). Therefore, peo-

ple’s experiences do signiﬁcantly affect the actual ac-

tion sequences for solving problems that are observed

when some tasks are given to them.

In our external environments of the 21st century,

there are an abundant number of artiﬁcial objects.

Any objects people interact with should have effect

on the memory that is directly associated with the ob-

jects, and some portions of memory that are indirectly

related with the objects. As people live in diverse

cultures and societies, a same object should be used

differently in terms of use of memory at the time of

using an object, and formation of memory after hav-

ing used an object. PS activities should occur in the

process of object use. After successful PS activities

that deal with any single object, it is differently asso-

ciated with each person’s memory, and socio-cultural

ecology should be a primary factor.

Interactions with novel artiﬁcial objects for ac-

complishing some goal inevitably involve APS ac-

tivities because the number of artifacts that surround

a person becomes larger and they should inﬂuence

his/her primary task in unpredictable ways to force

him/her to manage the situation by carrying out prob-

lem solving adaptively. The increase of the number

of surrounding artifacts results in the increase of the

number of dimensions to deal with for a person to live

satisfactorily in the 21st century.

The purpose of this paper is to give a characteri-

zation of PS activities in general and APS in partic-

ular from the viewpoint of well-deﬁnedness of prob-

lem, to provide a detailed analysis of respective PS

activities by using a cognitive architecture, MHP/RT

(Model Human Processor with Realtime Constraints),

and discusses the nature of PS and APS skills, re-

quired and potentially acquired, as a function of the

degree of well-deﬁnedness of the problems, and to

suggest the kinds of HCI tasks that could be accom-

plished by performing the cycles shown in Figure 1

to nourish APS skills, one of critical skills required in

the 21st century.

Nourishing Problem Solving Skills by Performing HCI Tasks - Relationships between the Methods of Problem Solving (Retrieval,

Discovery, or Search) and the Kinds of Acquired Problem Solving Skills

133

2 PROBLEM SOLVING

ACTIVITIES

2.1 Problem-Solving in Well-Deﬁned

Problem Spaces

A problem space is deﬁned by an initial state, a goal

state, a set of intermediate states, and operators to

move from one state to another if a set of conditions

to move is satisﬁed. A problem space is well-deﬁned

if the initial state, the goal state, and the interme-

diate states are represented unambiguously and pre-

cisely, and the conditions for move are deﬁned ap-

propriately. A person who solves a problem in a well-

deﬁned problemspace (or if he/she is able to represent

the problem as an instance of a well-deﬁned problem

space) would show a deterministic PS behavior for a

given goal, i.e., he/she solves a problem to achieve

the given goal starting from a speciﬁc initial state by

successively applying a ﬁxed sequence of operators,

which may be most efﬁcient in terms of, e.g., time,

energy consumption, etc.

2.1.1 GOMS Models

Card et al. (1983) suggests that the knowledge that a

person acquires for carrying out routine goal-oriented

tasks deﬁned by well-deﬁned problem spaces con-

sists of Goal, Operators, Methods, and Selection rules

(GOMS). Goal is represented as a rigid hierarchi-

cal goal structure, in which a goal is satisﬁed if and

only if all of its subordinate sub-goals are satisﬁed.

Method is deﬁned by a speciﬁc ﬁxed-ordered se-

quence of operators to accomplish a speciﬁc goal.

Hierarchically, operators reside at the bottom layer

of the goal structure, and methods are just above the

operator-layer, each of which is considered as a label

to identify a distinct sequence of operators, and asso-

ciated with a goal at that level of hierarchy, in such a

way as “goal G is accomplished by applying method

M.” Selection rule speciﬁes a consistent rule to select

a method among competing methods applicable for

the current state. An operator’s performance such as

execution time is required to be independent of states

in order for an operator to be qualiﬁed as operator of

GOMS. With these features, GOMS models are able

to predict performance times of skilled users in carry-

ing out routine goal-oriented tasks (Gray et al., 1993;

Haunold and Kuhn, 1994).

2.1.2 Production System

If a person has a well-deﬁned problem space for a

given goal, his/her operator sequence is predictable.

Models developed under ACT-R cognitive architec-

ture (Anderson, 2007; Anderson and Lebiere, 1998),

Production Systems, can simulate cognitive processes

involved in accomplishing goals by focussing on sin-

gle goals successively and searching for production-

rules deﬁned by a set of pairs of conditions and ac-

tions (equivalent to operators in this context). They

are stored in procedural memory and those that match

the conditions concerning the currently focussed goal

and the existence of required knowledge stored in

declarative memory are possible to ﬁre in the next cy-

cle. A person who becomes an expert level of a do-

main can be viewed as one who has constructed an

efﬁcient set of production rules, called “routine ex-

pertise” (Holyoak, 1991). GOMS model lies at one

extreme of a well-deﬁned problem space, where the

strong condition for operator qualiﬁcation is applied,

i.e., an operator has to be independent of context,

or an operator’s performance has to be indifferent to

which state it is applied, and due to this feature, it

can predict performance times of a speciﬁc kind of

problem solver. For those who can use a well-deﬁned

problem space when he/she needs to accomplish a

goal, it reduces just to cognitive activities to traverse

the problem space. It is very different cognitive activ-

ities than those that require adaptive skills. It may re-

quire, however, times for representing current states,

searching for the next operators that match the condi-

tions to movewith the current state, selecting the most

appropriate one among a set of competing operators,

and actually carrying out the selected operator.

2.2 Problem-Solving in Ill-Deﬁned

Problem Spaces

A well-deﬁned problem space can degrade to ill-

deﬁned problem spaces easily due to several reasons.

One may have the abilities to transform an initially

ill-deﬁned problem space to a less ill-deﬁned one.

Once an ill-deﬁned problem space is transformed

to a well-deﬁned one, one may traverse the well-

deﬁned problem space by successively ﬁring produc-

tion rules as modeled by the ACT-R cognitive archi-

tecture. Holyoak (1991) names this class of exper-

tise as “adaptive expertise”, who is good at generate

appropriate decoupling of the condition-action pairs

of the production rules ﬂexibly in the given situation

to produce appropriate actions. Procedural knowl-

edge represented as production rules are for travers-

ing in a well-deﬁned problem space. Given a top-

level goal, one is required to ﬁnd in his/her knowl-

edge a well-deﬁned problem space or somehow gen-

erate one. Once it is deﬁned, PS activities can be con-

sidered as the activities for traversing in the problem

HUCAPP 2018 - International Conference on Human Computer Interaction Theory and Applications

134

space. PS skills should involve the skills necessary

for transforming an ill-deﬁned one to well-deﬁned, or

less ill-deﬁned ones, which is APS skill.

2.3 Turning Well-Deﬁned Problem into

Ill-Deﬁned Ones

The following subsections describe how well-deﬁned

problem may turn into ill-deﬁned ones, and point out

necessary cognitive skills for changing ill-deﬁned one

to well-deﬁned one.

Imagining Goals is not Trivial: One may have dif-

ﬁculty in imagining a set of goals to achieve the top-

level goal. For example, a foreign traveler stands

in front of a ticketing machine with unfamiliar and

strange interface to him/her by which he/she believes

he/she is supposed to buy a train ticket. He/she is

unable to generate intermediate goals to achieve the

top-level goal, i.e., “he/she has a necessary ticket in

his/her hand.” This is the issue of mental models:

• the variety of mental models one has in the do-

main in question (mental model),

• the ability to switch from one mental model to an-

other if the current one is estimated as ineffective

(switching skill), and

• the ability to create effective mental models from

experience (model creation through experience).

Recognizing and Representing State is not Trivial:

In traversing a well-deﬁned problem space, it is ob-

vious where to attend to in the current state one has

reached, and how to represent it because every state

has already been represented unambiguously and pre-

cisely at the required level for selecting next opera-

tors. However, when goals are underspeciﬁed, one

may ﬁnd difﬁculty in controlling where to attend be-

cause a goal for the current state should provide the

semantic context that helps him/her to parse the cur-

rent state of the scene where he/she is in and compre-

hend it.

Comprehension requires background knowledge

which is different from person to person. In addition,

the ability to ﬁlter out the relevant objects from irrele-

vant ones may differ from person to person. Kitajima

and Toyota (2012) demonstrated that elderly partici-

pants who had inappropriate functioning of attention

had difﬁculty in acquiring relevant information from

rather complicated sign boards while performing a

way-ﬁnding task. This evidence shows that even if

one has a well-deﬁned problem space in his/her long-

term memory, it might be difﬁcult for him/her to acti-

vate it for use in the PS activity. No effective retrieval

cue is represented in working memory to activate task

relevant well-deﬁned problem space.

At this stage, an ill-deﬁned problem space might

change to a well-deﬁned or less ill-deﬁned one if one

has the following:

• background knowledge to make him/her possible

to represent objects in the current state appropri-

ately for the given goal (background knowledge

and comprehension skill),

• the ability to use the representation for deﬁning

to-be-searched-for objects (target objects) or eval-

uating the degree of relevance of attended objects

(searching skill), and

• the ability to pay attention to the relevant objects

in the current state (attention skill).

Selecting Next Action is not Trivial: In travers-

ing a well-deﬁned problem space, next actions are

selected by evaluating expected cost to be expended

moving from the current state to the ﬁnal goal state.

Means-ends analysis or the hill climbing strategy is

applied for this purpose. This is only possible when

the problem space is strictly deﬁned and one is able to

manipulate it as a whole. In reality, it is not possible

if the size of problem space is large. One cannot fore-

see the entire problem space. In addition, one cannot

fully specify the current situation. Under these condi-

tions, one’s action selection should not be rational but

is controlled by the bounded rationality principle and

the satisﬁcing principle uncovered by Simon (1956)

and further studied by Kahneman (2003) in generat-

ing situated next actions.

However, the farther one can foresee in the prob-

lem space, the lesser ill-deﬁned the current problem

space becomes. Kitajima and Toyota (2012) demon-

strated that elderly participants who had inappropri-

ate functioning of planning had difﬁculty in estimat-

ing relevanceof the acquired “right” information from

sign boards to the current goal while performing a

way-ﬁnding task because the current goal is under-

speciﬁed. This evidence shows that even if one is

able to attend to a right object, it might be difﬁcult

for him/her to estimate it as relevant for accomplish-

ing the current goal which is not detailed enough to be

matched with the representation of the right object.

An ill-deﬁned problem space might change to a

well- or less ill-deﬁned one if one has the following:

• the ability to foresee the future states by perform-

ing mental simulation (planning ability), and

• background knowledge to be used in the planning

activity (mental model).

Nourishing Problem Solving Skills by Performing HCI Tasks - Relationships between the Methods of Problem Solving (Retrieval,

Discovery, or Search) and the Kinds of Acquired Problem Solving Skills

135

!"#$%&'($)*+,-&'.

/(',"00&.1

!"#$%$&$"'()*'#+&

/"(,"2-3$)

4.5'(6$-&'.*/(',"00&.1

!"#$%$&$"'()*'#+&

+/0.,#('.'1/

!$.2

+/0.,#('.'1/*!$.2

!'7&)8*9''(7&.$-&'.

:'.&-'(&.1

!"#$%$&$"'()*'#+&

*!"#$%&"%'()*+,*-

*./&"0+(1$,"2",+(3/'",/#

4.5'(6$-&'.*5('6

;.%&('.6".-

30.,#('.'1/*!$.2

,-#.(/+01(2-&+(3$%'#40-%#'

:"6'(&<$-&'.*'5

!"#$%&'($)*+,-&'.0

$.7*-#"&(*="03)-0

25$(6-%7'

!"#$%$&$"'()*'#+&

9'.0,&'30

4.5'(6$-&'.*/(',"00&.1

!"#$%$&$"'()*'#+&

+3-'.'6'30*+3-'6$-&,

!"#$%&'(*9'.-(')*/(',"00&.1

12%"0%405(-5-/#+(6/#(&5$"7"/'(-%8"'9("'$0:&57;

<=(35-/#+(%$,"2%,5&(4+(#57/'%'$5(#5%$,"/'

>=()57"&:%00+(%$,"25(-5-/#+(

35-/#+(?#/$577"'9(@"-5

2869

:"6'(8*/(',"00&.1*>="0'.$.,"*="$,-&'.?

!"#$%$&$"'()*'#+&

@#'(-AB"(6*:"6'(8

C'(D&.1*:"6'(8

E'.1AB"(6*:"6'(8 :'-'(*:"6'(8

Memory

()

System 2

System 1

Figure 2: MHP/RT: Model Human Processor with Realtime Constraints.

3 A CLOSER LOOK AT

SELECTING NEXT ACTION

This section introduces a cognitive architecture,

MHP/RT (Model Human Processor with Realtime

Constraints) proposed by Kitajima and Toyota (2013,

2012), that is capable of simulating action selection

processes in any problem solving situations, which

have been described in the previous section phe-

nomenologically. MHP/RT consists of memory and

action selection processes, and describes in detail not

only how action selections are carried out and which

action will be performed but also how the results of

action selections are stored in memory (see Kitajima

(2016) for a full description of the architecture and its

applications).

3.1 Cyclic Processes of Action Selection

and Memorization

Action selection and memorization is a cyclic process

which works while one lives in the world. As one

interacts with the environment, memory is gradually

structured as multi-dimensional memory frames (see

the bottom part of Figure 1 for a schematic illustra-

tion). Constraints on behavioral processing are im-

posed by conscious and unconscious processes, Sys-

tem 2 and System 1 of Two Minds, respectively,

and behavior must be synchronized with the ever-

changing external and internal environments, which

is a form of self-organization. As one grows while

carrying out the cyclic processes of action selection

and memorization, he/she develops his/her memory

and shows distinct behavioral characteristics (Kita-

jima and Toyota, 2014, 2015).

3.2 MHP/RT: Model Human Processor

with Realtime Constraints

Figure 2 illustrates MHP/RT (Kitajima and Toyota,

2012, 2013), which consists of ﬁve autonomous sub-

systems. MHP/RT is an extension of a version of dual

processing theories, Two Minds, proposed by Kah-

neman (2003, 2011). Two Minds consists of uncon-

scious processes, System 1, and conscious processes,

System 2. System 1 is a fast feed-forward control pro-

cess driven by the cerebellum and oriented toward im-

mediate action. In contrast, System 2 is a very slow

HUCAPP 2018 - International Conference on Human Computer Interaction Theory and Applications

136

feedback control process driven by the cerebrum and

oriented toward future action.

MHP/RT focuses on synchronization between

System 1 and System 2 in the information ﬂow from

the perceptual system from the environment at the

left end to the motor system at the right end. Out-

put from the perceptual system is diverted into three

paths, one path leads to the conscious process of Sys-

tem 2, the other leads to the unconscious process of

System 1, and the last one leads to the memory sys-

tem. Information in memory activated by the input

from the environmentis become available to System 1

and 2. System 1 and 2 work in synchronous with each

other but the memory process works asynchronously

with System 1 and 2. The dotted oval shows the pro-

cess of memorization of output from the motor pro-

cess. These interactions between System 1 and 2, and

memory are not seriously considered in the original

Karhneman’s Two Minds. Processes associated with

unconscious System 1 are indicated by green lines.

And those associated with conscious System 2 are in-

dicated in orange lines.

3.3 MHP/RT’s Four-Processes: Use and

Modiﬁcation of Memory

MHP/RT works in one of four different modes when

one looks at it from a particular event that occurred

at the absolute time T. Two are before the event in

which MHP/RT uses memory and the other two are

after the event in which it modiﬁes memory:

• System 2 Before Mode: MHP/RT consciously

uses memory before the event for anticipating

the future event which takes relatively long time

(T − some amount of time).

• System 1 Before Mode: MHP/RT uncon-

sciously uses memory just before the event,

say 100 milli seconds before the event for au-

tomatic preparation for the future event (T −

a few hundreds mill seconds).

• System 1 After Mode: MHP/RT uncon-

sciously tunes the current network connections

related to the past event for better perfor-

mance for the same event in the future (T +

a few hundreds mill seconds).

• System 2 After Mode: MHP/RT consciously

reﬂects on the past event resulting in structural

changes in memory (T + some amount of time).

4 PROBLEM SOLVING

ACTIVITIES VIEWED FROM

MHP/RT

This section discusses how PS activities are described

from theviewpoint of MHP/RT’s action selection pro-

cesses and memory processes. There are three meth-

ods for deriving a solution for a problem with which a

person is facing; retrieval, inference, and exploration.

In the following subsections, each method is elabo-

rated by combining the description in Section 2 and

Section 3.

4.1 Solving a Problem by Retrieval

Solving a well-deﬁned problem: Given a problem

statement, irrespective of whether it is generated in-

ternally or provided externally, a person represents

both the goal state to achieve and the initial state

that he/she is in unambiguously and precisely enough

to retrieve the description of the action sequences

that should intervene the both ends of the states from

his/her memory. Therefore, this problem is consid-

ered as a well-deﬁned problem.

For solving the problem, he/she is required to just

carry out the retrieved sequence of actions. This prac-

tice should strengthen the retrieved memory trace that

connects the representation of the initial state and the

goal state. System 2 Before Mode is used when re-

trieving the sequences of actions, and System 1 Be-

fore Mode is used to actually solve the problem. Sys-

tem 1 or 2 After Mode strengthen the memory that has

used in the activity. In this way, he/she actually learns

from the PS practice to strengthen the memory traces

of successful performance.

Problem solved without problem solving activities:

Given a problem statement, a person uses external

memory to retrieve solutions. The description of the

problem statement is used literally as it is, and he/she

expects to reach any “solutions” that someone has al-

ready created for the problem. Since the solution is

external, he/she is required to follow consciously the

sequences of actions as the solution speciﬁes. In other

words, the problem is solved by borrowing the others’

thinking processes, and he/she just perceives the state

of the problem and moves eyeballs and hands as the

description of what to do in the perceived situation.

He/she could use System 1 After Mode for tuning

neural networks to just ﬁnished activities or deliberate

reﬂection in System 2 After Mode. Only when he/she

does them, he/she can learn from the practice, other-

wise he/she just memorizes the episode, “the problem

Nourishing Problem Solving Skills by Performing HCI Tasks - Relationships between the Methods of Problem Solving (Retrieval,

Discovery, or Search) and the Kinds of Acquired Problem Solving Skills

137

was given, and successfully solved by retrieving so-

lutions from the Web.” The link between the problem

and the solution may or may not be established. The

problem may not be solved by using his/her memory

when encountered it again in the future. Learning is

very limited compared with the former case.

4.2 Solving a Problem by Inference

Given a problem statement, irrespective of whether

it is generated internally or provided externally, one

represents both the goal state and the initial state that

he/she is in unambiguously and precisely enough to

retrieve the inference rules, i.e., procedural knowl-

edge, to carry out the necessary state transitions. This

is the way of problem solving modeled as production

systems like ACT-R (Anderson, 2007; Anderson and

Lebiere, 1998) or Soar (Newell, 1990; Laird et al.,

1987).

In this case, it is assumed that a person has pro-

cedural memory, which connects perceptual multi-

dimensional memory frame with behavior multi-

dimensional memory frame with the help of relation

multi-dimensional memory frame, which are associ-

ated with perceptual, System 1, and System 2 pro-

cesses, respectively. The inference rules are succes-

sively applied to transform the current state to the

next until the goal state is achieved. System 2 Be-

fore Mode is used for planning and System 1 Before

Mode is used to execute individual production rules.

System 1 After Mode is used to strengthen the suc-

cessfully applied production rules in behavior multi-

dimensional memory frame and declarative memory

in relation multi-dimensional memory frame. Sys-

tem 2 After Mode might be used to create a new

set of production rules that are expected to function

more efﬁciently for the kinds of situations deﬁned by

the just solved problem. Ultimately, the person will

have the ability to carry out the same task more and

more efﬁciently, very fast without error; it is said that

he/she has acquired routine expertise for the kinds of

tasks executable by applying well-learned procedural

knowledge (Holyoak, 1991).

The states that appear while solving the problem

are represented unambiguously and precisely enough

to retrieve production rules from procedural mem-

ory (behavior multi-dimensional memory frame), and

factual knowledge from declarative memory (relation

multi-dimensional memory frame), triggered by per-

ceptual multi-dimensional memory frame that is ac-

tivated by external perceptual stimuli. Therefore, the

performance of PS activity would depend on the con-

tents of memory and the functioning of perceptual

sensors. These should be inﬂuenced by the kinds of

experiences one has had from his/her birth. This is ob-

viously affected by the culture and the circumstances

one is in in everyday life while one grows up. This

suggests the external environment should be of cru-

cial importance for nourishing this kind of PS skill.

4.3 Solving a Problem by Exploration

Given a problem statement, a person can only repre-

sent both the goal state and the initial state vaguely,

and therefore it is not possible for him/her to retrieve

anything directly relevant to solve the problem. This

is the case where he/she is faced with an ill-deﬁned

problem as described in Section 2. He/she needs to

create effective retrieval cues to ﬁnd any action that

should move the current state to another along the un-

known successful path to the goal, whose representa-

tion should become less vague as one proceeds. This

is equivalent to make the initial vague representation

of the goal clearer; to convert the ill-deﬁned problem

to a less ill-deﬁned or hopefully a well-deﬁned one.

The notion of resonance is relevant for this to

happen. Memory system in MHP/RT is regarded

as an autonomous system, which means that mem-

ory should not be a passive database system to re-

turn data on request from the action selection process.

The MHP/RT’s memory system receives input from

the autonomous perceptual system and resonates with

the other autonomous systems, i.e., conscious system

(System 2 of Two Minds) and unconscious system

(System 1 of Two Minds) to make available the cur-

rently activated portion of memory. The mechanism

of resonance is used to make available any relevant

portion of behavior multi-dimensional memory frame

to the action selection process of MHP/RT. System 2

Before Mode and System 1 Before Mode use this res-

onance mechanism to select plan and next action.

The selected action may succeed or fail, or result

in uncertain outcome to fall into an impasse.The pro-

cesses are totally exploratory but the memory traces

of the executed actions will be created with the ﬂag

of success/failure. The memory trace with success-

ful performance will be strengthened because it is as-

sociated with rewards. Ultimately, the person will

have the ability to handle the same situation ﬂexi-

bly with less ineffective search; it is said that he/she

has acquired adaptive expertise (Holyoak, 1991) for

the kinds of ill-deﬁned tasks with a variety of well-

developed behavior and motion multi-dimensional

memory frames available through resonance with per-

ceptual multi-dimensional memory frame.

Development of ﬂexible and rich memory is nec-

essary for acquiring adaptive expertise through a va-

riety of experience with reality (not virtual). It is of-

HUCAPP 2018 - International Conference on Human Computer Interaction Theory and Applications

138

ten felt that a solutions for an ill-deﬁned problem is

discovered suddenly or the solution emerges sponta-

neously. It is because the critical process underlying

the discovery is memory resonance. However, the

richness of memory should affect the possibility of

successful discovery should happen; construction of

memory structure that integrates perception and body

movement with high reality is important, and again

a variety of experience with reality is important for

nourishing this problem solving skill.

5 CONCLUSIONS

This paper suggested that when faced with a problem,

either it is given externally or generated internally,

it could be carried out either by retrieval, search, or

discovery method, each of which is associated with

distinct cognitive processes for action selection and

memorization. Respective methods should have dif-

ferent implications to development of PS skills. Re-

trieval method would be effective when his/her own

memory is used; search method is effective for ad-

vancing efﬁcient use of inference rules; discovery

method is crucial for turning ill-deﬁned problem to

well-deﬁned one which should be critical for APS

skill development.

The importance of experience was emphasized for

nourishing PS skills. Some aspects of experience

needs to be carefully designed in education, society,

or wherever a person carries out some activity for the

purpose of nourishing PS skills by considering the

distinct features in terms of cognitive processes in-

volved in the different kinds of PS activities.

ACKNOWLEDGEMENTS

This work was supported by JSPS KAKENHI Grant

Number 15H02784.

REFERENCES

Anderson, J. R. (2007). How can the Human Mind Occur

in the Physical Universe? Oxford University Press,

New York, NY.

Anderson, J. R. and Lebiere, C. (1998). The Atomic Com-

ponents of Thought. Lawrence Erlbaum Associates,

Mahwah, NJ.

Card, S. K., Moran, T. P., and Newell, A. (1983). The Psy-

chology of Human-Computer Interaction. Lawrence

Erlbaum Associates, Hillsdale, NJ.

Gray, W. D., John, B. E., and Atwood, M. E. (1993). Project

Ernestine: Validating a GOMS Analysis for Predict-

ing and Explaining Real-World Task Performance.

Human-Computer Interaction, 8(3):237–309.

Greiff, S., Scheiter, K., Scherer, R., Borgonovi, F., Britt, A.,

Graesser, A., Kitajima, M., and Rouet, J.-F. (2017).

Adaptive Problem Solving, MOVING TOWARDS A

NEW ASSESSMENT DOMAIN IN THE SECOND

CYCLE OF PIAAC. OECD Education Working Pa-

pers, (156).

Haunold, P. and Kuhn, W. (1994). A Keystroke Level Anal-

ysis of a Graphics Application: Manual Map Digi-

tizing. In Proceedings of ACM CHI’94 Conference

on Human Factors in Computing Systems, pages 337–

343, New York, NY. ACM.

Holyoak, K. J. (1991). Toward a General Theory of Exper-

tise: Prospects and Limits, chapter Symbolic connec-

tionism: toward third-generation theories of expertise.

Cambridge University Press.

Kahneman, D. (2003). A perspective on judgment and

choice. American Psychologist, 58(9):697–720.

Kahneman, D. (2011). Thinking, Fast and Slow. Farrar,

Straus and Giroux, New York, NY.

Kitajima, M. (2016). Memory and Action Selection in

Human-Machine Interaction. Wiley-ISTE.

Kitajima, M. and Toyota, M. (2012). Simulating nav-

igation behaviour based on the architecture model

Model Human Processor with Real-Time Constraints

(MHP/RT). Behaviour & Information Technology,

31(1):41–58.

Kitajima, M. and Toyota, M. (2013). Decision-making and

action selection in Two Minds: An analysis based on

Model Human Processor with Realtime Constraints

(MHP/RT). Biologically Inspired Cognitive Architec-

tures, 5:82–93.

Kitajima, M. and Toyota, M. (2014). Hierarchical Structure

of Human Action Selections – An Update of Newell’s

Time Scale of Human Action. In Procedia Computer

Science, BICA 2014. 5th Annual International Confer-

ence on Biologically Inspired Cognitive Architectures,

volume 41, pages 321–325.

Kitajima, M. and Toyota, M. (2015). Multi-dimensional

memory frames and action generation in the MHP/RT

cognitive architecture. In Procedia Computer Sci-

ence, BICA 2015. 6th Annual International Confer-

ence on Biologically Inspired Cognitive Architectures,

volume 71, pages 202–207.

Laird, J. E., Newell, A., and Rosenbloom, P. S. (1987).

Soar: An architecture for general intelligence. Arti-

ﬁcial Intelligence, 33:1–64.

Newell, A. (1990). Uniﬁed Theories of Cognition (The

William James Lectures, 1987). Harvard University

Press, Cambridge, MA.

Newell, A. and Simon, H. A. (1972). Human Problem Solv-

ing. Prentice-Hall, Englewood Cliffs, NJ.

Simon, H. A. (1956). Rational choice and the structure of

the environment. Psychological Review, 63:129–138.

Nourishing Problem Solving Skills by Performing HCI Tasks - Relationships between the Methods of Problem Solving (Retrieval,

Discovery, or Search) and the Kinds of Acquired Problem Solving Skills

139