DEVELOP ADAPTIVE WORKPLACE E-LEARNING
ENVIRONMENTS BY USING PERFORMANCE
MEASUREMENT SYSTEMS
Weijia Ran and Minhong Wang
Division of Information & Technology Studies, Faculty of Education, The University of Hong Kong, Hong Kong
Keywords: Workplace Learning, Adaptive E-learning, Performance Measurement System.
Abstract: Workplace learning is an environmental contextual and dynamic procedure. Needs and desires in workplace
learning arise from actions and practices in working environment; learning contents consist of explicit and
tacit knowledge dynamically created and intertwined with working and practicing. The development of an
effective workplace E-learning system is faced with several problems: 1) How to specify and update
learning needs and desires in contextual and dynamic workplace settings? 2) How to activate and formalize
knowledge sharing and contribution procedure for collecting knowledge emerged during practices in
working communities? 3) How to organize and store knowledge pieces in a way that reflects workplace
learning needs and supports adaptive learning content delivery? 4) How to incessantly update and adjust
learning contents to keep up with the changing working context? In order to solve these problems, we
propose an adaptive workplace learning model, in which the performance measurement result is used as an
indication of working proficiency, a reflection of learning needs, and a sign of the level and quality of
knowledge shared and contributed for achieving specific performance, with a view to organizing learning
contents and effectively guiding learning and knowledge sharing process.
1 INTRODUCTION
Recently the focus of E-learning is shifting from
implementation infrastructures and simply delivering
learning materials (Shute and Towle, 2003) by
incorporating theoretical aspects from domains such
as education or cognitive science. The concept of
adaptive E-learning environment is brought forward
in this trend. The theory ground of adaptive E-
learning is that learning effectiveness is influenced
by differences among individuals. The true power of
adaptive E-learning resides in its ability to provide
instructional contents that adapt to learners’ needs
and desires (Shute and Towle, 2003).
Workplace learning is an environmental
contextual and dynamic procedure. Needs and
desires in workplace learning arise from actions,
practices and activities in working environment and
concern job requirements, working performances
and proficiencies. A fundamental part of learning
contents which can fulfil these needs and desires
usually consists of explicit or tacit workplace
knowledge dynamically created and intertwined with
working and practicing. Many adaptive E-learning
systems are designed in school settings where
learning needs and contents are much different from
those in workplace sceneries. To facilitate an
adaptive E-learning environment in workplace E-
learning by utilizing the typical adaptive E-learning
mechanism is faced with several problems. Firstly,
how to specify and update learning needs and desires
in contextual and dynamic workplace settings?
Second, how to activate and formalize the
knowledge sharing and contribution procedure for
collecting working knowledge emerged during
practices in communities in workplace settings?
Third, once workplace knowledge is collected, how
to organize and store them to reflect workplace
learning needs and desires and smooth the progress
of adaptive learning content delivery? Fourth, how to
incessantly update and adjust learning contents to
keep up with the changing working context?
In order to solve these problems, we propose an
adaptive workplace learning model using
performance measurement systems to organize
learning contents and effectively guide the
knowledge sharing and adaptive learning content
142
Ran W. and Wang M. (2008).
DEVELOP ADAPTIVE WORKPLACE E-LEARNING ENVIRONMENTS BY USING PERFORMANCE MEASUREMENT SYSTEMS.
In Proceedings of the Tenth International Conference on Enterprise Information Systems - SAIC, pages 142-147
DOI: 10.5220/0001703601420147
Copyright
c
 SciTePress

delivery process. We hypothesis employees’
performance measurement results are the indication
of their working proficiencies, the reflection of their
learning needs and desires, and the sign of the level
and quality of knowledge shared and contributed by
them.
In the following, we review the typical
mechanism of adaptive E-learning system in the first
section. The second and third sections examine
problems encountered when using this mechanism
under a workplace learning context and review
relevant workplace learning theories, which lay the
foundation for our solution to these problems. The
fourth section presents our adaptive workplace E-
learning system model, and the fifth section
introduces the architecture design of this system.
Finally, there is a conclusion of the current work and
a brief introduction to the further work.
2 ADAPTIVE E-LEARNING
MECHANISM
The primary goal of an adaptive E-learning system is
to deliver the right content, to the right person, at the
right time, in the most appropriate way (Shute and
Towle, 2003). In other words, by referring to
individuals’ needs and desires, an adaptive E-
learning system decides what should be delivered,
when should be delivered and how should be
delivered.
In order to achieve the adaptability, the first step
in an adaptive E-learning system is to capture
learners’ needs and desires. A diversity of needs and
desires are identified to be adapted, such as learners’
learning goals, general abilities, and curriculum
achievement levels, etc. (Mödritscher et al., 2004).
There are typically two approaches to retrieve all
these different needs and desires. Macro-adaptation
approach retrieves needs and desires by using
diagnostic assessments before learners start to learn
certain learning contents. These needs and desires
are called domain-dependent information, which
shows learners’ proficiencies in learning contents
(Shute and Towle, 2003). Micro-adaptation approach
retrieves needs and desires by conducting
assessments of learners’ on-task performance such as
response errors, latencies etc. These needs and
desires are called domain-independent information,
which includes learners’ cognitive abilities,
preferences and so on (Shute and Towle, 2003).
Conventional tests are usually used in the first
approach, while in the second approach assessments
embedded within interactive, problem-solving, or
open-ended tasks may be used (Shute and Towle,
2003).
After learners’ needs and desires are identified
and retrieved, the next issue in adaptive E-learning
concerns how to interpret these needs and desires to
deduce what content should be delivered at what
time and in what an approach. Course materials are
decomposed into a set of Learning Objects (LOs)
(Learning Technology Standards Committee of the
IEEE, 2002) and stored in the content model.
Information about dependency relations among LOs
are identified and subsequently are used to decide
upon adaptations (Brusilovsky, 2003). The learner
model contains information that is collected from
assessments and is used by system to decide what,
how and when to deliver next (Shute and Towle,
2003). The instruction model defines adaptation
rules to describe how adaptation should be
performed (Paramythis and Loidl-Reisinger, 2004).
At last, adaptive model presents adaptive learning
contents depending on information retrieved from
former models (Shute and Towle, 2003).
To summarize, an adaptive E-learning system
delivers tailored learning contents at proper time in
an appropriate way relying on learners’ specific
needs and desires. Learning contents are stored in
the content model as LOs with a knowledge
structure. Needs and desires are deduced from
learners’ assessment results and represented in
learners’ models related to LOs. The adaptive model
picks up suitable LOs and presents to learners in a
proper way by interpreting adaptation rules defined
in instruction model.
3 PROBLEMS OF ADAPTIVE
E-LEARNING MECHANISM IN
WORKPLACE SETTINGS
Adaptive E-learning systems are usually designed
and implemented in school settings. However,
learning in workplace is much different from school
learning in terms of the learner, learning content and
learning context. In order to facilitate adaptive E-
learning in workplace, we should tackle the
following questions: 1) What are learners’ needs and
desires in a workplace context and how to capture
them? Can we use the usual approach such as pre-
task and on-task assessments to capture them? 2)
What are learning contents in a workplace context
and how to present and organize them? Are they
common course materials and can be presented as a
set of related LOs?
DEVELOP ADAPTIVE WORKPLACE E-LEARNING ENVIRONMENTS BY USING PERFORMANCE
MEASUREMENT SYSTEMS
143

In workplace learning, learning needs and desires
are more objective oriented and considerably
inspired by learning environments. From the
viewpoint of Malcom Knowles’s adult learning
study, adult learners are goal-oriented, relevancy-
oriented and practical. Needs and desires of adult
learning are distinct from college students learning
in that in organizations. Employees learn in the aim
of immediate application. Learners in school context
may put more emphasis on understanding of theories
and concepts and habitually aim at higher grades
(Constantine, 2004). From the perspective of
expectancy model in workplace learning transfer
theory (DeSimone et al., 2002; Kontoghiorghes,
2002), employees are motivated to learn if they
believe skills and knowledge learned can be utilized
back to job and are linked to intrinsic and extrinsic
rewards, and skills and knowledge learned can help
to enhance individual and organizational
performance. Other workplace learning theories
include learning is driven by learners’ needs of
meaning making(Winch and Ingram, 2002) and
social identity establishing (Brubaker and Cooper,
2000) in organizations.
To sum up, most significant learning needs and
desires in workplace learning are neither domain-
dependent (knowledge proficiencies of course
content) nor domain-independent (personal traits
such as cognitive abilities and preferences), rather,
they are driven by job contents, working
performances, achievements and recognitions in
organizational settings. Therefore, it might be hard
to deduce needs and desires in a workplace context
using assessment approaches in typical adaptive E-
learning systems.
In terms of learning contents, knowledge in
workplace is environmental contextual and dynamic
(Wang et al., 2005, 2006). Situated learning theory
believes that knowledge is defined under a specific
setting or context and facts are determined by
cultural standards and social practices (Tyre and
Hippel, 1997). (Lave and Wenger, 1991) have
described workplace learning as a process of
“changing participation in the culturally designed
settings of everyday life” and indicated that knowing
is created and intertwined with doing and knowledge
emerges during practices in communities. Working
knowledge is local and constantly created and
recreated inside communities eventually (Fenwick,
2001). Explicit knowledge can be generalized,
codified and formally transmitted within
organizations (Brookfield, 1992; Megginson, 1996;
Rigano and Edwards, 1998). In contrast, tacit
knowledge embeds in actions and practices of
specific social and cultural context in an
organization and is hard to capture. Much research
efforts have been put into how to convert tacit
knowledge into explicit knowledge with the
intention that to utilize tacit knowledge to benefit the
organizations (Nonaka, 1994; Eraut, 2000).
In brief, rather than pre-defined and fixed course
materials in conventional school instructions,
workplace learning contents dynamically generate
from working environments and are in explicit or
tacit forms inhabiting in various carriers such as
work documents, employees’ experience, experts’
advice etc. They are discrete and independent pieces
of information loosely distributing in an
organization. Although each piece of information
can be treated as an LO, it is difficult and less
meaningful to organize them within a knowledge
structure defining interdependent relationships such
as learning sequences or abstract levels.
4 ADAPTIVE WORKPLACE
E-LEARNING MODEL
In light of all this background considerations, we
propose an adaptive workplace E-learning model
driven by the performance measurement system in
organizations. The underlining hypothesis is that the
performance measurement results are the indicator of
employees’ in time working proficiencies, learning
needs and desires. Learning contents are contributed
by employees, stored and organized in the system
tagged with contributors’ performance measurement
results, and adaptively delivered to employees
relying on their performance expectations.
Performance measurement is a crucial procedure
for organization development and a main driver of
employees’ learning activities (Stephanie, 2005).
(Slizyte and Bakanauskiene, 2007) have summarized
it as a systematic procedure to improve performance
by setting performance objectives, assessing
performance, collecting and analyzing performance
data, and utilizing performance results to drive
performance development. There is a diversity of
performance measurement systems, such as Key
Performance Indicators (KPI), Balanced Scorecard
(BSC), and Excellence Model (EFQM) etc. Different
system emphasize on measurement of different
aspects. For example, BSC assesses performance
from perspectives such as financial, customers,
processes, learning and growth; EFQM focuses on a
range of elements such as people, leadership,
products etc. KPI is a flexible performance
measurement system which is used to assess almost
ICEIS 2008 - International Conference on Enterprise Information Systems
144

any perspective, whatever financial or non-financial,
depending on individual organization’s design. Plus,
it is still in the dominant role although there are a
growing number of organizations who have
implemented BSC and EFQM (Robinson et al.,
2005). Hence, KPI is selected as the performance
measurement system in our model.
KPI show a clear picture for each individual in
organization what is important and what they need to
do (Slizyte and Bakanauskiene, 2007). In a KPI
performance measurement system, organizational
vision and mission are interpreted into clear defined
department goals and objectives, which are then
broke down into performance targets related to each
job category reflecting specific organization
strategies, official and technical requirements for
individual employees. Thus, each job is assigned a
specific KPI, which is a set of items measuring
different performance perspectives. During a
performance assessment process, an employee’s
actual working performance is compared with
performance targets defined in his/her job KPI and
each KPI item is marked with a score similar to the
score in examinations. The set of score is called the
employee’s performance result or KPI Value. An
organization conducts performance measurements
with a certain frequency and an employee obtains a
certain performance result in each performance
measurement event.
Learning contents in our model come from
working knowledge contributed by employees in the
form of digital files. Each digital file is similar to an
LO in a typical adaptive E-learning system, and is
called a learning case in our model. Different
learning cases may contain different materials. A
case can be a piece of course material, a paragraph
of programming code, a recommended booklist or an
article about a project experience. We provide four
groups (Study Plan, Course Material,
Recommendation, and Experience) to categorize
cases. Each case is stored in the system under a
certain group, indexed with contributor’s system ID
and performance result obtained at the time of
contribution.
Learning content storage or delivery is triggered
when an employee inputs his/her current
performance result and expected performance result
into the system to retrieve a tailored learning
solution. If the employee’s performance result meets
predefined criteria, we assume that there might be
valuable working knowledge embedded in this
employee and the system would ask the employee to
contribute learning cases. Otherwise, the system
would deliver a learning solution containing cases
matching the employee’s input according to
predefined matching rules. Each learning solution
belongs to an employee and has its own lifetime,
which starts when it is delivered and ends when its
owner closes it. Employees control learning pace by
themselves and can evaluate and revise learning
cases once they finish learning them. Learning cases
with too much negative comments or too low access
rate would be eliminated from the learning content
base.
There are three major differences between a
typical adaptive E-learning mechanism and our
adaptive workplace E-learning model: 1) the former
retrieves the learner’s attributes relevant to learning
contents through assessments within the system; the
later deduces these information relying on
performance measurements in realistic
organizational settings; 2) the former has fixed
learning contents within a course scope; the later
bears a learning content base generated from
workplace knowledge and dynamically contributed,
adjusted and refined by learners and managers; 3)
the former stores learning contents as interdependent
LOs with distinct attributes; the later organizes
learning contents as learning cases under the KPI
system hierarchy and indexes them with
contributor’s KPI Value.
5 ARCHITECTURE DESIGN
Architecture of our adaptive workplace E-learning
system is designed as a conventional three-layer
structure (see Figure 1). Interface Layer is
responsible for interaction with employee-learners.
Application Layer contains four function modules.
The Learning Solution Manager deals with basic
events happened in a learning solution’s lifetime and
acts as a platform to use other modules when
necessary. It updates a learning solution record when
the learning solution starts or ends, uses the Learning
Case Creation Manager to store learning cases when
a learner has knowledge contribution, asks the Case
Retrieval Manager to search learning cases when a
learner requires a learning solution, and requests the
Case Adjustment Manager to refine a learning case
when a learner ends the learning case. The
Repository Layer stores learning content as well as
accessorial information supporting adaptive learning
content delivery (such as data stored in the KPI
System and the Employee). The Learning Case is the
organizational learning content base, including all
DEVELOP ADAPTIVE WORKPLACE E-LEARNING ENVIRONMENTS BY USING PERFORMANCE
MEASUREMENT SYSTEMS
145

Le arne r In te rf ace
Learner Layer
L earn ing Solu ti on
Ma nag er
St art Sol ution ()
SearchSoluti on()
EndCase()
EndSolution()
Le arni ng C ase
Cre at ion Mana ger
Cre ate()
C ase Re t rieva l
Ma nag er
Private Search()
Private GetKPIID()
Search()
Case Adjustment
Mana ger
Adju stTo Comm ent ()
AdjustTo Revision()
SolutionID, EmpID, State, StartTim e,
EndTime
Current KPI Value, Expecte d KPI Value
L earn ing Case <Co nt en t, Ca seSt at e,
St art T ime , End T ime, Co nt ribu to rI D>
Le arn ing So lu ti on
Learning Case
CaseID,
Type , C onten t , Co nt rib ut orI D, KPI ID ,
KPI V alue, Case Comm ent
<CommentID, Com mentValue,
R evisi on Sugg estio n>
KPI Syste m
KPI ID , D epa rtm ent
Position, ItemID
I t emN ame , It e m D escr i pt ion
Emplo yee
EmpID
Rea lNa me,
Department, Position
Ap pl icat io n La yer
R epo sit ory La yer
Figure 1: Adaptive Workplace E-learning System Architecture.
learning cases contributed by all learners; The
Learning Solution is individual learning content
base, including a set of personal adaptive learning
cases.
6 CONCLUSIONS
In order to improve the effectiveness of workplace
E-learning, we have proposed an adaptive workplace
E-learning model based on adaptive E-learning
mechanism and workplace learning theories. It aims
to facilitate an adaptive workplace E-learning
environment where 1) employees’ workplace
learning needs can be clearly specified and
personally satisfied; 2) working knowledge in
organizations can be well organized and
dynamically refined. The objective is achieved by:
1) using employees’ performance measurement
results to define their learning needs and desires; 2)
collecting working knowledge from employees,
storing them as learning cases, and indexing them
with contributors’ performance measurement results;
3) adjusting and refining learning cases based on
learners’ feedback. Three-layer system architecture
has been designed based on this model. The further
study consists of a detailed system design, a
prototype construction and the justification of the
effectiveness of this system.
ACKNOWLEDGEMENTS
This research is supported by a UGC CERG
research grant (No. RGC/HKU7169/07E) from the
Hong Kong SAR Government and a Seeding
Funding for Basic Research (200711159052) from
The University of Hong Kong.
REFERENCES
Brookfield, S. 1992. Uncovering Assumptions: The Key to
Reflective Practice. Adult Learning, 3(4):13-18.
Brubaker, R., Cooper, F. 2000. Beyond ‘identity’. Theory
and Society, 29:1-47.
ICEIS 2008 - International Conference on Enterprise Information Systems
146

Brusilovsky, P. 2003. Developing adaptive educational
hypermedia systems: From design models to
authoring tools. In Murray, T., Blessing, S.,
Ainsworth, S (eds.). Authoring tools for advanced
technology learning environment. Dordrecht: Kluwer
Academic Publishers.
Constantine, K. 2004. Reconceptualizing the learning
transfer conceptual framework: empirical validation
of a new systemic model. International Journal of
Training & Development, 8(3).
Derouin, R. E., Fritzsche, B. A. & SALAS, E. 2005 E-
Learning in Organizations. Journal of Management,
31:920.
Desimone, R. L., Werner, J. M. & Harris, D. M. 2002.
Human Resource Development. 3rd ed. Orlando, FL:
Harcourt, Inc.
Eraut, M. 2000. Non-formal Learning and Tacit
Knowledge in Professional Work. British Journal of
Educational Psychology, 70:113-136.
Fenwick, T. 2001 Tides of Change: New Themes and
Questions in Workplace Learning. New Directions for
Adult and Continuing Education, 92:3-17, Win.
Kontoghiorghes, C. 2002. Predicting motivation to learn
and motivation to transfer learning back to the job in
a service organization: a new systemic model for
training effectiveness. Performance Improvement
Quarterly, 15:114-129.
Lave, J., & Wenger, E. 1991. Situated Learning:
Legitimate Peripheral Participation. Cambridge:
Cambridge University Press.
Learning Technology Standards Committee of the IEEE.
2002. Draft Standard for Learning Object Metadata.
IEEE 1484.12.1-2002 ed. New York, Institute of
Electrical and Electronics Engineers.
Megginson, D. 1996. Planned and Emergent Learning.
Management Learning, 27(4):411-428.
Modritscher, F., Garc A-Barrios, V. M. & G Tl, C. 2004.
The Past, the Present and the Future of adaptive E-
Learning. Proceedings of ICL. Villach, Austria.
Nonaka, I. 1994. A Dynamic Theory of Knowledge
Creation. Organization Science, 5(1):14-37. Pedler,
M. (ed.). 1991. Action Learning in Practice. 2nd ed.
Aldershot: Gower.
Paramythis, A. & Loidl-Reisinger, S. 2004. Adaptive
Learning Environments and e-Learning Standards.
Electronic Journal on e-Learning, 2(1):181-194.
Rigano, D. & Edwards, J. 1998. Incorporating Reflection
into Work Practice. Management Learning,
29(4):431-446.
Robinson, H. S., Anumba, C. J., Carrillo, P. M. & Al-
Ghassani, A. M. 2005. Business performance
measurement practices in construction engineering
organisations. Measuring Business Excellence,
9(1):13-22.
Shute, V. & Towle, B. 2003. Adaptive E-Learning.
Educational Psychologist, 38(2):105-114.
Slizyte, A. & Bakanauskiene, I. 2007. Designing
performance measurement system in organization.
Organizacij Vadyba: Sisteminiai Tyrimai, 43.
Stephanie, S. 2005. Signs of the times. Training
Magazine.
Tyre, M. J. & Hippel, E. V. 1997. The Situated Nature of
Adaptive Learning in Organizations. Organization
Science, 8(1):71-82.
Wang, M., Wang, H. & Xu, D. 2005. The Design of
Intelligent Workflow Monitoring with Agent
Technology. Knowledge-based Systems, 18 (6): 257-
266.
Wang, M. & Wang, H. 2006. From Process Logic to
Business Logic -- A Cognitive Approach to Business
Process Management. Information & Management,
43(2):179-193.
Winch, A. & Ingram, H. 2002. Re-defining the focus of
workplace learning. International Journal of
Contemporary Hospitality Management, 14:361-367.
DEVELOP ADAPTIVE WORKPLACE E-LEARNING ENVIRONMENTS BY USING PERFORMANCE
MEASUREMENT SYSTEMS
147