MERLIN COLLABORATION HANDBOOK
Challenges and Solutions in Global Collaborative Product Development
Päivi Parviainen, Juho Eskeli, Tanja Kynkäänniemi and Maarit Tihinen
VTT Technical Research Centre of Finland, Kaitoväylä 1, Oulu, Finland
Keywords: Collaborative development, global software development practices.
Abstract: Global, collaborative and distributed development is increasingly common in software development.
However, the traditional product and software development technologies do not support this way of working
well, e.g., time and cultural differences cause new requirements for these technologies. In this paper, we
introduce a public web-based handbook, collecting the challenges encountered in global collaborative
development by the companies, and also a large number of solutions that help in tackling these challenges.
The handbook was implemented using an ontology editor and generating HTML pages. In the final phase of
the development the handbook was validated by several external testers, with main feedback being that the
handbook was found useful, but more practical solutions would be welcome. Handbook was also updated
based on the feedback.
1 INTRODUCTION
In the perspective of growing size and complexity of
embedded systems, companies are not able to
develop all the required functionality by themselves.
As a result, suppliers specialize in specific
functionality or specific skills which they can sell to
others. This is clearly visible in the growing
numbers of the outsourcing constructions in the past
years. For example, a survey (VA Software, 2005)
found that 74% of the participated companies had
more than one development location. 48% had four
or more locations and 26 % had more than 20
locations. Furthermore, a major survey carried out
by the Software & Information Industry Association
(in January 2007) indicates that companies are
increasing their global development efforts: 57% of
the survey participants have significantly increased
offshore work in the past 18 months and many plan
to add still more in the next 18 months. Growth
strategy was cited as an important or critical driver
for 84% of respondents, while increasing speed to
market and productivity were the next most
important drivers. Collaborative engineering of
embedded systems has become a fact of life, and
currently there is no way back anymore; companies
have already outsourced large parts of their
developments to other companies, resulting in no
longer having the related skills available in their
own organisations. Instead, the companies need to
manage a complex situation of many partners, sub-
contractors, suppliers, software platforms and so on.
Practice has shown that the traditional single
company development technologies do not support
collaborative product development well. For
example, another survey shows that 80% of
companies are unsatisfied with their overall
collaborative development efforts. Survey
respondents expressed as main problems the poor
foundation for collaboration and poor management
of partner relationships. These problems are often
caused by, e.g., time difference and geographical
distribution that cause new requirements to the ways
of working and tools. Also, understanding each other
is not straightforward due to different backgrounds,
e.g., different terminologies, cultures etc. but needs
to be supported by technologies.
There are some experience reports about
challenges companies have faced in collaboration,
for example, Philips (Kommeren et al., 2007),
Siemens (Bass and Paulish, 2004), Motorola (Battin
et al., 2001), Alcatel (Ebert and Neve, 2001), and
Lucent Technologies (Herbsled et al., 2003). Also
several books that are discussing the problematics in
collaboration and solutions to address them have
been published (Karolak, 1998), (Carmel, 2005),
(Sahay et al., 2004), (Carmel and Tija, 2005),
(Damian, 2007). There are also conferences and
339
Parviainen P., Eskeli J., Kynkäänniemi T. and Tihinen M. (2008).
MERLIN COLLABORATION HANDBOOK - Challenges and Solutions in Global Collaborative Product Development.
In Proceedings of the Third International Conference on Software and Data Technologies, pages 339-346
DOI: 10.5220/0001874803390346
Copyright
c
SciTePress
workshops such as ICGSE (International Conference
on Global Software Engineering) dedicated to global
software engineering. However, these sources cover
only some viewpoints of collaborative software
development and until now no comprehensive
collection of challenges and solutions for product
development in collaboration could be found. Still,
all these sources have been used as input for the
Merlin Collaboration Handbook.
In this paper collaboration means that two or
more entities work together to create mutual value.
These entities can be companies, departments or
even teams and they can combine in any one of
several different business relationships and for very
different periods of time. Importantly, the entities
are physically in different locations, i.e., the
development is distributed.
This paper is organized as follows: first, in
chapter 2 we discuss the process of writing the
handbook. Next we present the contents of the
handbook in general level, including the structure
and technical implementation of the handbook in
chapter 3. In chapter 4 we discuss the validation of
the handbook and end the paper with some
conclusions and thoughts for further work.
2 MERLIN HANDBOOK
DEVELOPMENT
The purpose of the handbook - defined in co-
operation with the Merlin project consortium - is to
support operational collaborative development, i.e.,
help companies to take care of all critical aspects
during various phases of the collaborative project. In
practice this would be done by collecting, listing and
structuring these critical aspects as well as ways to
address them, called solutions into a handbook.
Furthermore, in order to make the handbook usable,
ways to access parts of the contents based on user’s
interests should be made easy.
The building of the handbook started by defining
it’s structure; an initial framework for the structure
of the handbook was derived from literature. CMMI
(CMMI, 2006) was used as the basic structure due to
its wide acceptance in software world and challenges
reported by others grouped according to the CMMI
structure. Based on the initial framework an
industrial inventory was carried out, including
interviewing the industrial partners of the Merlin
project. These interviews lead to many refinements
to the framework, especially in the details, although
several of the challenges encountered by the
interviewed companies were also mentioned in
literature. We have discussed these differences in
(Hyysalo et al., 2006), main being that the
approaches represented by literature, on one hand,
and industrial practitioners, on the other, towards
problems related to collaborative work are different.
The industry emphasizes technical aspects and
detailed problems concerning engineering practices,
while the literature focuses on solutions for more
general issues like communication and team
building. We found plenty of solutions for
management and support practices in the literature
but only few solutions for engineering practices.
Thus, in order to provide more content to the
handbook, collection of best practices from Merlin
industrial partners was also done via focused
interviews on selected topics. Results of these
focused interviews were then included as solutions
and experiences related to them in the handbook.
Finally, also the research and development work
done during the Merlin project was added to the
handbook as solutions.
In order to support usability, e.g., the different
views, and due to very large amount of content, the
handbook was not implemented as a physical book,
but a structured documentation solution was used to
support readability. Implementation is discussed in
more detail in section 3.3.
In practice, the handbook was developed in bi-
monthly workshops with the Merlin consortium to
refine implementation and contents of the handbook
based on prototypes. The workshops had also
representatives with experience on such repositories
and usability, for example. Also, in the last phase of
the project, the Handbook was validated by 16
external testers (this is discussed more in section 4).
3 HANDBOOK CONTENTS
This section describes the contents of the handbook
in general level. The complete handbook is available
in the internet (http://www.merlinhandbook.org).
3.1 Structure
The handbook structure is presented in the following
Figure 1.The handbook structure is formed using
topics; three main topics, namely management,
engineering and support practices and 21 subtopics,
such as collaboration management, requirements
definition, testing, configuration management, and
co-operative work.
ICSOFT 2008 - International Conference on Software and Data Technologies
340
Figure 1: Merlin collaboration handbook structure.
Each subtopic has number of important items,
altogether about 80 of them, such as product
roadmapping, conditions for collaboration, practices
for resolution of conflicting requirements, sharing of
test cases, unified CM practices, cultural differences,
etc.
Items are then still refined to questions, that
further detail the item. For example, “Are supplier
agreements and long-term framework agreements
used as input for roadmapping?” is a question under
“product roadmapping” item. The challenges faced
by industry are especially visible in the items and
questions of the handbook; they were first gathered
in the industrial inventory and critically refined
during the handbook development. The topics are
mainly general, following CMMI structure to a large
degree and the collaboration specific issues are
visible in the item, question and solution level; we
included only items, that are either specific for
collaborative development or as in most cases are
more difficult and complex to manage in
collaborative situation.
In addition to topics, items and questions, also
roles are defined. There are both responsible role
and main participating roles defined for each item.
The roles include all common product development
roles, for example, senior manager, project manager,
chief architect, etc. By roles, a checklist of
questions the role is responsible for can be retrieved
from the handbook. For example, Table 1 shows the
checklist for the role Quality manager.
For each item also solutions are included in the
handbook. Solutions are methods, techniques, tools,
and practices that help in taking care of important
items and they are classified according to their level
of validation:
Academic case, meaning that the
solution is proposed in literature, often
with academic case studies. These types
of solutions were included also to
provide ideas for items that were not so
well covered with industrially proven
solutions.
Industrial case, meaning that the
solution is proposed in literature or
developed in Merlin with industrial case
studies.
Legislation or standards.
Each solution has also a standard description
including ID, name, summary, description, evidence
of suitability (level of validation), type of solution,
collaboration dimensions, and references to further
information.
3.2 Example of Contents
An example of contents of the handbook is
requirements engineering. In the handbook
requirement engineering is divided to two topics,
requirements development and requirements
management. Requirements development has eight
important items and requirement management has
three important items defined in the handbook.
MERLIN COLLABORATION HANDBOOK - Challenges and Solutions in Global Collaborative Product Development
341
Table 1: Quality manager’s checklist.
A. Management practices
Are relationships between common quality management process and partners own quality practices defined? Responsible
Does the contracting process take into account all involved parties or stakeholders and do they have the required power of
authority or signing?
Participates
Are suppliers or customers co-operation capability analysed beforehand? Participates
Are the partners processes and quality management system of enough maturity? Participates
B. Engineering practices
Are the costs for non-quality taken into account while releasing the product. Have the costs for non-quality of the various
suppliers been estimated?
Participates
Are performed tests and test results communication practices between partners defined and followed? Participates
Are practices for incorporating feedback from customers to requirements development process defined and working? Participates
Have the results of acceptance testing been taken adequately into account? Participates
C. Support practices
Is common process across sites or partners as thin as possible and forced as little as possible? Responsible
Is the in-house review process defined and followed by each partner? Responsible
Are best practices recoded and used between partners? Responsible
Are common practices for quality assurance defined? Responsible
Is shared process improvement work defined and agreed upon in long term relationships? Responsible
Are common templates defined and used where applicable? Participates
Is the effectiveness of collaboration evaluated for example as part of end-of-project evaluations? Participates
An example of important item for requirements
development is “Clear prioritization rules and
practices / trade-off of the requirements”. This item
has five solutions in the handbook that help taking
care of priorisation. These solutions are methods that
base on giving values to different requirements
(pairwise comparisons, e.g. AHP), negotiation
approaches that base on the idea that the priority is
determined by reaching agreement between the
different stakeholders and dedicated requirements
priorisation methods and techniques specifically
supporting collaboration.
Another example of topic is collaboration
management. For this topic nine important items
have been defined, including for example,
“establishing / evaluating conditions for
collaboration” and “clear agreements with
suppliers”. The latter has four solutions, e.g.,
guidelines for acceptance criteria definition and
creating a contract.
3.3 Technical Implementation
Handbook data is stored and managed using Protégé.
In general, Protégé is an open platform tool for
ontology modelling and knowledge acquisition
framework (Protégé Web Pages). It offers a way to
manage the cross-references in a mass of textual
data in a RDF/OWL format (RDF Pages). There are
predefined ontologies available on the web, which
can be imported into Protégé and can then be used as
a basis for other ontologies.
To develop the Handbook with Protégé, a data
model of the Handbook was created. This was done
by studying the structure of the Handbook
(practices, topics, etc.), their relationships and the
requirements for categorizing scopes. According to
this study, the data model was designed. A decision
was made to use the owl: Tool -ontology as a basis
for the Handbook ontology. The Tool -ontology was
chosen because it’s content and link structure closely
resembled that of the handbook. The Tool -ontology
was then modified to reflect the Handbook data
model. When the structure was finalised, the
instances, that is, the content of the Handbook with
their relationship information, was inserted into the
ontology.
In order to be able to represent the Handbook
data in web format, conversion from OWL/RDF
format into something more suitable for our
purposes was needed. This was done by using
Protégé’s possibility to extend its functionality via
plugins. A special purpose plugin was written which
exports the OWL/RDF information into easily
usable XML format. In the new format the data is
structured as a tree, that is, on top there is a practice
with its attributes, a practice has items with their
own attributes and so on.
The next challenge was how to represent the
information to users via web interface. Major
requirement specified for the HTML Handbook was
to have a possibility to scope the content according
to user specifications in order to offer different
views into the contents.
Scoped view into handbook was achieved by
defining scoping parameters which can be used to
bring out the different views. When entering the
handbook, the user is first presented with a form in
ICSOFT 2008 - International Conference on Software and Data Technologies
342
Prótége XML
converter plugin
Handbook in
OWL/RDF format
Protégé conversion
terminal
Generated HTML
pages
XML data file
Handbook server
Scoping servlet
Client
Figure 2: Merlin Handbook data flow.
which he/she can tick suitable parameters to trigger
the scoping process. Consequently, the same
parameters are inserted in the Handbook ontology as
attributes. These attributes make it possible to scope
the content by using the XML data file generated by
the Protégé converter and the Java servlets which
ultimately render the selected content for the user.
The following Figure 2 illustrates how handbook
operates.
This approach makes the handbook content
management simple; after the updated content has
been defined in Protégé, it is only necessary to run
the converter plugin and to deploy the new XML file
to handbook web pages. This solution also
guarantees that all the web pages have uniform
layout.
From the usability point of view the scoping
alone was not enough for navigating the handbook
data. Therefore the handbook offers a search
mechanism for its users by the means of Apache
Lucene search-engine. Lucene offers many powerful
features, most notable being the offline indexing
support. Initially several other search engines were
tested, but these were deemed too slow, mainly
because they lacked the offline indexing feature and
instead relied on to dynamical crawling. Because
Lucene is available as open source it could be easily
modifed to support the special features of handbook,
namely the scoping.
The handbook was developed iteratively; bi-
monthly workshops were arranged where the
development versions of the handbook were
presented to project members. Most of the feedback
received from these sessions were improvements to
the user interface (UI) and general usability, but also
the scoping mechanism was defined in the
workshops. These workshops were found to be
especially important for the developers so they could
see they were on the right track and could receive
further guidance when needed. Also, they provided
the project members continuous updates to current
situation and opportunities to influence the
Handbook.
4 HANDBOOK VALIDATION
During the project, the handbook was validated in
two different ways in addition to the workshops that
gave continuous feedback for the development.
Firstly, 16 external testers used the handbook and
provided feedback and secondly the handbook was
used in an industrial case to support improving
subcontracting efficiency in a company. Both of
these are discussed in this chapter.
4.1 External Testers
In the final phase of the Merlin project, the
handbook was tried out by external testers. These
testers were found by asking the Merlin industrial
partners to find persons in their organizations that
have not been involved in the Merlin project but
who are knowledgeable in the topic, i.e.,
collaborative product development. Another source
for these external testers was the public seminars,
where the handbook prototypes were presented and
volunteers for testing asked for. The users were
typically project managers that had experience in
collaborative projects and altogether 16 external
users were granted access to the handbook during
development.
During the validation, the external users were
asked to think of a typical problem they would have
MERLIN COLLABORATION HANDBOOK - Challenges and Solutions in Global Collaborative Product Development
343
in collaborative product development and try to find
answers and help from the handbook to address this
topic. No further instructions on using the handbook
were given at first. Also, the feedback was to be
given in free format including the chosen problem
and the findings and other comments.
Feedback from the external users related mainly
to the usability or the content of the handbook.
Based on the testers’ experiences, the handbook
looked nice and worked fine. Also, one of the testers
noted that handbook had very clear page lay-out.
However, some handbook mechanisms needed
explanations or guidelines for use. For instance
scope selection page was not self-explanatory (e.g.,
what was meant by item, type of source, agreement
base, etc.). Also, bookmark mechanism was not
explained, thus it was not clear where the bookmarks
were accessible.
The comments to the content of the handbook,
related to new solutions that should be added to
handbook. As one of the testers reported, on many
problem area’s underlying documentation was not
yet given. Thus, it was suggested that following
information should be added to the handbook:
reporting practices, multi-site peer reviews, selecting
configuration management tools, interactive process
model based on best practices, and data on
measurements. Also, one of the testers reported that
the handbook included too little information on
measurements, metrics, reporting and follow-up in
both management and project management. It was
also requested that the handbook should answer to
following questions: what are readiness assessments
or checklists for collaboration, and why to work
collaboratively?
Furthermore, according to one of the testers, the
overall information in the handbook tended to be
quite theoretical. Hence, for practitioners, practical
or implementation examples were missing, meaning
that findings remained abstract and theoretical.
Some guidelines were provided in the handbook, but
it could not be found how to do that in practice. It
was also pointed out by another tester that the usage
of proven practices was a good idea, since a larger
number of examples would bring additional value to
the handbook.
It was also pointed out by one of the tester’s that
references to Google and Scholar Google sites
should be improved, e.g., to include the key words
of the specific publication in the URL, so that the
users wouldn’t have to retype the words themselves.
It was also noticed that common terminology for the
handbook should be provided in order to avoid
inconsistency of the terms used in the handbook.
Based on these comments by external testers,
handbook usability and content has been improved.
Now, the handbook is faster and easier to use. For
instance, help texts and terminology have been
added to the handbook in order to facilitate the use.
Also, information is much easier to find, then before
the users comments, since scoping and search
operations have been added to the handbook. With
the scoping operation, the content of the handbook
can be shown based on user’s needs and situations
and with the search operation specific topics, items
or solutions can be easily found from the handbook.
Also, based on the external testers’ comments and
wishes, the content of the handbook has been
improved, for instance, new solutions have been
created. User experiences also affected to the
content of the solutions, e.g., what attributes (i.e.
geographical distance, cultural differences, time
difference, etc.) are taken into account in the
solutions. Also, solutions’ references have been
updated. Furthermore, at the end of Merlin project,
all results from the project have been written in
solution format, so that the content of the handbook
would be more extensive and to include more
experience-based solutions than before, as the users
requested.
4.2 Industrial Case
The Merlin Handbook was used to first analyse and
then to improve the subcontracting practices of a
company participating in the Merlin project. The
aim of the case was specifically to improve
controllability and efficiency of the subcontracting.
The Handbook structure of items and questions was
used as interview framework to find out the
strengths and weaknesses of the current practices.
This resulted in several improvement areas for the
current practices but also to some additions to the
Handbook items, as some challenges identified were
not yet included in the Handbook. Then the
handbook was used to find solutions to the
improvement areas. Several solutions were found
and were then applied to the company’s needs.
The Handbook helped especially in gaining
confidence to change as the solutions and
experiences presented in the handbook supported the
company’s own ideas. Use of handbook helped also
in minimizing risks, as the handbook could help in
providing proven guidelines that can then be applied
to own situation. Also, instead of having to reinvent
the wheel the knowledge gained by larger network
of people could be utilized. That saved time, due to
ICSOFT 2008 - International Conference on Software and Data Technologies
344
avoiding using effort on basic issues and being able
to focus on adapting proven solutions to own needs.
As a result, due to the improved, more effective
practices, the number of the subcontracted personnel
could be significantly increased, meaning that more
work can now be subcontracted, freeing the
company’s own personnel to other tasks.
5 CONCLUSIONS AND
FURTHER WORK
In this paper we have introduced a public web-based
handbook, collecting the challenges encountered in
global collaborative development by the companies,
and also a large number of solutions that help in
tackling these challenges. The handbook was
implemented using an ontology editor and
generating HTML pages. In the final phase of the
development the handbook was validated by several
external testers, with main feedback being that the
handbook was found useful, but more practical
solutions would be welcome. Handbook was also
updated based on the feedback. The Handbook was
also found useful in improving a company’s
subcontracting practices, especially as it provided
confidence to change.
The handbook is a collection of both literature
and industrial experience. Especially the structure of
the handbook, the topics, items and questions are
based on challenges faced by industry. We have then
made a collection of available solutions to these
challenges and provided sources for further
information. Many of the solutions are based on
industrial experience, however, this is always
situation dependent and it is up to the user to
consider the usefulness of the solution to his/her
situation. Also, the topic and different usage
situations to be covered by the handbook is very
large. Thus, we realize we could not cover
everything during this three year project. However,
based on the feedback from the users of the
handbook so far, we can say that the handbook is
helpful for a company working in collaboration with
others; it can at least give ideas and triggers to
consider while doing the work, and even provide
complete, validated solutions to tackle the faced
challenges.
In order to further advance the handbook, we
have included an opportunity in the handbook to add
new solutions by the users. However, these new
solutions will first be reviewed by the Merlin project
partners before they are accepted to the handbook.
We also welcome other feedback. The handbook is
publicly available from http://www.merlinhandbook.
org.
ACKNOWLEDGEMENTS
The work described in this paper has been carried
out in the Merlin
1
ITEA
2
project in co-operation
with the whole consortium. The authors would like
to thank the Merlin project members for the active
participation in the handbook development.
REFERENCES
VA Software, 2005, Application Development and Open
Source Process Trends: Survey Analysis and Findings,
white paper, January 2005, available from:
http://www.vasoftware.com/gateway/pollresults.php
Kommeren, R., Parviainen, P.: Philips experiences in
global distributed software development. Empirical
Software Engineering Journal. Vol. 12, No. 6 -- 647-
660 (2007)
Bass, M., Paulish, D.: Global software development
process research at Siemens. In: The 3rd international
workshop on global software development, May 24,
2004, In proceedings of ICSE 2004, International
Conference on Software engineering, Edinburgh,
Scotland, May (2004)
Battin, R.D., Crocker, R., Kreidler, J., Subramanian, K.:
Leveraging Resources in Global Software
Development. IEEE Software, March/April 2001, pp
70–77 (2001)
Ebert, C,, De Neve, P. : Surviving global software
development, IEEE Software, March/April 2001, pp
62–69 (2001)
Herbsleb, J.D., Mockus, A., Finholt, T.A., Grinter, R.E.
An empirical study of global software development:
distance and speed. In the Proceedings of 23rd
International Conference on Software Engineering,
IEEE, Toronto, 2001.Also in. IEEE Trans Softw Eng
29(6):481–494, June 2003 (2001, 2003)
Karolak, D.W.: Global Software Development: Managing
Virtual Teams and Environments, Wiley-IEEE
Computer Society Pr; 1st edition (December 27, 1998)
Carmel, E.: Global Software Teams: Collaborating Across
Borders and Time Zones, Prentice Hall (December
1998)
Sahay, S., Nicholson, B., Krishna, S.: Global IT
Outsourcing: Software Development across Borders,
Cambridge University Press (January 12, 2004)
1
http://www.merlinproject.org
2
http://www.itea-office.org
MERLIN COLLABORATION HANDBOOK - Challenges and Solutions in Global Collaborative Product Development
345
Carmel, E., Tjia, P.: Offshoring Information Technology:
Sourcing and Outsourcing to a Global Workforce,
Cambridge University Press (June 13, 2005)
D. Damian. Stakeholders in Global Requirements
Engineering: Lessons learned from practice. IEEE
Software, Mar/Apr 2007.
CMMI for development, version 1.2., Technical Report
CMU/SEI-2006-TR-008. (2006)
Hyysalo, J., Parviainen, P. and Tihinen, M.: Collaborative
Embedded Systems Development: Survey of State of
the Practice. Proceedings of 13th Annual IEEE
International Symposium and Workshop on
Engineering of Computer Based Systems (ECBS
2006). pp. 130–138. (2006)
Protégé web pages: http://protege.stanford.edu/
RDF pages: RDF/XML Syntax Specification
http://www.w3.org/TR/rdf-syntax-grammar/.
ICSOFT 2008 - International Conference on Software and Data Technologies
346