A FIRST APPLICATION OF A COLLABORATION MATURITY

MODEL IN THE AUTOMOTIVE INDUSTRY

Imed Boughzala

and Gert-Jan de Vreede

2,3

Institut Télécom, 46 Rue Barrault, 75634 Paris Cedex 13, France

Center for Collaboration Science, University of Nebraska at Omaha, 6001 Dodge St., Omaha NE 68182, U.S.A.

Faculty of Technology, Policy and Management, Delft University of Technology, Delft, The Netherlands

Keywords: Collaboration, Collaboration quality, Performance, Collaboration technology, IT, Maturity model, Virtual

team.

Abstract: Trends like globalization and increased product and service complexity have pushed organizations to use

more distributed, cross-disciplinary, cross-cultural, virtual teams. In this context, the quality of collaboration

directly affects the quality of an organization’s outcomes and performance. This paper reports on the first

field application of a Collaboration Maturity Model (Col-MM) through an automotive industry field study.

This model was empirically developed during a series of Focus Group meetings with professional

collaboration experts to maximize its relevance and practical applicability. Col-MM is intended to be

sufficiently generic to be applied to any type of collaboration and useable to assess the collaboration

maturity of a given team holistically through self-assessments performed by practitioners. The purpose of

the study reported in this paper was to apply and evaluate the use of the Col-MM in practice. The results

should be of interest to academic researchers and information systems practitioners interested in

collaboration maturity assessment. The research contributes to the collaboration performance and (IT)

project management literature, theory and practice through a detailed case study that develops artefacts that

provide evidence of proof of value and proof of use in the field.

1 INTRODUCTION

Organizations form to create value and products that

individuals cannot create alone (Mintzberg 1979).

To ensure their organizational performance and

competitive advantage it is thus critical for

organizations to achieve successful collaboration

(Clark and Fujimoto, 1991; Hansen and Nohria,

2004). In today’s increasingly unstable and

competitive socio-economic environment, trends

like globalization and increased product and service

complexity have pushed organizations to use more

distributed, cross-disciplinary, cross-culture, virtual

teams (Chudoba et al., 2005). In this context, the

quality of collaboration directly affects the quality of

an organization’s outcomes and performance (Jordan

et al., 2002; Banker et al., 2006). This means that the

disposition and capabilities of an organization’s

work force to collaborate will directly affect

organizational performance, productivity and

profitability (Frost and Sullivan, 2007; Hansen and

Nohria, 2004).

It is important for organizations to assess the

quality of the collaboration in their teams. This will

enable them to identify measures to improve

collaboration by better selecting and designing the

appropriate collaboration technologies (IT/IS) and

therefore to improve the management of their virtual

teams and projects. This requires organizations to

answer questions such as: Under what conditions do

teams collaborate better? Are there different levels

of collaboration quality that can be recognized and

that teams should aim for? To what extent should

management styles be taken into account? Which

role should collaboration technologies play to foster

effective collaboration? How can we measure the

impact of collaboration on organizational

performance?

Several studies propose models and methods for

collaboration assessment from different points of

view: collaboration processes (see e.g. Pinsonneault

and Kraemer, 1997; Den Hengst et al., 2006) or

collaboration technologies and their usage (see e.g.

Damianos et al., 1999; Pinelle and Gutwin, 2003;

Boughzala I. and de Vreede G..

A FIRST APPLICATION OF A COLLABORATION MATURITY MODEL IN THE AUTOMOTIVE INDUSTRY.

DOI: 10.5220/0003633500280037

In Proceedings of the International Conference on Knowledge Management and Information Sharing (KMIS-2011), pages 28-37

ISBN: 978-989-8425-81-2

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

Herskovic et al., 2007). One way to assess the

overall collaboration quality of teams is through

maturity model approaches used extensively in

quality assurance for product development (Fraser et

al., 2002). Using these types of models as an

assessment instrument helps an organization to

identify best practices and trouble spots, and to

stimulate discussion among practitioners to initiate

activities for continuous improvement (Fraser et al.,

2003). However, few efforts have been reported on

using maturity models to assess collaboration. Those

that have are limited in that they apply only to

certain domains or just cover a few phases of the

project life cycle (Daoudi and Bourgault, 2007).

This paper reports on the first field application

and evaluation of a Collaboration Maturity Model

(Col-MM) in an automotive industry for assessing a

virtual team distributed in two European countries.

The purpose of the study reported in this paper was

to apply and evaluate the use of the Col-MM in

practice. Col-MM was empirically developed

through a Design Science perspective approach

(Hevner et al., 2004) during a series of Focus Group

meetings with professional collaboration experts to

maximize its relevance and practical applicability

(Boughzala & Vreede, 2012). It was intended to be

sufficiently generic to be applied to any type of

collaboration, virtual or not (e.g. project teams,

organizational teams, cross functional/organizational

teams, inter-organizational team, or communities of

practice) and useable to assess the collaboration

maturity of a given team holistically by practitioners

for conducting self-assessments.

The remainder of this paper is structured as

follows. We first introduce the methodological

background related to maturity models in general

and the Col-MM in particular. Next, we report on

the application and the evaluation of the Col-MM in

a field study in the automotive industry. Last, we

discuss the appropriateness and usefulness of Col-

MM, followed by our conclusions which summarize

the limitations of this study and present future

research directions.

2 BACKGROUND

Maturity, literally meaning ‘ripeness’, describes the

transition from an initial to a more advanced state,

possibly through a number of intermediate states

(Fraser et al., 2002). The fundamental underlying

assumption of maturity models is that a higher level

of maturity will result in higher performance.

Maturity models reflect the degree to which key

processes or activities are defined, managed,

measured, and executed effectively. They typically

describe the characteristics of an activity at a

number of different levels of performance (Fraser et

al., 2003). “At the lowest level, the performance of

an activity may be rather ad hoc or depend on the

initiative of an individual, so that the outcome is

unlikely to be predictable or reproducible. As the

level increases, activities are performed more

systematically and are well defined and managed. At

the highest level, ‘best practices’ are adopted where

appropriate and are subject to a continuous

improvement process” (Fraser et al., 2003 p.1500).

2.1 Maturity Models

Approaches to determine process or capability

maturity are increasingly applied to various aspects

of product development, both as an assessment

instrument and as part of an improvement

framework (Dooley et al., 2001). Most maturity

models define an organization’s typical behaviour

for several key processes or activities at various

levels of ‘maturity’ (Fraser et al., 2003). Maturity

models provide an instantaneous snapshot of a

situation and a framework for defining and

prioritizing improvement measures. The key

strengths of maturity models include:

• They are simple to use and often require simple

quantitative analysis.

• They can be applied from both functional and

cross-functional perspectives.

• They provide opportunities for consensus and

team building around a common language and a

shared understanding and perception.

• They can be performed by external auditors or

through self-assessment.

One of the earliest maturity models is Crosby’s

Quality Management Maturity Grid (QMMG)

(Crosby, 1979), which was developed to evaluate the

status and evolution of a firm’s approach to quality

management. Subsequently, other maturity models

have been proposed for a range of activities

including quality assurance (Crosby, 1979), software

development (Paulk et al., 1993), supplier

relationships (Macbeth and Ferguson, 1994),

innovation (Chiesa et al., 1996), product design

(Fraser et al., 2001), R&D effectiveness (McGrath,

1996), product reliability (Sander and Brombacher,

2000), and knowledge management (Hsieh et al.,

2009). One of the best-known maturity models is the

Capability Maturity Model (CMM) for software

engineering (based on the Process Maturity

A FIRST APPLICATION OF A COLLABORATION MATURITY MODEL IN THE AUTOMOTIVE INDUSTRY

Framework of Watts Humphrey, quoted in Paulk et

al., 1993), developed at the Software Engineering

Institute (SEI). Unlike the other maturity models,

CMM is a more extensive framework in which each

maturity level contains a number of key process

areas (KPAs) containing common features and key

practices to achieve stated goals. A number of

studies of the software CMM have shown links

between maturity and software quality (e.g. Harter et

al., 2000). This model (with multiple variations) is

widely used in the software industry as part of

quality certification (SEI Certification).

Nowadays several maturity models have been

proposed that aim at clearly identifying the

organizational competences associated with best

practices (Fraser et al., 2002). In practice, however,

many maturity models are intended to be used as

part of an improvement process, and not primarily as

absolute measures of performance (Fraser et al.,

2002). Few maturity models have been validated in

the way of performance assessment. An exception is

Dooley et al.’s (2001) study that demonstrated a

positive correlation between New Product

Development (NPD) process maturity and outcome.

A few maturity models related to collaboration

have been proposed. Lebrun et al.’s (1998) model

defined maturity levels of concurrent engineering in

a virtual company. Their model emphasizes the

management of new products and processes in

temporary collaborative projects. Fraser et al.’s

(2003) model is intended to apply to all product

development activities; it is not restricted to software

products. Their model gives particular importance to

organizational-level collaboration between partners

in a product development network. Finally, the

model by Ramasubbu et al. (2005) focuses on

distributed software development. It represents an

effort to fill the gap in models like CMM by

introducing several dimensions related to

collaboration in distributed development settings.

Each of the above collaboration maturity models

is founded on the assumption that the quality of a

product is related to the quality of the collaboration

process. The value of each of the models is that they

emphasize and raise awareness on the issue of

collaboration maturity in an organizational setting.

Notwithstanding the individual strengths of each of

the above models, a number of key limitations exist.

First, few applications have been reported (limited

information on their model in practice) and reported

ones have not been validated empirically (Daoudi

and Bourgault, 2007). Second, their application is

specific for only certain types of collaboration (e.g.

inter-organizational, virtual organizations, or

distributed projects), for certain application domains,

or for certain project life cycle phases. Third, most

models are descriptive in nature, helping to identify

collaboration-related problems without proposing

solutions. Finally, little is known about whether the

use of these models leads to actual performance

improvements.

2.2 The Design and Structure of the

Collaboration Maturity Model

In the literature, collaboration has been defined in

different ways (Levan, 2004; Briggs et al., 2006;

Boughzala, 2007). In the context of this study, we

define collaboration as a process in which two or

more agents (individuals or organizations) share

resources and skills to solve problems so that they

can jointly accomplish one or more activities.

During this process, the agents communicate with

each other to coordinate their tasks. Based on this

definition of collaboration, we define collaboration

maturity as a team’s current maximum capability to

collaborate where team members effectively

communicate, reach shared understanding, and

adjust their tasks and behaviours to produce high

quality outcomes.

The main objective of our research is to

introduce a new collaboration maturity model that

addresses some of the limitations described above.

This model aims to holistically assess the

collaboration maturity of a (virtual) team that uses

several collaboration technologies. However, its

applicability is not limited to a particular form of

collaboration and the model can be used for different

settings. Further, it supports the development of

recommendations in form of an action plan to reach

improved project management, collaboration

performance and quality of collaboration outcomes.

The Col-MM was designed during a design

science study in which we cooperated with a Focus

Group consisting of professional collaboration

experts. These experts included 15 Chief Knowledge

Management Officers (CKMOs) from companies of

different sizes in different sectors, holding at least a

master-level degree from different areas, and having

at least 15-19 years of work experience with 50% of

them having 5-9 years as a CKMO. Their average

age was 48 and 73% of them were male. They were

accustomed to meet in the context of a business

association to share their best practices regarding

methods, techniques and tools in the collaboration

and knowledge management area. The involvement

of the experts group enabled us to combine

relevance and rigor by meeting a business need with

KMIS 2011 - International Conference on Knowledge Management and Information Sharing

applicable knowledge and so to maximize the

resulting artefacts’ relevance and applicability. The

experts expressed the following critical requirements

for the Col-MM:

• Resource Efficient: The Col-MM should be fast

to complete.

• Rich Data: The Col-MM should report on

different points of view and concerns from the

workplace, using both quantitative and

qualitative data.

• Limited need for Further advanced Data

analysis: The supporting tool should provide

integrated support for results interpretation.

• Self-assessment: Practitioners should be able to

apply the Col-MM themselves.

• Constructive Learning: The Col-MM should

promote team building and organizational

learning rather than control and sanction.

After a series of meetings with the Focus Group

in which several initial versions of the Col-MM

were presented and pilot results were shared, the

first full version of Col-MM was completed

(Boughzala and Vreede 2012). The Col-MM

consisted of a number of artefacts including: The

Col-MM structure that describes the collaboration

areas of concerns (topics) and their related criteria;

The Col-MM questionnaire that includes questions,

levels of rating and mathematical equations for

analysis; The Col-MM method that (a) defines the

steps and provides guidance on how to run the Col-

MM questionnaire in the field, and (b) supports the

development of recommendations; and the Col-MM

tool which is a customized MS Excel application

that represents the implementation of the above

artefacts, and enables the execution of a concrete

assessment by enabling the collection and analysis

of quantitatively and qualitatively questionnaire

data. It provides different presentations of results

(e.g. individual and team spider diagrams,

comparison curves, and cloud matrices) and the

results’ report generation.

The Col-MM distinguishes between four

maturity levels: Ad-hoc, Exploring, Managing, and

Optimizing. At the Ad-hoc level, teams are

collaboratively immature. Individuals have many

difficulties to communicate effectively, to reach

shared understanding, and to adjust their tasks and

behaviours to produce high quality outcomes

together. At the Exploring level, teams are well

aware of their weaknesses in terms of collaboration

quality. Individuals try work together to produce

valuable outcomes, but are faced with many

collaboration challenges. Some initiatives to address

these are attempted but without major impacts. At

the Managing level, individuals are able to produce

collaborative outcomes of good quality. They have

overcome many challenges to collaborate

productively, but there still is room for

improvement. At the Optimizing level, teams are

collaboratively mature. Teams work together

optimally and accomplish high quality collaborative

outcomes. Furthermore, they engage in critical self-

reflection and continuous improvement efforts.

The Col-MM explores the maturity of a given

team holistically from different perspectives related

to collaboration. The following perspectives, or

areas of concerns, were considered essential by the

participants in the Focus Group meetings

(Boughzala and Vreede, 2012):

• Collaboration Characterizing: This covers the

characteristics of the collaboration.

• Collaboration Steering: This covers the way in

which collaboration processes and activities are

managed.

• Collaboration Processing: This covers how

actors perform collaboration on a daily basis.

• Information and Knowledge Integration: This

covers how actors manage the information and

knowledge required for productive collaboration.

Table 1: Col-MM areas of concerns and criteria.

Areas of concern Criteria

Collaboration

Characterizing

1. Collaboration object

2. Collaboration depth

3. Working mode

4. Interaction intensity

5. Collaboration forms

6. Formalization of relationships

7. Commitment and availability of

individuals

8. Collaboration boundaries

Collaboration

Steering

9. Collaboration goal

10. Management style

11. Decision-making

12. Leadership endorsement

13. Rewarding

14. Collaboration progress

Collaboration

Processing

15. Collaboration framework

16. Resources sharing

17. Awareness

18. Conflicts management

19. Engineering (methods and

technologies)

Information and

Knowledge

Integration

20. Information collection

21. Information structuring

22. Information access

23. Knowledge validation

24. Knowledge reusing

25. Knowledge creation

A FIRST APPLICATION OF A COLLABORATION MATURITY MODEL IN THE AUTOMOTIVE INDUSTRY

For each area of concern, a number of criteria

were defined (see Table 1). These criteria represent

the topics for a questionnaire (Col-MM

questionnaire). Each criterion is represented by an

item that is evaluated on a 4-point scale. To support

the respondents, the levels of each criterion are

described briefly, with examples wherever possible.

An example of a criterion item is provided in Figure

1. When a respondent cannot answer, no score is

recorded. The more often criteria are rated at 4 by

the respondents, the higher the collaboration

maturity of the community under investigation is.

Figure 1: Example of criterion in Col-MM.

In essence, the Col-MM is structured as a library

of criteria. Sometimes, not all criteria are relevant.

So, an organization can decide which criteria fit its

particular context. It can also decide to expand the

set of criteria. Also, for some organizations certain

criteria may be more important than others. In such

situations, it is possible to assign different weights to

the criteria.

3 METHOD

The Col-MM was developed following Hevner et

al.’s (2004) design science approach. In this paper

we will not report on the development of the Col-

MM but only on its first field application and

evaluation to demonstrate the model’s practical

feasibility and utility. This study therefore answers

Hevner et al.’s Design Evaluation Framework

recommendation for the use observational methods

(2004 p. 86). Our role as researchers was limited to

the organization and execution of (group)

interviews, the analysis of collected interview data,

and the gathering of participants’ feedback regarding

Col-MM. Our interventions during the study were

only aimed at supporting the organization in

achieving its goals in the project. The researchers

had no personal stake in the project, neither with the

problem situation nor with the solutions that were to

be explored. The primary motivation for the client

organization to involve the researchers was its desire

to assess and improve the collaboration in a number

of its key teams.

Research data was collected from both

quantitative and qualitative sources to enable a rich

understanding of the application of the Col-MM in

practice. First, while observing the different

activities in the study, we kept notes of incidents,

remarks and events that conveyed critical

information. Second, the (group) interview results

were analyzed to gain insight into (1) the

participants’ reaction and understanding of the

interview questions, and (2) analyze specific

feedback regarding the Col-MM. Finally, we invited

participants on all levels to share feedback on the

Col-MM method and artefacts.

4 APPLICATION IN AN

AUTOMOTIVE INDUSTRY

FIELD STUDY

A large multinational automotive firm had a desire

to assess the collaboration performance of some of

their virtual teams. This company had previously

established a new organizational matrix structure,

based on the “management by project” principle. To

assess the ‘fit’ of this new structure in the context of

a recent merger-acquisition and to see if all the

constituent brands work as a one single group, the

company decided to assess the overall organizational

performance in terms of synergy between the

different sites and brands, productivity, quality of

the products, and the balance between product

diversity and process complexity. The collaboration

maturity assessment was part of this larger

organizational performance assessment.

As a first step it was decided to apply the Col-

MM to measure the collaboration maturity of one

virtual team distributed over two European countries

(two sites) with different cultures, different work

habits, and different management styles. This virtual

team was in charge of the “Engine After Treatment

System” (EATS) that was part of a larger

development project of a new diesel engine that was

taking place under the responsibility of a business

unit distributed over three countries. The leading site

in this project will be referred to as site A below.

4.1 Field Study Steps

The field study was performed over the course of 5

KMIS 2011 - International Conference on Knowledge Management and Information Sharing

Figure 2: The seven steps in the Col-MM method.

Figure 3: The Col-MM tool data collection and analysis.

months. It was applied and followed the Col-MM

method steps (Figure 2).

At the scoping step, the purpose of the Col-MM

analysis was defined according to the company’s

requirements. As presented to the stakeholders, the

objectives of the Col-MM analysis were to check:

• If the organization had adequate capabilities to

effectively support high quality collaboration.

• If collaboration technologies were well selected

and configured.

• If collaboration technologies provided were

effectively used.

• If there were critical issues related to cultural

differences (national, organizational,

technological, etc), given the merger-acquisition

context.

The data collection was performed through

individual and/or collective interviews based on the

Col-MM questionnaire (quantitative data). The

selection of respondents was done with the

assistance of the firm’ CKMO according to criteria

such as job position, responsibility, process step

intervention, and working experience. All

respondents had similar levels of education (MSc

degree) but from different engineering fields:

mechanical, electronic, electric, and industrial. The

Col-MM questionnaire was sent to the respondents

before the meeting with an introduction of the

company’s Col-MM objectives. Anonymity and

confidentiality of the treatment of the responses

were formally assured. Nine individual interviews

were conducted face-to-face in the respondents’

native language in the two European countries (3 in

the site A and 6 in the site B). Each interview lasted

about 90 minutes. During the interviews the Col-

MM tool (Figure 3) was used for data collection,

followed by a first quantitative data analysis. This

analysis presented individual perceptions about the

collaboration maturity of the team. It also helped to

identify perception differences concerning the

different criteria. All interviews were recorded for

further qualitative data analysis, consisting of

content/thematic analysis. This analysis helped to

get a more in-depth understanding of these

perception differences for each criterion or group of

criteria (area of concern). Two collective interviews

(one for each site) were conducted to examine these

perception gaps on some criteria. Follow-up

discussions and consensus building efforts were

carried out for relevant scores, in order to settle on

an acceptable assessment. The cross analysis yielded

additional interpretations by combining criteria for

specific measurements of capabilities according to

the focus of the assessment, such as project

management, knowledge management, IT adequacy,

value creation, and organizational learning.

The last step of the Col-MM method concerns



Team spider diagram

Comparison curve

Individual spider diagram

Superposition of two diagrams

0,0

1,0

2,0

3,0

4,0

0,01,02,03,0

Cloud matrix

Team spider diagram

Comparison curve

Individual spider diagram

Superposition of two diagrams

0,0

1,0

2,0

3,0

4,0

0,01,02,03,0

Cloud matrix

A FIRST APPLICATION OF A COLLABORATION MATURITY MODEL IN THE AUTOMOTIVE INDUSTRY

the definition of an action plan. This plan was

included in the report. An initial version of the

report was sent to the respondents to solicit any

corrections before the final report was prepared. A

final presentation to the company’s top management

reported on the results and provide

recommendations in form of a list of suggested

future actions.

4.2 Findings

The findings were reported as observations and

discussions of the different recorded perceptions

related to the Col-MM criteria and topics. Examples

of findings reported to top management according to

the four Col-MM areas of concern include:

• Collaboration Characterizing: There were

virtually no differences between the different

sites in terms of their perceptions regarding the

nature of collaboration. We found similar

understandings of collaboration goals and team

members’ commitment for both sites. This may

have been facilitated by the technical subject

matter that the team members in the different

sites had to collaborate on; this created a

common language and hence understanding.

• Collaboration Steering: We noticed different

perceptions between the two sites with respect to

project management style and decision-making

(hierarchical management vs. consensual

management). Site B respondents felt unfairly

rewarded compared to site A. They felt that

because site A had the project lead, its

employees always had an advantage.

• Collaboration Processing: We noticed that site

B respondents had less awareness about different

collaboration approaches to enhance the team’s

performance. Because of their positions and

responsibilities in the process, they focused more

on their individual contribution to the overall

process rather than on developing collaborative

relationships. We also noticed differences in

terms of conflict management by the leadership

in each site: Conflict management in the site A

was based on consensus while in site B it was

based on hierarchical decision making and

negotiation.

• Information and Knowledge Integration: We

found different perceptions regarding

information access. For site A respondents,

access to information was not organized as well

as they wished. Information was very distributed

and access should be simplified. We found

consistent perceptions between the two sites

regarding collaborative knowledge creation; both

sites felt this process was well organized.

Through the qualitative data analysis we found

that some cultural differences between sites

appeared to be related more the organizational

culture rather than to the national culture. For

example, the balance between private and

professional life appeared to be different. Also, there

was a different brand identity: Site B respondents

felt they were still belonging to their original brand

(i.e. from before the merger) rather than to the group

of brands. We also found different work attitudes: In

site B respondents were more reactive compared to

the respondents in site A being more proactive.

According to some respondents, this was because of

their position in the project. Possible explanations

could be related to their contracts type (tenure

status) and social protection.

The general findings reported can be summarized

as follows:

• Collaboration was mainly based on “individuals’

goodwill” as for example related to resource

sharing and knowledge management.

• The team was not as collaboratively mature as

was expected – they were at the Exploring Level.

Because of the asymmetric collaboration

awareness between the two sites, their

collaboration is mostly of a coordination nature.

This makes it difficult to further improve the

quality of their outcomes.

• The new matrix structure did not resolve all

problems with respect to the imbalance between

responsibility and authority.

In the final report, various recommendations

were proposed, including:

• Make collaboration a clear strategic goal in all

project management initiatives.

•

Re-think the management of collaboration

(steering) and provide training for managers.

• Nominate full-time facilitators for collaboration.

• Take into account diversity aspects related to

culture.

• Make explicit recognitions for the contributions

of every actor toward effective collaboration.

After six months, we learned that three of the

suggested recommendations were followed up with

concrete actions:

• The first recommendation was clearly mentioned

in the company’s project management standard.

• Following the third recommendation, one full

time collaboration facilitator was assigned to

each business unit.

KMIS 2011 - International Conference on Knowledge Management and Information Sharing

• Following the fifth recommendation, a

‘collaboration capability’ criterion was added to

the annual individual performance assessment.

5 DISCUSSION

During the application of the Col-MM during this

field study, we gathered various experiences and

feedback regarding the appropriateness and

usefulness of Col-MM. According to the

respondents, the Col-MM analysis was satisfactory

and correctly represented their perceptions. It

focused on real collaboration problems and allowed

traditionally ‘unspoken issues’ to surface. They were

also satisfied with the feedback provided to top

management and the subsequent actions that were

taken related to the assessment’s recommendations.

According to the operational managers, the results

were relevant. Further, most of them felt able to

reuse the Col-MM by themselves in the future.

According to the top managers, the study was

satisfactory in terms of results and

recommendations, as they confirmed and reinforced

some of their own perceptions. This allowed them,

for example, to focus more on the organizational

culture than on national culture and to understand

the problems related to the project-based new

organizational structure.

We also received feedback and recommendations

from the respondents on the Col-MM questionnaire

such as the possibility to review some criteria and

questions. The respondents stated that some criteria

were a little difficult to understand. Also, the

nuances between levels of responses were

sometimes subjective or difficult to distinguish. In

addition, they proposed to add some criteria such as

culture, work experience, and practice diversity, and

to rename some areas of concern such as

”collaboration readiness” instead of ”collaboration

characterizing” and ”collaboration management”

instead of ”collaboration processing”. Finally, they

suggested putting a stronger focus on virtualness

(i.e. the extent to which a process can be virtualized

(Martins et al., 2004)) and collaboration technology

rather than on information and knowledge

integration. Interestingly, this was contrary to the

wishes expressed by the focus group. However,

since the Col-MM is developed as a library of

criteria, the review of the Col-MM structure

according to a specific context is possible and

therefore the respondents’ suggestions can be easily

accommodated. In terms of execution, most

respondents expressed that they preferred the use of

collective rather than individual interviews as this

would enable a faster application of the Col-MM

process.

Based on the experiences and feedback from this

field study, we observe the following regarding the

extent to which Col-MM meets the requirements as

proposed by the Focus Group experts:

• Resource efficient: Col-MM appears to be

resource efficient. A total of 36 hours were

spent: 1.5 hours for the assessment preparation,

16.5 hours for the engineering interviews, 3

hours for the CKMO interviews, 3 hours for the

top management interviews and 12 hours for the

analysis and report preparation. We feel that this

is a modest and reasonable effort in terms of

resources spent.

• Rich data: The combined use of quantitative and

qualitative data analysis enabled richer findings.

We felt that qualitative observations enabled us

to better uncover and interpret the various points

of views expressed by the respondents through

the Col-MM questionnaire.

• Limited need for further advanced data analysis:

The analysis needs in the field application were

limited and the Col-MM tool provided sufficient

support (among others the report generation)..

• Self-assessment: The operational managers

expressed confidence that they could perform

future applications of the Col-MM themselves.

• Constructive learning: The respondents’

feedback shows that when the Col-MM study is

carefully communicated, participation can be

effective and generate discussions on real

problems that further facilitate the acceptance of

proposed solutions. In this respect, anonymity

and confidentiality seem to be crucial. This was

confirmed by feedback from the participants.

6 CONCLUSIONS

In this paper, we report on the first field application

and evaluation of an initial version of a new

collaboration maturity model, Col-MM, to assist in

the assessment of teams’ collaboration performance.

The Col-MM was developed prescriptively to meet a

real business need as expressed by 15 CKMOs and

others experts that are regularly confronted with

collaboration performance challenges. Our

contribution is both theoretical and practical as we

propose a model, an application method, a

supporting tool, and empirical evidence of their

application. Our experiences show that the Col-MM

A FIRST APPLICATION OF A COLLABORATION MATURITY MODEL IN THE AUTOMOTIVE INDUSTRY

can be applied in a resource-efficient fashion and

yields results that are useful for organizations.

However, there are limitations related to this

work with respect to the Design Science Evaluation

Framework. First, our empirical evidence is based

on several pilot studies but only a single field

application. Further field studies have to be executed

to expand the evaluation of the Col-MM artefacts

and to further enhance the Col-MM. Particular care

will have to be taken to ensure that Col-MM can

take into account all levels of collaboration and all

collaboration processes in an organization in

different settings. This cannot be achieved by just

expanding the number of criteria as this will overly

complicate the use of the model. Second, at this

stage, the Col-MM cannot yet be used to investigate

a correlation between collaboration maturity levels

and organizational/team performance. However, it

provides a first step into this direction.

We recommend several directions for future

research to enhance the current version of Col-MM.

First, the model has to be applied in different types

of organizations for different types of teams. The

experiences from these applications will assist in the

further development and evaluation of the Col-MM

artefacts. Second, organizational and team

performance measures have to be developed to

enable an analysis of the relationship between

collaboration maturity and organizational

productivity. Fourth, from a behavioural science

perspective, some further confirmatory studies

should be performed using Structural Equation

Modelling (Bollen, 1989) to validate the correlation

between these variables (i.e. Col-MM constructs and

performance).

REFERENCES

Banker, R., Bardhan, I., Asdemir, O. (2006).

Understanding the impact of collaboration software on

product design and development, Information Systems

Research, 17, 4.

Bollen K. A. (1989). Structural Equations with Latent

Variables, Wiley.

Boughzala I. (2007). Ingénierie de la collaboration:

théories, technologies et pratiques, Paris, Hermès,

September 2007.

Boughzala, I., Vreede, G. J. de (2012). A Collaboration

Maturity Model: Development and Exploratory

Application, Proceedings of the 45

Hawaii

International Conference on System Sciences, Hawaii,

January 2012.

Briggs, R. O., Kolfschoten, G. L., Vreede, G. J. de, &

Dean, D. L. (2006, August). Defining Key Concepts

for Collaboration Engineering, in: Irma Garcia, Raúl

Trejo (eds.). Proceedings of the 12

Americas

Conference on Information Systems, Acapulco,

Mexico, 121-128.

Chiesa, V., Coughlan, P. and Voss, C. (1996).

Development of a technical innovation audit. Journal

of Product Innovation Management, 13, 2, 105-136.

Chudoba K. M., Wynn E., Lu M., & Watson-Manheim M.

B. (2005). How virtual are we? Measuring virtuality

and understanding its impact in a global organization,

Information Systems Journal, 15, 279–306.

Clark, K. B., T. Fujimoto. (1991). Product development

performance: Strategy, organization and management

in the world auto industry. Harvard Business School

Press, Boston, MA.

Crosby, P. B. (1979). Quality is Free, McGraw-Hill, New

York.

Damianos L., Hirschman L., Kozierok R., Kurtz J.,

Greenberg A., Holgado R., Walls K., Laskowski S.,

Scholtz J. (1999). Evaluation for Collaborative

Systems, ACM Computing Surveys (CSUR), 31, 2,

Article 15.

Daoudi J., Bourgault M. (2007) What Do We Know About

Collaborative Maturity? 4th Annual Sprott Doctoral

Symposium, April 19-20.

Den Hengst M., Kolfschoten G., Dean D. L., &

Chakrapani, A. (2006). Assessing the quality of

collaborative processes. Proceeding of the 39th annual

Hawaii International Conference on System Sciences,

IEEE Computer Society.

Dooley, K., Subra, A., & Anderson, J. (2001). Maturity

and its Impact on New Product Development Project

Performance, Research in Engineering Design, 13, 30-

29.

Fraser, P., Farrukh, C., & Gregory, M., (2003). Managing

product development collaborations – A process

maturity approach, Proceedings of the Institution of

Mechanical Engineers, 217, 11, 1499-1519.

Fraser, P., Moultrie, J., & Gregory, M., (2002). The use of

maturity models / grids as a tool in assessing product

development capability, IEEE International

Engineering Management Conference, Cambridge,

August 19–20, 2002.

Fraser, P., Moultrie, J., & Holdway, R. (2001).

Exploratory studies of a proposed design maturity

model. 8th International Product Development

Management Conference, University of Twente,

Holland, June 11-12.

Frost and Sullivan (2007). Meetings Around the World:

The Impact of Collaboration on Business

Performance. White paper available at http://news

center.verizon.com/kit/collaboration¬/MAW_WP.pdf.

(Last accessed: 3 April 2011)

Hansen, M. T., Nohria, N. (2004), How to build

collaborative advantage. MIT Sloan Management

Review, 46, 1, 22-30.

Harter, D. E., Krishman, M. S., & Slaughter, S. A. (2000).

Effects of Process Maturity on Quality, Cycle Time,

and Effort in Software Product Development.

Management Science, 46, 4, 451-466.

KMIS 2011 - International Conference on Knowledge Management and Information Sharing

Herskovic V., Pino J. A., Ochoa S. F., & Antunes P.

(2007). Evaluation Methods for Groupware Systems,

Groupware: Design, Implementation, and Use, Lecture

Notes in Computer Science, Springer Berlin /

Heidelberg, Volume 4715/2007, 328-336.

Hevner A. R., March S. T., Park J., & Ram S. (2004).

Design Science in Information Systems Research, MIS

Quarterly, 28, 1.

Hsieh, P. J., B. Lin, & C. Lin (2009). The Construction

and Application of Knowledge Navigator Model

(KNM TM): The Evaluation of Knowledge

Management Maturity. Expert Systems with

Applications, 36, 4087-4100.

Jordan, M. H., Field, H. S., & Armenakis, A. A. (2002).

The relationship of group process variables and team

performance—A team-level analysis in a field setting.

Small Group Research, 33, 121-150.

Lebrun, J., Gosset, P., Pallot, M., Roux, P., Gandelot, D.,

& Vannier, M. (1998). FREE Capability Assessment

Framework (FREE-CAF). Vol. 1: Overview,

Deliverable 132-1, ESPRIT Project 23286 FREE.

Levan, S. K. (2004). Travail collaboratif sur Internet:

Concept, méthodes et pratiques des plateaux projet,

Paris, France: Vuibert.

Macbeth, D., Ferguson, N. (1994). Partnership sourcing:

an integrated supply chain management approach,

London: Financial Times: Pitman Publishing.

Martins L. L., Gilson L. L., & Maynard, M. T. (2004).

Virtual teams: what do we know and where do we go

from here? Journal of Management, 30, 6, 805-835.

McGrath, M. E. (Ed.) (1996). Setting the PACE in Product

Development: A Guide to Product and Cycle-Time

Excellence, Butterworth-Heinemann, Oxford.

Mintzberg, H. (1979), The Structuring of Organizations,

Prentice-Hall, Englewood cliffs, NJ.

Paulk, M., Curtis, B., Chrissis, M., & Weber, C. (1993,

February). Capability maturity model for software,

Version 1.1. Technical Report CMU/SEI-93-TR-024

ESC-TR-93-177.

Pinelle D., Gutwin C. (2003, December). Task analysis for

groupware usability evaluation: Modeling shared-

workspace tasks with the mechanics of collaboration,

ACM Transactions on Computer-Human Interaction

(TOCHI), 10, 4, 281-311.

Pinsonneault, A., & Kraemer, K. L. (1997). Middle

management downsizing: An empirical investigation

of the impact of information technology. Management

Science, 43, 5, 659–679.

Ramasubbu, N., Krishnan, M. S., Kompalli, P. (2005). A

process maturity framework for managing distributed

development, IEEE Software, May/June, 80-86.

Sander, P. C., Brombacher, A. C. (2000). Analysis of

quality information flows in the product creation

process of high-volume consumer products,

International Journal Of Production Economics, 67, 1,

37-52.

A FIRST APPLICATION OF A COLLABORATION MATURITY MODEL IN THE AUTOMOTIVE INDUSTRY