Towards a New Conceptualization of Information System Benefits

Assessment

Sylvain Goyetteand Luc Cassivi

Department of Management and Technology, Université du Québec à Montreal, Montreal, Canada

Keywords: Information System Benefits, Evaluation Process.

Abstract: Different perspectives on benefit evaluation are presented in the information technology literature, from the

perceptual assessment of benefits to the financial calculation of return on investment. This study aims to

complement the literature by integrating the IT capital expense literature and Delone and McLean’s (2003)

information systems success model. A model was developed using a qualitative approach with respondents

from three manufacturing organizations responsible for the information system evaluation process. The five-

stage model is composed of project identification, proposal development, proposal selection, IS creation/use

and organizational benefit evaluation. This conceptualization adds a new and enriched perspective to the

literature by integrating financial and perceptual benefit assessment with an organizational assessment

process. The analysis of the data collected confirmed the inefficiency of user perceptions for organizational

success assessment but also revealed top management perceptions to be a critical factor in the evaluation

process.

1 INTRODUCTION

In the last half-century, information systems (IS)

have assumed an important role in the operational

and administrative activities of organizations of all

sizes. However, the progress of information systems

is a paradox; although success stories exist, a

number of significant failures have also taken place

(Brynjolfsson, 1993). Top management in the

information technology (IT) field has identified the

inability to fully define the contribution made by IS

as one of the main challenges (IT Governance

Institute, 2004). Numerous research initiatives have

focused on explaining the relationship between IS

benefits and the improvement of IS implementation

activities (Pan and all., 2008; Chen, and all., 2009).

However, other factors such as IS selection, IS usage

and investment assessment may also explain this

phenomenon. Delone and McLean’s (Delone and

McLean, 2003)

information systems success model

is a prominent example of the use of IS selection and

usage dimensions to evaluate success.

The other research stream analyzed in this study

is based on investment assessment practices. It

originates from the financial field, where IS

investments are included in the capital expense

evaluation process. This financial view of IS success

does not have a high profile in the IS benefit

evaluation literature as very few articles have been

published on this subject (Bajaj and Bradley, 2009).

Finance researchers have developed a repertoire

of capital expense assessment practices (Bennouna,

and all, 2010; Burns and Walker, 2009), but they are

not applied in the IS success literature. However,

this research stream richly documents IS investment

evaluation through perceptual measures of benefits.

These different assessment perspectives represent

complementary approaches to explain IS’s benefits.

The combination of these two perspectives led us to

ask the following research question:

How do organizations evaluate success when

selecting and implementing an information system?

The objective of this research is to identify the

stages that an organization should follow to

adequately evaluate the success of its information

systems, from the identification of the project to the

post-implementation activities.

In the next section, the literature review presents

IS evaluation models, which leads in section 3 to the

development of a conceptual model for IT benefit

evaluation. Methodological aspects are then covered

before findings are exposed in section 5. The paper

concludes with the contributions and limitations of

this research initiative.

238

Goyette, S. and Cassivi, L.

Towards a New Conceptualization of Information System Beneﬁts Assessment.

DOI: 10.5220/0006272102380245

In Proceedings of the 19th International Conference on Enterprise Information Systems (ICEIS 2017) - Volume 2, pages 238-245

ISBN: 978-989-758-248-6

2 MEASURES TO EVALUATE IS

BENEFITS

2.1 Delone and McLean’s IS Success

Model

Research using Delone and McLean’s model focuses

on the identification and comprehension of the

elements that explain the success of IS. Their first

model was developed in 1992 but it was revised in

2003 (Delone and McLean, 1992; Delone and

McLean, 2003). This model (see figure 1) was

selected due to its predominance in the literature but

mostly because of its capacity to be transformed

from its current state to a process model.

Initial model (1992)

Revised model (2003)

Figure 1: IS success models (source: Petter et al. 2008).

The new version of the model differs from the older

one in three ways: (i) the incorporation of Intent to

Use into the Use variable, (ii) the addition of Service

Quality as an antecedent to user satisfaction and to

use/intent to use, and (iii) the combination of

Individual and Organizational Impact to form the

Net Benefits variable.

Delone and McLean (2003) grouped the model’s

variables into three categories: System creation,

System use and Consequences of system use. The

first category, System creation, measures two types

of IS-related activities. The System quality and

Information quality variables measure the

characteristics of the information system, while

Service quality measures the IS user support. The

second category, System use, comprises the User

satisfaction and Intent to use/Use variables. The

latter variable involves measuring how and how

much users apply the system’s functionalities. User

satisfaction is concerned with users’ appreciation of

the reports, websites and support provided by the IS.

It is important to note the duality of measures to

distinguish real use from appreciation of use, as

intense IS use does not guarantee user satisfaction.

The third category includes only the Net benefits

variable, which is the system’s contribution to the

success of individuals, groups, organizations, and

industrial sectors. For the sake of parsimony, this

variable was simplified, although, for some studies,

finer granularity may be appropriate (Delone and

McLean, 2003).

2.2 Evaluation of Capital Expenses

The second research field identified centers on the

evaluation of capital expenses, which is mainly

addressed in the accounting and finance literature.

As this paper examines the evaluation of IS benefits,

our analysis will be limited to capital expense

practices. In this stream of research, the literature

focuses not on IS investments alone but on capital

expenses in general. Hence, researchers analyze the

activities and tools used by practitioners in their

capital expense management processes. Burns and

Walker (2009) provide a sound synthesis of the

available documentation on the subject by

classifying 19 articles on capital expense

management practices in American organizations

between 1984 and 2008. In their classification,

Burns and Walker (2009) identified the four stages

presented in table 1: (i) Identification, (ii)

Development, (iii) Selection, and (iv) Control.

Table 1: Burns and Walker’s (2009) capital expense

management stages.

Phase Definition

Identification

(stage 1)

 Initiation of capital expense projects, in a

continuous process and for ad hoc needs

 Hierarchical level of idea generation

 Identification and understanding of a

formal idea submission process

 Identification of incentives associated

with the generation of relevant ideas.

Development

(stage 2)

 Project proposal selection and

transformation of ideas into proposals

 Data collection to justify projects

Selection

(stage 3)

 Workforce and practices to prioritize

proposals

 Project approval

Control

(stage 4)

 Post-implementation project evaluation

 Identification of incentives associated

with post-implementation evaluation

Towards a New Conceptualization of Information System Beneﬁts Assessment

239

3 CONCEPTUALIZATION OF AN

IS BENEFIT ASSESSMENT

MODEL

3.1 Developing the Model

To initiate the conceptualization of the model, the

literature on Delone and McLean’s (2003) model

was used. It should first be mentioned that Delone

and McLean’s is a variance model but the objective

of our conceptualization is to obtain a process. Thus,

we needed to return to the model descriptions in

order to transform the variables into sequential

components. Hence, our analysis grouped the

variables of the model into three sequential

components: Information system implementation, IS

use and Net benefits. For the model

conceptualization, the Net benefits component was

divided into two to reflect Delone and McLean’s

1992 model (Delone and McLean, 1992), which

distinguishes between individual and organizational

benefits. The decision to return to the previous

format of benefits measurement was based on the

fact that researchers have only succeeded in

validating the link between the model’s different

variables and the individual benefits variable.

Moreover, the financial literature used in our model

does not measure individual benefits.

Figure 2: New conceptualization of IS benefits

assessment.

Our analysis of the capital expense assessment

literature led us to adopt Burns and Walker’s (2009)

four stages as presented above. The definitions of

these four phases were then compared to the three

components (adapted from Delone and McLean),

leading to the discovery that three of the four stages

(Identification, Development and Selection) were

not covered in Delone and McLean’s models. In

fact, their models are based on measures that

characterize the information system once it has been

implemented. Burns and Walker’s last stage,

Control, was integrated into organizational benefits,

as their definitions were similar (Burns and Walker,

2009; Petter, Delone and McLean., 2008). Figure 2

presents the sequential model that integrates both

perspectives.

3.2 Defining the Model’S Stages

This section will define each of the stages identified

in the previous section. The definitions of the first

three components outlined by Burns and Walker

(2009) were retained. Hence, as table 2 shows,

Identification comprises activities associated with

the initiation and submission of projects by different

stakeholders in an organization for planned or ad

hoc capital expenses. Development covers the

selection of ideas for projects and the transformation

of these ideas into concrete proposals requiring

elements of justification to feed the next stage

(Selection). The third stage, Selection, includes the

analysis of the different quantitative and qualitative

justification elements and the project approvals to be

conducted by the organization.

As mentioned previously, the next three stages of

the model originate from Delone and McLean’s

(2003) work. The definition of the IS

implementation stage is the same as Delone and

McLean’s, which includes System quality,

Information quality, and Service support quality.

The IS use stage groups two of Delone and

McLean’s variables: Use of IS and User satisfaction.

Finally, the approaches to establish the benefits

are different. At the individual level, the user’s

absolute appreciation of the system is measured,

whereas at the organizational level, an improvement

is required compared to the initial situation (old or

no IS in place). Furthermore, Delone and McLean’s

original 1992 configuration directly relates the

Organizational benefits variable to Individual

benefits (Delone and McLean, 1992). A distinction

is therefore essential as the Individual benefits

component is important but insufficient to explain

the Organizational benefits. Optimal use of a system

is possible without making a significant contribution

at the organizational level.

The Individual benefits component was therefore

defined based on Delone and McLean’s (1992,

2003) most commonly used validation measures

(Perception of usefulness, Perception of success,

ICEIS 2017 - 19th International Conference on Enterprise Information Systems

240

Processing speed/delay, Improved decision making,

Quality/ accuracy of the output). For the

Organizational benefits component, only the

elements associated with capital expenses were used,

as the literature arising from Delone and McLean’s

model did not demonstrate a significant relationship

with this variable.

Table 2: Definition of the stages of the IS benefit

assessment model.

Stages Elements of definition Source

Identification

 Project initiation

 Project submissions

Burns and

Walker (2009)

Development

 Selection of ideas

 Project justification

Burns and

Walker (2009)

Selection

 Quantitative and

qualitative analyses

 Project approval

Burns and

Walker (2009)

Information

systems

creation

 System quality

 Information quality

 Service support

quality

Delone and

McLean

(2003)

IS use

 System use

 User satisfaction

Delone and

McLean

(2003)

Individual

benefits

 Perception of

usefulness

 Perception of success

 Processing

speed/delay

 Improved decision

making

 Quality/accuracy of

output

Delone and

McLean

(2003)

Organizational

benefits

 Quantitative analysis

 Qualitative analysis

Burns and

Walker (2009)

4 RESEARCH METHODOLOGY

The use of a new conceptualization, combined with

the limited literature on IT capital expense practices

and the unclear distinction between the phenomenon

and the context, justifies the case study approach

(Yin, 1994). This methodological approach enables

researchers to retain the holistic, meaningful

characteristics of real-life events (Yin, 1994). The

unit of analysis in this study is the process, which

also makes the case study approach appropriate to

collect data.

A five-step methodology was followed in this

research initiative. Organizations and respondents

were selected and sampling was done at both levels.

Selection criteria were defined to ensure adequate

information quality and to validate the subsequent

research results (Patton, 2002). Data collection was

then conducted via semi-structured interviews and

document analysis to guarantee triangulation of the

data (Yin, 1994). All interviews were recorded and

transcribed.

In the third step, data analysis, narrative and

graphical representations of the process were

created. An example of a process (organization B) is

presented in Appendix 1. A mixed interpretation

strategy was used at this step to analyze each case

individually (Langley, 1999). To identify similarities

and differences in the process and develop a process

model, a cross-case data analysis was then

conducted so we could understand and validate the

process applied by the organizations (Eisenhardt,

1989). Finally, to validate and understand the results

of the research, interviews were conducted with the

respondents from each company involved

(Eisenhardt, 1989).

4.1 Description of the Cases

Three organizations in different sectors with annual

capital expenses between $5 million and $50 million

were selected to conduct this research.

Organization A, which employs more than 6,000

people in the aeronautic sector, with service points

and manufacturing sites in America, Europe and

Asia, has a $25-million to $50-million IT capital

expense budget. Respondent A, director of global

infrastructures, supervises the IT capital expense

evaluation process, from the initiation of IT projects

to their completion. The physical infrastructure and

applications to support activities are the main

elements of the IT capital expenses.

Organization B is a manufacturing company that

employs 3,000 people at six sites in Canada. It has

an IT capital budget of $25 million to $50 million

and is controlled by an American conglomerate that

has establishments in 30 countries. The role of

Respondent B, vice-president of IT, is to supervise

the entire IT capital expense evaluation process and

to ensure the respect of corporate IT policies for all

worldwide IT projects. IT capital expenses in

Organization B are centered on physical

infrastructure and applications to support

transactional, administrative and logistic activities at

the different manufacturing sites.

Organization C, a large manufacturing firm with

30,000 employees worldwide (North and Central

America, Europe and Asia), has an IT capital budget

of between $5 million and $10 million. Respondent

C, vice-president of information technology/business

Towards a New Conceptualization of Information System Beneﬁts Assessment

241

applications, is responsible for the activities related

to the implementation of new applications and

transformation of existing applications. IT capital

expenses concentrate on infrastructure investments,

network technologies and applications to support

administrative and transactional activities

worldwide.

5 FINDINGS AND MAIN

RESULTS

In this section, the IS benefit assessment model

presented previously (figure 2 and table 2) is

compared with the processes followed in the three

organizations and particularly with the common

routines of the different organizations’ processes.

The result of this analysis is a five-stage process

presented in table 3 and described in the following

sections.

Table 3: Comparison of the stages of the IT capital

expenses process.

Activities

Organiza-

tion A

Organiza-

tion B

Organiza-

tion C

1.Project

identification

Identification

of initiatives

Project

identificat-

ion

Project

identification

2.Proposal

development

(included in

next stage)

Project

analysis

Development

of project

summaries

3.Proposal

selection

Proposal

selection

Proposal

selection

Proposal

selection

4.IS creation

/use

Project

implementa-

tion

Project

implementa-

tion

Project

implementa-

tion

5. Individual

benefits

- - -

6.Organizational

benefits

Project

closure

Ad hoc

analysis (top

management

perception)

Ad hoc

analysis (top

management

perception)

5.1 Project Identification

When comparing the three organizations’

assessment processes, the first element involved the

project identification plan; all organizations had

activities leading up to the identification of IT

projects. There are few such activities and all are

included in this first stage of the IT capital expense

evaluation process.

5.2 Proposal Development

This stage, which is defined as the activities that

establish the nature and impact of IT projects, is also

found in all three organizations. A specific proposal

development stage exists in Organizations B and C.

In Organization A, proposal development is included

in the selection stage.

The results for this stage were categorized in

three parts: descriptive project information, impact

analyses and stakeholders concerned by the proposal

development processes.

All three organizations document project-specific

information during the IT capital expense process

The elements used to describe IT proposals are very

similar in all three. A total of eight elements were

found for this specific part of the stage: brief project

description, client identification, link with corporate

strategy, internal resource evaluation, external cost

evaluation, operational cost assessment, relationship

with other projects, and calendar.

Two kinds of impact analyses are carried out:

qualitative analysis and quantitative analysis. All

three organizations conduct qualitative analyses on

proposals by listing the potential benefits for the

organization. The three organizations use different

quantitative return on investment tools. However,

the respondents indicated that these quantitative

analyses faced major hurdles linked to specific

benefit assessment in a project portfolio context and

also to the evaluation of the opportunity cost related

to technology upgrades. For instance, Respondent B

mentioned:

There is a project analysis that is done in terms

of cost and benefits. […]Establishing a cost for a

project at a global level and establishing the

benefits as well – it is not always obvious.

Along with the IT group, operational and

administrative groups are generally involved in the

proposal development stage. Proposals that originate

from operational and/or administrative groups are

generally business-oriented but require support from

IT. Proposals from the IT group are usually related

to the improvement and maintenance of the IT

infrastructure.

5.3 Proposal Selection

All three processes include a proposal selection

stage. In this stage, the selection process identifies

the proposals that justify the annual IT capital

expense budget, as indicated by Respondent B:

ICEIS 2017 - 19th International Conference on Enterprise Information Systems

242

We have preliminary evaluations of different

projects for costs and for benefits. All of these

projects are then moved into a group of projects

that are IT and non-IT. A committee looks at

them, categorizes the most important things and

approves an annual list of projects.

Two specific elements stood out from the

analysis of the different interviews – proposal

prioritization and IT capital budget allocation –

which are described below.

Proposal Prioritization

The group responsible for proposal development

in Organization A also has the responsibility to

conduct the prioritization exercise, which is then

validated by top management. In Organizations B

and C, prioritization is the job of top management.

To better understand the logic behind this choice,

Organizations B and C were analyzed in detail.

In Organization B, the set of proposals for the

organization is selected, as the respondent explained:

The vice-presidents will for sure look at the

big projects, and all of the other small

projects ahead of them may be decided by

individuals as well. Therefore, the VPs might

decide on the ten biggest projects, and in

Canada, there are not only IT projects.

There are several types of projects in our

company. And IT, it’s just one component

among others […] So, at a certain time, all

these projects will collide. We created a

committee that tries to manage the ten

largest global and local projects.

These comments from Respondent B also show

that top management prioritizes the most important

projects; less important projects are prioritized in the

organization’s departments. Respondent C

mentioned that the prioritization process is more a

question of maturity than a prioritization strategy

choice:

We don’t have measures that move forward

from one project to another. Right now, we

use the measures that we are able to obtain.

I mean that, for one project, there are

certain things that we are able to measure,

and for other projects, we measure

something else. Then yes, when comes the

time to prioritize, sometimes it is a bit

difficult because we’re comparing apples to

oranges, but I can see that there may be a

tendency emerging for which, with the new

management team, we will try to set more

global criteria. We recently talked about

standardizing projects so that we could have

a more common base to evaluate projects

more easily when we compare them.

Considering these two perspectives, the selection

of a prioritization strategy seems to be based on the

scale and coverage of the prioritization process

along with the level of maturity of selection

activities.

IT Capital Budget Allocation

As described above, Organization B allocates its

IT capital budget during a global capital allocation

exercise. For the other two organizations, a

preliminary global capital budget is allocated before

the selection process since targets are defined

specifically for IT capital expenses. Respondent A

explained how the activity is carried out in that

organization:

Usually, the firm will try to keep a standard

level because, for most companies, the IT side

is an overhead cost. So, everything is charged

back to the production groups, and we see if

there is an increase directly linked to the

manufactured product. Then, we try to have

something more stable for that.

5.4 IS Creation/Use

It was no surprise to find that all three organizations

consider project implementation to be a crucial stage

in their IT project assessment process. However, the

organizations do not distinguish between creation

and use in their evaluation process since project

implementation practices always involve activities

associated with system use. As this research

initiative did not have the objective of exhaustively

analyzing implementation practices, these activities

were not studied in depth.

During the discussions of IT project

implementation, all the respondents mentioned the

existence of a Project Management Office (PMO) in

their organization to control their projects.

According to the data collected, a PMO is necessary

for IT project management, as Respondents B and C

stated:

We have the concepts of business partner,

project link and project manager. We have a

structure that is not deployed in the rest of the

organization, but I think that we won’t have a

choice about adhering to a specific

methodology, because projects are becoming

more and more complex and because there are

more and more functions. Plus, the

Towards a New Conceptualization of Information System Beneﬁts Assessment

243

stakeholders are both internal and external

because we outsourced certain functions. It’s

the coordination of these stakeholders that

makes the PMO inevitable in my view. I was

very skeptical myself when we created the

PMO. (Respondent B)

There is another position that we created

three years ago, Head of the PMO. Under

this person, project managers operate in a

matrix, but they really made a difference at

the execution level. Before, we didn’t have

these things, but now we develop project

budgeting management practices. We put

project schedules in place based on effort.

We also put performance measures in place

for these projects at a performance index

level and a cost level.. (Respondent C).

5.5 Individual Benefit Evaluation

After we analyzed the data, it was clear that user

benefits and individual evaluation were not

mentioned by any of the respondents. This fact was

confirmed during the validation interviews, as the

respondents did not consider user perspectives

appropriate for evaluating IS benefits at the

organizational level. The respondents justified this

approach by the negative reaction of individuals to

change. Users react more strongly to the impact of

the technology on their own tasks than to the impact

on the organization. The respondents presumed that

top managers have a better feel for the overall

situation, which enables them to identify the

advantages after the adaptation period. This stage is

therefore not included in the model.

5.6 Organizational Benefit Evaluation

The literature on capital expense assessment

practices mentions that few capital investment

projects undergo post-project analyses. Our results

demonstrated the absence of systematic post-project

validation of pre-implementation evaluations in all

three organizations. However, evaluation

mechanisms are present in two of the three

organizations, which are based on top management

perception, the nature of the IT projects and other

success criteria. These elements are presented in the

following paragraphs.

First, the evaluation and control activities

identified in Organization B are executed only if top

management has doubts about the success of a

project. Respondent B highlighted this particular

finding:

When political questions emerge for certain

deliverables, analyses are carried out.

We therefore presume that, in these

organizations, IT projects are a success if top

management seems satisfied with the solution

implemented. Perception of IT performance seems

to be the most important IT success factor, as

Respondent C stated:

If you did it and it works, OK, nobody says

anything […] but if it does not work, then

you’ll hear about it.

Organization A decided to set up IT success

evaluation mechanisms by identifying success

criteria other than the ones identified during the pre-

implementation analyses. Hence, Organization A

identified a set of tangible success criteria in order to

take the nature of projects into account. Respondent

A mentioned:

I have a goal, but what are my success

criteria? What will tell me that I succeeded in

that, and that I was successful with my

project? Do you know that I delivered 1,500

telephones, that I updated everybody to PCs

that are less than four years old? What are my

success criteria?

Pre-implementation evaluations do not seem to

be aligned with the post-implementation evaluations.

During our validation interviews, we questioned the

respondents on the reasons for this incoherence

between the measures used before and after the

projects. Overall, we noticed that the executives’

lack of motivation and willingness to measure the

success of IS mainly explains this incoherence.

6 CONTRIBUTIONS AND

FUTURE RESEARCH

6.1 Contributions

The process model developed during this study

contributes to the evolution of the IT benefit

evaluation field by combining the literature capital

expense and IT benefits. The model, which displays

how organizations evaluate success when selecting

and implementing an information system, also has

practical implications as it identifies the best IT/IS

assessment practices that management of

organizations can use to better assess their

information systems.

The model also explains the validation problem

ICEIS 2017 - 19th International Conference on Enterprise Information Systems

244

identified by Petter, Delone and McLean (2008) by

confirming that end users do not accurately perceive

the impact of IS use on an organization, which

suggests that management’s perception should be

used instead to capture this impact.

6.2 Research Limitations

The first limitation on our research is a result of the

qualitative approach we used, which we chose due to

the richly detailed information it provides. This

choice led to a sampling strategy using just a few

organizations. The conclusions of this research

might be different with a larger number of

organizations, but our methodological approach and

the importance of the identified routines allowed us

to achieve the desired semantic and theoretical

saturation.

During the data analysis, the two data analysis

strategies we used to reach our research objective

also involved the limitations identified by Langley

(Langley, 1999). With the narrative strategy, the

richness of the data presented prevents the

development of a simple or generalized theory. This

explains why we combined a narrative strategy with

a graphical visualization strategy, which simplifies

the interview data in order to generate a sequential

model. This combined strategy makes it impossible

to identify factors that influence the process’s

activities or to predict the presence (or absence) of

certain activities.

6.3 Future Research Avenues

A first avenue of research may be the development

of a theoretical model, as we limited our literature

review to Delone and McLean’s (2003)

model and

to IT capital expense evaluation practices. It would

be relevant to explore the literature in other related

fields such as IT productivity or project

management.

Hence, the conceptualization of the proposed

model could be improved by increasing our

understanding of the stages of the model or by

identifying new ones.

A quantitative validation of the model also

represents a natural research avenue since the

qualitative approach limits the generalizability of the

results. A quantitative approach could quantify the

importance of the different components of the

model, which we were not able to do in this study.

Finally, our analysis of the organizational benefit

evaluation component demonstrates the absence of

validation of ex-ante assessments after IT use. This

situation also seems to give rise to new initiatives to

measure IT project success after implementation. A

study to identify the obstacles to post-project

evaluation could be developed to understand the

reasons for the lack of evaluations, but also to

identify obstacles and measures used by the few

organizations that do carry out post-project

evaluations.

REFERENCES

Bajaj, A., and W. Bradley. 2009. Linking System Analysis

Outputs to Capital Budgeting Measures: A

Framework and Case Study Validation, Journal of

Information Science and Technology, 6(1), p. 3-24.

Bennouna, K., G. G. Meredith and T. Marchant. 2010.

Improved Capital Budgeting Decision Making:

Evidence from Canada”, Management Decision,

48(2), p. 225-247.

Brynjolfsson, E. 1993. The Productivity Paradox of

Information Technology: Review and Assessment.

Communications of the ACM, December.

Burns, M. B., and J. Walker. 2009. Capital Budgeting

Surveys: The Future Is Now, Journal of Applied

Finance, 1-2, p. 78-90.

Chen, C. C., C. C. H. Law and S. C. Yang. 2009.

Managing ERP Implementation Failure: A Project

Management Perspective, IEEE Transactions on

Engineering Management, 56(1), p. 157-170.

Delone, W. H., and E. R. McLean. 1992. Information

Systems Success: The Quest for the Dependent Variable,

Information System Research, 3(1), p. 60-95.

Delone, W. H., and E. R. McLean. 2003. The Delone &

McLean Information Systems Success: A Ten-Year

Update, Journal of Management Information Systems,

19(4), p. 9-30.

Eisenhardt, K. M. 1989. Building Theories from Case

Study Research, The Academy of Management

Review, 14(4), p. 532-550.

IT Governance Institute. 2004. IT Governance Global

Status Report. Rolling Meadows, IL, USA.

Langley, A. 1999. Strategies for Theorizing from Process

Data, The Academy of Management Review, 24(4), p.

691-710.

Pan, G., R. Hackney and S.L. Pan. 2008. Information

System Implementation Failure: Insights from Prism,

International Journal of Information Management, 28,

p. 259-269.

Patton, M. Q. 2002. Qualitative Research and Evaluation

Methods, 3rd ed. Thousand Oaks, CA: Sage Publications.

Petter, S., W. H. Delone and E.R. McLean. 2008.

“Measuring Systems Success: Models, Dimensions,

Measures and Interrelationships”, European Journal of

Information Systems, 17, p. 236-263.

Yin, R. K. 1994 Case Study Research: Design and

Methods, 2nd ed. Thousand Oaks, CA: Sage

Publications.

Towards a New Conceptualization of Information System Beneﬁts Assessment

245