Neural Networks based Software Development Effort Estimation: A

Systematic Mapping Study

Fatima Ezzahra Boujida

1 a

, Fatima Azzahra Amazal

1 b

and Ali Idri

2 c

LabSIV, Department of Computer Science, Faculty of Science, Ibn Zohr University, BP 8106, 80000 Agadir, Morocco

Software Projects Management Research Team, ENSIAS, Mohammed V University,

Madinate Al Irfane, 10100 Rabat, Morocco

Keywords:

Systematic Mapping Study, Software Development Effort Estimation, Artiﬁcial Neural Networks.

Abstract:

Developing an efﬁcient model that accurately predicts the development effort of a software project is an im-

portant task in software project management. Artiﬁcial neural networks (ANNs) are promising for building

predictive models since their ability to learn from previous data, adapt and produce more accurate results. In

this paper, we conducted a systematic mapping study of papers dealing with the estimation of software devel-

opment effort based on artiﬁcial neural networks. In total, 80 relevant studies were identiﬁed between 1993

and 2020 and classiﬁed with respect to ﬁve criteria: publication source, research approach, contribution type,

techniques used in combination with ANN models and type of the neural network used. The results showed

that, most ANN-based software development effort estimation (SDEE) studies applied the history-based eval-

uation (HE) and solution proposal (SP) approaches. Besides, the feedforward neural network was the most

frequently used ANN type among SDEE researchers. To improve the performance of ANN models, most

papers employed optimization methods such as Genetic Algorithms (GA) and Particle Swarm Optimization

(PSO) in combination with ANN models.

1 INTRODUCTION

Over the last decades, software effort estimation

techniques have seen increasing demand among re-

searchers and practitioners. The researchers working

in the software development effort estimation (SDEE)

ﬁeld are facing greater challenges in order to produce

a measurement tool that accurately estimates the de-

velopment effort. In this regard, a reliable estimate of

software effort is crucial to ensure that time and bud-

get constraints are met (Sommerville, 2010). Erro-

neous estimates can lead to situations where the soft-

ware cannot be produced on time and within the bud-

get set in the initial planning, which in turn may result

in loss of contracts (Jones, 2007).

To get accurate estimates, several SDEE models

have been developed. They can be grouped into three

main categories (de Barcelos Tronto et al., 2008):

(1) Parametric models (Boehm, 2000; Mendes, 2008)

which presume that the function expressing the Rela-

tionship between effort and software attributes has a

https://orcid.org/0000-0001-8733-0085

https://orcid.org/0000-0002-9008-656X

https://orcid.org/0000-0002-4586-4158

well-deﬁned form; (2) Machine learning (ML) mod-

els (Wen et al., 2012; Huang et al., 2008; Kumar

et al., 2008; Elish, 2009; Shepperd and Schoﬁeld,

1997; Ahmed and Muzaffar, 2009) which rely on the

use of artiﬁcial intelligence (AI) techniques such as

case-based reasoning (CBR) (Idri et al., 2015; Idri

et al., 2002a; Idri and Zahi, 2013; Idri et al., 2019;

Idri, 2002), decision trees (DT)(Idri and Elyassami,

2011), genetic algorithms (GA) and artiﬁcial neural

networks (ANN) (Idri et al., 2002b; Idri et al., 2007);

and (3) Expert judgment (Hughes, 1996) which is

purely based on the experience of one or more ex-

perts in previously completed projects to derive esti-

mates. Machine learning (ML) techniques have re-

cently received special attention from SDEE com-

munity. Wen et al. (Wen et al., 2012) conducted a

systematic literature review (SLR) on the use of ML

models in SDEE. Their SLR revealed that ANN and

CBR were the most frequently used techniques by

SDEE researchers. Further, among the eight ML tech-

niques that were identiﬁed in the study, ANN models

were the most accurate in terms of arithmetic mean of

Preds (25) and arithmetic mean MMREs (mPred(25)=

64% and mMMRE = 37%).

102

Boujida, F., Amazal, F. and Idri, A.

Neural Networks based Software Development Effort Estimation: A Systematic Mapping Study.

DOI: 10.5220/0010603701020110

In Proceedings of the 16th International Conference on Software Technologies (ICSOFT 2021), pages 102-110

ISBN: 978-989-758-523-4

ANN models have been widely used among soft-

ware researchers since they have several advantages.

First, they are able to learn from historical data (At-

tarzadeh and OW, 2014) and model complex relation-

ships between effort and cost drivers (Iwata et al.,

2010). Second, it is possible to obtain more accu-

rate estimates with the correct conﬁguration of the

weights (Huang and Chiu, 2007). Third, they can

be adapted to the environmental changes through free

parameters (synapses) (Kamlesh et al., 2019; Kaur

and Singh Salaria, 2013).

As the research area related to the use of ANN

in SDEE has deepened, the number of ANN-based

SDEE papers has increased. Therefore, it becomes

important to summarize the existing works and pro-

vide an overall view. Consequently, it is of crucial

importance to construct a classiﬁcation scheme and

structure the reported studies on ANN-based SDEE,

in order to understand and facilitate their application.

To the best of the authors’ knowledge, no system-

atic mapping study has been conducted on the use of

ANN in SDEE. Therefore, in this paper, a systematic

mapping study (SMS) is conducted to investigate the

use of artiﬁcial neural networks in SDEE. As stated

in (Kitchenham, 2010), a mapping study aims to

ﬁnd and classify primary studies in a speciﬁc thematic

area. It can be used to identify available literature.

The purpose of this SMS is to: 1) identify the

existing Neural Network-based SDEE papers pub-

lished until 2020; and 2) classify and evaluate the

selected studies with respect to ﬁve criteria: publi-

cation source, research approach, contribution type,

techniques used in combination with ANN models

and type of the ANN used.

This paper is organized as follows: Section II re-

ports the research methodology used to conduct our

SMS. Section III presents the results of the mapping

study. Section IV shows the implications for research

and practice. Conclusions and future works are pre-

sented in Section V.

2 RESEARCH METHODOLOGY

To conduct our study, we adopted the mapping

process suggested by Kitchenham and Charters

(Kitchenham, 2007). According to Ref. (Kitchen-

ham, 2007), the purpose of a mapping study is to ﬁnd

and classify primary studies related to a speciﬁc the-

matic area. This process is based on ﬁve steps: (1)

deﬁne the mapping questions,(2) conduct an exhaus-

tive search for primary studies, (3) select studies, (4)

extract data, and (5) synthesize data. The description

of each of these steps is given below.

2.1 Mapping Questions

The ﬁrst step of the mapping process consists on

deﬁning the set of the mapping questions (MQs) to

be addressed. Five MQs were deﬁned. The MQs and

their main motivations are listed in Table 1. These

MQs are related to the properties and categories pre-

sented in Table 2.

2.2 Search Strategy

The objective of this step is to identify the relevant

ANN-based SDEE papers that treat the MQs of Ta-

ble 1. To carry out the search, we used four digi-

tal libraries: IEEE Xplore, ACM Digital library, Sci-

ence Direct and Google Scholar. The IEEE, ACM

and Science Direct were chosen to provide full-text

access to the highest quality engineering and techni-

cal literature. Google Scholar was used to ﬁnd ad-

ditional relevant studies since it explores other elec-

tronic databases. Note that, these four databases were

used in our previous mapping and review studies (Idri

et al., 2015; Amazal and Idri, 2019; Idri et al., 2016a;

Idri et al., 2016b). They were also adopted by other

researchers to conduct their SMS and SLR such as

Wen et al. (Wen et al., 2012). All searches were lim-

ited to the papers published in the 1993-2020 period.

To conduct the search using the above-mentioned dig-

ital libraries, a search string was constructed. To this

end, we identiﬁed the major terms related to our MQs

as well as their synonyms and alternative spellings.

Then, we used the Boolean operators OR and AND

to join synonymous and main terms (Idri et al., 2015;

Amazal and Idri, 2019). The constructed search string

was as follows: (”neural network” OR ”ANN” OR

”MLP” OR ”multi-layer perceptron”) AND (”soft-

ware” OR ”system” OR ”application” OR ”project”)

AND (”cost” OR ”effort”) AND (estimate* OR pre-

dict*).

To make sure that all papers that address the

MQs of Table 1 were retrieved, we divided the

search process into two phases. In the ﬁrst phase, we

applied the search string on the four digital libraries

to retrieve the set of candidate studies. In the second

phase, we evaluated each of the candidate papers

using a set of inclusion and exclusion criteria to

decide whether it should be included or rejected. The

evaluation was based on title, abstract and keywords.

In case of doubt, the full text was examined. The

reference lists of all retained papers (papers that

satisfy the inclusion and exclusion criteria) were

checked to ensure that no ANN-based SDEE paper

was missed in the ﬁrst phase.

Neural Networks based Software Development Effort Estimation: A Systematic Mapping Study

103

Table 1: Mapping questions.

ID Mapping Question Motivation

MQ1 Which sources are the main targets for ANN To identify the main publication channels targeted

based SDEE papers? by ANN based SDEE studies.

MQ2 What research approaches are applied in ANN To investigate the research approaches most appli-

based SDEE papers? ed in SDEE studies using ANN models.

MQ3 In which contribution types are ANN based To discover the different contribution types of

SDEE papers classiﬁed? ANN based SDEE studies.

MQ4 What are the most frequently used techniques To identify the techniques and models that are

and models in combination with ANN based combined with ANN based SDEE models to impr-

SDEE models? ove their performance.

MQ5 What are the main types of ANNs used in To identify the most used types of ANNs in SDEE

SDEE studies? papers.

Table 2: Classiﬁcation criteria.

Research approach History-based evaluation (HE), solution proposal (SP), case study (CS), review (-

RV), survey (SV).

Contribution type Technique, comparison, validation, metric, model.

Techniques used in Optimization method (Opt), Constructive Cost Model (COCOMO),Clustering tec-

combination with ANN hniques (CT), Use Case Points (UCP), Functional Point (FP), Class Point Analys-

models is (CPA), K-Nearest Neighbors (KNN), Case Based Reasoning (CBR), Bayesian

Regularization (BR), Morphological operator (Mor), Associative Memory Techni-

que (AMT).

Neural network used Feed-Forward Neural Network (FFNN), General Regression Neural Network

(GRNN), Functional Artiﬁcial Neural Network (FLANN),Cascade Correlation

Neural Network (CCNN), Adaptive Neuro Fuzzy Inference System (ANFIS),

Wavelet Neural Network (WNN), Radial Basis Function Neural Network

(RBFNN), Recurrent Neural Networks (RNN), Resilient Back Propagation Neural

(RBPNN), Feedforward back-propagation (FBNN), Elman Neural Network

(ENN), Multi-Layer Perceptron (MLP), Hybrid Neural Network (HNN)

2.3 Study Selection

This step aims to select the relevant studies that an-

swer the mapping questions posed in Table 1. To

this end, we deﬁned a set of inclusion and exclusion

criteria to evaluate each of the candidate studies and

decide on its relevance to our SMS and determine

whether it should be included or discarded. These cri-

teria are linked by the Boolean operator OR and are

given bellow.

• Inclusion criteria

– Develop, improve or use an ANN-based model

to predict software effort.

– Evaluate and/or compare the performance of

an ANN-based SDEE model with that of other

SDEE techniques/models (e.g. linear regres-

sion, decision tree, etc.).

– Propose a hybrid model that uses a combina-

tion of neural networks and other techniques to

predict software effort.

• Exclusion criteria

– Same study with duplicate publications (only

the most complete paper is taken into account).

– Studies with focus on how to predict mainte-

nance or testing effort.

– Studies with focus on how to predict software

size or time.

– Studies estimating effort for construction

projects.

2.4 Data Extraction Strategy and

Synthesis Method

Once the studies that are relevant to our SMS have

been selected, the data necessary to address the MQs

were collected. To facilitate data extraction, we used

a data extraction form that we ﬁlled out for each of

the selected studies. Table 3 shows the data extracted

from each study. To aggregate evidence and synthe-

size the extracted data with respect to each of the

mapping questions, we used a narrative synthesis ap-

proach. Further, we used some visualization tools

ICSOFT 2021 - 16th International Conference on Software Technologies

104

such as bar graphs and pie charts to facilitate the anal-

ysis of the results.

Table 3: Data extraction form.

Data extractor

Paper identiﬁer

Author(s) name(s)

Article title

Publication year

Data checker

(MQ1) Publication source

(MQ2) Research approach (History-based evalua-

tion, solution proposal, case study, review, survey)

(MQ3) Contribution type (Technique, comparison

, validation, metric, model)

(MQ4) Techniques used in combination with

ANN

(MQ5) Type of the ANN used

3 RESULTS AND DISCUSSION

This section presents and discusses the results ob-

tained from the 80 selected studies with respect to the

ﬁve mapping questions listed in Table 1.

3.1 Overview of the Selected Studies

Figure 1 shows the results of the selection process.

Conducting the search using our search string and the

four digital databases returned 1817 candidate stud-

ies. Since many of them would not be useful to an-

swer the MQs, the inclusion and exclusion criteria

were applied to decide on their relevance to our SMS.

As motioned earlier, the evaluation of the candidate

studies was performed based on keywords, title, ab-

stract and full text. This resulted in 80 selected stud-

ies. No extra relevant papers were found by scanning

the reference lists of the selected studies.

Figure 1: Results of selection process.

The list of selected articles can be sent upon request

to researchers for further research. In addition, the

list will be available in our next systematic literature

review paper.

3.2 Publications Sources and Trends

(MQ1)

Two main publication sources were targeted by the

selected studies: journals and conferences. Speciﬁ-

cally, 46 (57.5%) papers came from journals and 34

(42.5%) papers were published in conferences. Ta-

ble 4 indicates the publication sources of the selected

papers with at least 2 papers on the use of ANN in

SDEE. Four journals were identiﬁed with 2 studies

addressing the use of ANN in SDEE: Expert Sys-

tems with Applications (ESWA), International Jour-

nal of Information Technology (IJIT), Global Jour-

nal of Computers and Technology (GJCT), and Soft-

ware Engineering Notes (SEN). Two conferences

were identiﬁed with 2 papers: International Joint

Conference on Neural Networks (IJCNN), and World

Congress on Services (SERVICES). The other publi-

cation sources were not listed in the table since they

were used only once to publish ANN-based SDEE

studies.

Table 4: Publication sources of the selected studies.

Publication venue Type # of

studies

Expert Systems with Journal 2

Applications (ESWA)

International Journal Journal 2

of Information

Technology (IJIT)

Global Journal of Journal 2

Computers and

Technology (GJCT)

Software Engineering Journal 2

Notes (SEN)

International Joint Conference 2

Conference on Neural

Networks (IJCNN)

World Congress on Conference 2

Services (SERVICES)

To investigate the publication trends of ANN-

based SDEE studies, we analyzed the number of pub-

lished papers over the years. Figure 2 shows the dis-

tribution of the number of papers over the 1993-2020

period. As can be noticed, no ANN-based SDEE pa-

per was found in 1997, 1999, 2002, 2003, and 2006.

Besides, the use of ANN in SDEE has gained research

interest between 2011 and 2019 (74% of the selected

papers).

Neural Networks based Software Development Effort Estimation: A Systematic Mapping Study

105

Figure 2: Publication trends of the selected studies.

3.3 Research Approaches (MQ2)

Five main research approaches were adopted by the

authors of the selected studies as shown in Table

5: Solution Proposal (SP), History-based Evaluation

(HE), Case Study (CS), Review (RV) and Survey

(SV). As can be seen from Table 5, , HE was the most

frequently used approach (89% of papers) followed

by SP (76% of papers). Besides, only 5% (4 out of

80) of the selected papers were reviews or surveys.

It can also be seen that, most of the selected papers

used historical datasets (89%) or Case Studies (4%)

to empirically validate their works.

When investigating the use of the HE approach in

the selected studies, we found that most papers (81%)

used historical datasets to evaluate the performance

of their proposed ANN-based SDEE model or to per-

form a comparison with other SDEE techniques. His-

torical datasets were also employed to study the effect

of some dataset properties on the prediction accuracy

of ANN-based SDEE models.

As for the datasets used to evaluate ANN-based

SDEE models, the selected papers used various

datasets with different sizes and characteristics. Fig-

ure 3 shows the datasets used as well as the number

of papers using these datasets. It can be noticed that,

COCOMO was the most frequently used dataset (27

studies) followed by Nasa (20 studies) and Desharnais

(10 studies). Of the 71 studies using the HE approach,

38 datasets were employed in 108 evaluations. Note

that, each study may perform experiments using more

than one dataset.

Figure 3: Distribution of studies using the HE research ap-

proach over the datasets.

3.4 Contribution Type (MQ3)

By analyzing the contribution types of the selected

papers, ﬁve main contribution types were identiﬁed:

Comparison, model, technique, validation and metric.

Figure 4 shows the number of studies per contribution

type. As can be seen, most studies are included in the

Comparison contribution type (87%). These studies

either compared various conﬁgurations of the same

ANN-based SDEE model or performed comparisons

with other SDEE models. Researchers were also in-

terested on developing new ANN-based SDEE mod-

els or improving existing ones (66%). Note that, no

tool was developed to estimate software effort using

ANN. This lack of ANN-based SDEE Tools may limit

the use of ANN models to estimate software effort in

industry.

Figure 4: Number of studies per contribution type.

3.5 Techniques Used in Combination

with ANN Models (MQ4)

Different techniques were used in combination with

ANN-based SDEE models to overcome a set of difﬁ-

culties related to (1) the selection of the optimal pa-

rameters; (2) the reduction of the size of the input

characteristics; and (3) the optimization of the num-

ber of neurons in the hidden layer. Figure 5 shows

the techniques used in combination with ANN mod-

els and the number of studies in which they were ap-

plied. It can be seen from Figure 5 that neural net-

works were most often combined with optimization

techniques (Opt) (15%), followed by clustering tech-

niques (CT) and COCOMO model (11% for each).

The most frequently used optimization techniques

were genetic algorithms (GA) and particle swarm op-

timization (PSO). The former was used to optimize

the number of neurons in the hidden layers, reduce

the dimensions of the set of features, or reduce the

complexity of neural network models (Bisi and Ku-

mar Goyal, 2016; Goyal and Bhatia, 2019; Tirimula

et al., 2012; K.S., 2000; Oda and Nakazato, 2016).

The latter was applied for its global classiﬁcation ca-

ICSOFT 2021 - 16th International Conference on Software Technologies

106

Table 5: Distribution of ANN-based SDEE research approaches over the years.

Research Approach 1993 - 1999 2000 - 2006 2007 - 2013 2014 - 2020 Total

HE 3 4 27 37 71

SP 4 3 23 31 61

CS 1 0 2 0 3

RV 0 0 1 2 3

SV 0 0 1 0 1

Figure 5: Techniques used in combination with ANN-based

SDEE models.

pabilities and to adjust the parameters of the adhe-

sion function (Bisi and Kumar Goyal, 2016; Tirimula

et al., 2013; Suharjito et al., 2016).

We investigated the use of CT and COCOMO in

combination with ANN models for SDEE. The main

goals for using CT and COCOMO were the follow-

ing:

• CT: (1) to aggregate datasets and make data as

normal as possible; (2) to increase the training ef-

ﬁciency of ANN models; and (3) to improve the

convergence speed of back-propagation and deal

with imprecise and uncertain data (Azath et al.,

2018; Dasheng and Shenglan, 2012; Praynlin and

Latha, 2018; Hassankashi and Hanchate, 2017;

Huang and Chiu, 2007; Kanmani et al., 2008;

Kanmani et al., 2008; Anita et al., 1998).

• COCOMO: for its characteristics and capabil-

ities such as the use of a clear deﬁnition of

the attributes for software projects (Attarzadeh

and OW, 2014; Kamlesh et al., 2019; Kaur

and Singh Salaria, 2013; Kumar and Ku-

mar, 2014; Sivakumar, 2014; Sarno et al.,

2015; Satyananda Reddy and Raju, 2009;

Satyananda Reddy and Raju, 2010; Tadayon,

2005).

3.6 Types of Neural Networks Used

(MQ5)

Various neural network models were developed in the

last years. Figure 6 shows the types of the ANN mod-

els used in the selected papers as well as the num-

ber of studies using each type. As can be seen, the

Feedforward neural network was the most frequently

used ANN type in the selected studies (36% for FFBN

and 21% for FFNN), followed by Multilayer Per-

ceptron (MLP) and Adaptive Neuro Fuzzy Inference

System(ANFIS) (15% and 9% respectively). The

widespread use of these ANN models among re-

searchers may be due to the fact that they are sim-

pler to use than other ANN models and more suited

to the problem of software effort estimation. Other

ANN types were rarely used, such as Resilient Back

Propagation Neural Network (RBPNN) (1%), wavelet

neural network (WNN) and Functional Link Artiﬁcial

Neural Network (FLANN) (2% each).

Figure 6: Number of studies per ANN type.

4 IMPLICATION FOR

RESEARCH AND PRACTICE

This study aims to present an overview of the use of

ANNs in software development effort estimation. In

this section we provide some recommendations to re-

searchers and practitioners based on the ﬁndings of

our mapping study.

This study revealed a lack of research on how

to evaluate ANN-based SDEE models in real-life

contexts. In fact, only one case study was identi-

ﬁed among the 80 ANN-based SDEE selected pa-

pers. Therefore, we recommend for researchers to

cooperate with practitioners in order to investigate

in depth the use of ANN models in industry to es-

Neural Networks based Software Development Effort Estimation: A Systematic Mapping Study

107

timate software effort. Besides, most studies used

historical datasets to evaluate or compare the per-

formance of their proposed ANN-based SDEE mod-

els. The datasets used in these studies are not large

enough to get good results. It is therefore recom-

mended for practitioners to provide researchers with

larger datasets.

No tool was developed to encourage the use of

ANN models among SDEE practitioners. This im-

plies that, researchers should develop tools that im-

plement their ANN-based SDEE models and facil-

itate their use among practitioners and researchers.

Furthermore, this study found that optimization and

clustering techniques are the most frequently used

techniques in combination with ANN-based SDEE

studies. Other techniques such as Bayesian Regular-

ization (BR) and K-Nearest Neighbors (KNN) were

rarely used. Therefore, researchers are encouraged to

conduct further research works using other techniques

in combination with ANN.

We noticed that, some types of ANNs were rarely

used, such as Resilient Back Propagation Neural Net-

works (RBPNN), wavelet neural networks (WNN)

and Functional Link Artiﬁcial Neural Net- works

(FLANN) while others were used more often such

as Feedforward neural networks, Multilayer Percep-

tron (MLP) and Adaptive Neuro Fuzzy Inference Sys-

tem (ANFIS). Therefore, it is suggested to SDEE re-

searchers to explore the use of other types of ANNs

such as Long short-term memory (LSTM) to improve

prediction accuracy.

5 CONCLUSION AND FUTURE

WORK

The aim of this systematic mapping study was to iden-

tify and classify the existing works on ANN-based

SDEE. The paper identiﬁed 80 relevant ANN-based

SDEE studies and classiﬁed them according to publi-

cation source, research approach, contribution type,

techniques used in combination with ANN models

and type of the neural network used. The main ﬁnd-

ings of our SMS are the following.

(MQ1): Journals and conferences were the main

publication sources of ANN-based SDEE papers. Be-

sides, the use of ANN in SDEE has gained research

interest between 2011 and 2019.

(MQ2): History-based evaluation was the most

frequently used research approach followed by so-

lution proposal. The use of both approaches by re-

searchers is increasing over time.

(MQ3): Most of the selected studies focused on

developing a new ANN model or evaluating and com-

paring the performance of their ANN model with

other SDEE models.

(MQ4): Optimization methods and clustering

techniques were the most frequently used techniques

in combination with artiﬁcial neural networks, fol-

lowed by COCOMO.

(MQ5): Most papers used feedforward neural

networks followed by multilayer perceptron and the

adaptive neuro fuzzy inference system.

Conducting this SMS allowed us to build a classi-

ﬁcation scheme of ANN-based SDEE research area.

However, many issues related to the performance of

ANN-based SDEE models need to be investigated in

depth.

Therefore, we see a need to systematically analyze

and summarize the evidence of ANN-based SDEE

models performance by conducting a systematic liter-

ature review that aggregates the results of SDEE stud-

ies proposing new or modiﬁed ANN models.

To this end, a systematic literature review is ongoing

to analyze the use of ANN in SDEE by taking into

consideration the ﬁndings of this SMS.

Another important issue that should be addressed

when dealing with ANN-based SDEE models con-

sists on how to interpret ANNs to gain practitioners

acceptance (Idri et al., 2002b; Idri et al., 2004; Idri

et al., 2010). In fact, ANNs are viewed as black boxes

which may prevent them from being widely used by

SDEE researches and practitioners.

REFERENCES

Ahmed, M. and Muzaffar, Z. (2009). Handling impreci-

sion and uncertainty in software development effort

prediction: a type-2 fuzzy logic based framework. In-

formation and Software Technology, 51(3):640–654.

Amazal, F. and Idri, A. (2019). ”handling of categorical

data in software development effort estimation: A sys-

tematic mapping study”. In Proceedings of the 2019

Federated Conference on Computer Science and In-

formation Systems, FedCSIS 2019, Leipzig, Germany,

September 1-4, 2019, volume 18 of Annals of Com-

puter Science and Information Systems, pages 763–

770.

Anita, L., Cheng, C., and Balakrishnan, J. (1998). Soft-

ware development cost estimation: Integrating neural

network with cluster analysis. Inf. Manag., 34(1):1–9.

Attarzadeh, I. and OW, S. (2014). Proposing an effective

artiﬁcial neural network architecture to improve the

precision of software cost estimation model. Int. J.

Softw. Eng. Knowl. Eng., 24(6):935–954.

Azath, H., Mohanapriya, M., and Rajalakshmi, S. (2018).

Software effort estimation using modiﬁed fuzzy C

means clustering and hybrid ABC-MCS optimization

in neural network. J. Intell. Syst., 29(1):251–263.

ICSOFT 2021 - 16th International Conference on Software Technologies

108

Bisi, M. and Kumar Goyal, N. (2016). Software develop-

ment efforts prediction using artiﬁcial neural network.

IET Softw., 10(3):63–71.

Boehm, B. (2000). Software cost estimation with cocomoii.

NJ: Prentice-Hall.

Dasheng, X. and Shenglan, H. (2012). Estimation of project

costs based on fuzzy neural network. In 2012 WICT

World Congress on Information and Communication

Technologies. Trivandrum, India. IEEE.

de Barcelos Tronto, I., da Silva, J. S., and Sant’Anna,

N. (2008). An investigation of artiﬁcial neural net-

works based prediction systems in software project

management. The Journal of Systems and Software,

81(3):356–367.

Elish, M. (2009). Improved estimation of software project

effort using multiple additive regression trees. Expert

Systems with Applications, 36(7):10774–10778.

Goyal, S. and Bhatia, P. (2019). Ga based dimensionality

reduction for effective software effort estimation us-

ing ann. Advances and Applications in Mathematical

Sciences, 18(8):637–649.

Hassankashi, M. and Hanchate, D. (2017). Role of ann and

fuzzy in software cost estimation. Journal of Basic

and Applied Research International, 21(1):1–9.

Huang, S. and Chiu, N. (2007). Applying fuzzy neural net-

work to estimate software development effort. Inter-

national Journal of Research on Intelligent Systems

for Real Life Complex Problems, 30(2):73–83.

Huang, S.-J., Chiu, N.-H., and Chen, L.-W. (2008). Integra-

tion of the grey relational analysis with genetic algo-

rithm for software effort estimation. European Jour-

nal of Operational Research, 188(3):898–909.

Hughes, R. (1996). Expert judgment as an estimat-

ing method. Information and Software Technology,

38:67–75.

Idri, A., Abnane, I., Hosni, M., and Abran, A. (2019). Anal-

ogy software effort estimation using ensemble KNN

imputation. In 45th Euromicro Conference on Soft-

ware Engineering and Advanced Applications, SEAA

2019, Kallithea-Chalkidiki, Greece, August 28-30,

2019, pages 228–235. IEEE.

Idri, A., Abran, A., and Khoshgoftaar, T. (2002a). Esti-

mating software project effort by analogy based on

linguistic values. In Proceedings Eighth IEEE Sym-

posium on Software Metrics, pages 21–30.

Idri, A., Amazal, F., and Abran, A. (2015). ”analogy-based

software development effort estimation: A systematic

mapping and review”. Information and Software Tech-

nology, 58.

Idri, A. and Elyassami, S. (2011). Applying fuzzy ID3

decision tree for software effort estimation. CoRR,

abs/1111.0158.

Idri, A., Hosni, M., and Abran, A. (2016a). Systematic lit-

erature review of ensemble effort estimation. J. Syst.

Softw., 118:151–175.

Idri, A., Hosni, M., and Abran, A. (2016b). ”systematic

mapping study of ensemble effort estimation”. in

Proc. 11th International Conference on Evaluation of

Novel Software Approaches to Software Engineering.

Idri, A., Khoshgoftaar, T., and Abran, A. (2002b). Can neu-

ral networks be easily interpreted in software cost esti-

mation? In 2002 IEEE World Congress on Computa-

tional Intelligence. 2002 IEEE International Confer-

ence on Fuzzy Systems. FUZZ-IEEE’02. Proceedings

(Cat. No.02CH37291), volume 2, pages 1162–1167.

Idri, A., Mbarki, S., and Abran, A. (2004). Validating

and understanding software cost estimation models

based on neural networks. In Proceedings. 2004 In-

ternational Conference on Information and Commu-

nication Technologies: From Theory to Applications,

2004., pages 433–434.

Idri, A. and Zahi, A. (2013). Software cost estimation by

classical and fuzzy analogy for web hypermedia ap-

plications: A replicated study. In Proceedings of the

2013 IEEE Symposium on Computational Intelligence

and Data Mining, CIDM 2013 - 2013 IEEE Sym-

posium Series on Computational Intelligence, SSCI

2013, pages 207–213. cited By 1.

Idri, A., Zahi, A., Mendes, E., and Zakrani, A. (2007). Soft-

ware cost estimation models using radial basis func-

tion neural networks. In Software Process and Prod-

uct Measurement, International Conference, IWSM-

Mensura 2007, Palma de Mallorca, Spain, November

5-8, 2007. Revised Papers, volume 4895 of Lecture

Notes in Computer Science, pages 21–31. Springer.

Idri, A., Zakrani, A., and Zahi, A. (2010). Design of radial

basis function neural networks for software effort es-

timation. International Journal of Computer Science,

7(3).

Idri, A. e. a. (2002). Estimating software project effort by

analogy based on linguistic values. In 8th IEEE In-

ternational Software Metrics Symposium (METRICS

2002), 4-7 June 2002, Ottawa, Canada, page 21.

IEEE Computer Society.

Iwata, K., Nakashima, T., Anan, Y., and Ishii, N. (2010).

Applying an artiﬁcial neural network to predicting

effort and errors for embedded predicting effort and

errors for embedded software development projects

software development projects. IEEJ Transactions on

Electronics, Information and Systems, 130(12):2167–

2173.

Jones, C. (2007). Estimating software costs: bringing real-

ism to estimating. In McGraw-Hill, 2nd edn.

Kamlesh, D., Varun, G., and S.D., V. (2019). Analysis and

comparison of neural network models for software de-

velopment effort estimation. J. Cases Inf. Technol.,

21(2):88–112.

Kanmani, S., Kathiravan, J., Senthil Kumar, S., and Shan-

mugam, M. (2008). Class point based effort estima-

tion of OO systems using fuzzy subtractive clustering

and artiﬁcial neural networks. In Proceeding of the 1st

Annual India Software Engineering Conference, ISEC

2008, Hyderabad, India, February 19-22, 2008, pages

141–142. ACM.

Kaur, H. and Singh Salaria, D. (2013). Bayesian regular-

ization based neural network tool for software effort

estimation. Global Journal of Computer Science and

Technology, 13(2).

Kitchenham, B. e. a. (2010). ”systematic literature re-

Neural Networks based Software Development Effort Estimation: A Systematic Mapping Study

109

views in software engineering–a tertiary study”. In

Inf. Softw. Technol.

Kitchenham, B. et Charters, S. (2007). ”guidelines for per-

forming systematic literature reviews in software en-

gineering”. Technical report.

K.S., K. (2000). Neuro-genetic prediction of software de-

velopment effort. Inf. Softw. Technol., 42(10):701–

713.

Kumar, G. and Kumar, B. (2014). Automation of soft-

ware cost estimation using neural network tech-

nique. International Journal of Computer Applica-

tions, 98(20):11–17.

Kumar, K., Ravi, V., Carr, M., and Kiran, N. (2008).

Software development cost estimation using wavelet

neural networks. Journal of Systems and Software,

81:1853–1867.

Mendes, E. (2008). The use of bayesian networks for web

effort estimation: further investigation. In Proc. 8th

Int Conf on Web Engineering, New York, pages 203–

216.

Oda, S. and Nakazato, O. (2016). An improved multilayer

perceptron artiﬁcial neural network with genetic algo-

rithm for software cost estimation. International Jour-

nal of Academic Research in Computer Engineering,

1(1):40–46.

Praynlin, E. and Latha, M. (2018). Performance analysis

of FCM based ANFIS and ELMAN neural network in

software effort estimation. Int. Arab J. Inf. Technol.,

15(1):94–102.

Sarno, R., Sidabutar, J., and Sarwosri (2015). Comparison

of different neural network architectures for software

cost estimation. In IC3INA 2015 International Con-

ference on Computer, Control, Informatics and Its Ap-

plications, Bandung, Indonesia.

Satyananda Reddy, C. and Raju, K. (2009). A concise neu-

ral network model for estimating software effort. In-

ternational Journal of Recent Trends in Engineering,

1(1).

Satyananda Reddy, C. and Raju, K. (2010). An optimal

neural network model for software effort estimation.

Int. J. of Software Engineering.

Shepperd, M. and Schoﬁeld, C. (1997). Estimating software

project effort using analogies. IEEE Transactions on

Software Engineering, 23(12):736–743.

Sivakumar, M. (2014). Enhancement of prediction accuracy

in cocomo model for software project using neural

network. In Fifth International Conference on Com-

puting, Communications and Networking Technolo-

gies (ICCCNT).

Sommerville, I. (2010). Software engineering - 9th edition.

In Addison-Wesley.

Suharjito, Nanda, S., and Soewito, B. (2016). Modeling

software effort estimation using hybrid pso-anﬁs. In

iSITIAInternational Seminar on Intelligent Technol-

ogy and Its Application.

Tadayon, N. (2005). Neural network approach for software

cost estimation. In International Symposium on In-

formation Technology: Coding and Computing (ITCC

2005), Volume 2, 4-6 April 2005, Las Vegas, Nevada,

USA, pages 815–818. IEEE Computer Society.

Tirimula, R., Chinnababu, K., Mall, R., and Satchidananda,

D. (2013). A particle swarm optimized functional

link artiﬁcial neural network (pso-ﬂann) in software

cost estimation. FICTAProceedings of the Interna-

tional Conference on Frontiers of Intelligent Comput-

ing: Theory and Applications, 199:59–66.

Tirimula, R., Satchidananda, D., Suresh, C., and Madhu-

rakshara, S. (2012). Genetic algorithm for optimiz-

ing functional link artiﬁcial neural network based soft-

ware cost estimation. In Proceedings of the Interna-

tional Conference on Information Systems Design and

Intelligent Applications (INDIA 2012), pages 75–82.

Wen, J., Li, S., Lin, Z., Huc, Y., and Huang, C. (2012). ”sys-

tematic literature review of machine learning based

software development effort estimation models”. In-

formation and Software Technology, 1(54).

ICSOFT 2021 - 16th International Conference on Software Technologies

110