Expectation vs Reality: Analyzing the Competencies of Software Testing

Teams

Nayane Maia

1,2 a

, Ana Carolina Oran

1 b

and Bruno Gadelha

1 c

Institute of Computing, Federal University of Amazonas (UFAM), CEP 69080-900, Manaus, AM, Brazil

Eldorado Research Institute, CEP 69057-000, Manaus, AM, Brazil

Keywords:

Software Testing, Soft Skill, Hard Skill, Software Engineering Competencies, Competency Assessment.

Abstract:

Process and product quality are essential to maintain competitiveness in the software industry. Software test-

ing is one of the activities to assess product quality, and its process permeates all phases of the development

process lifecycle. However, this requires software testing professionals to master different technical skills and

general skills. To reduce the skills gap required for the testing team, it is necessary to conduct a skills assess-

ment of all team members. Given this, the objective of this research is to present the results of a competence

assessment based on a competence mapping model aimed at software testing teams. To support this research,

a questionnaire was applied to 25 industry professionals from a development company to assess the general

and technical skills of all roles in the software testing process. As a result, we identiﬁed which competencies

managers desire for their testing teams. In addition, we identiﬁed how professionals see themselves through

a self-assessment based on this competence mapping model. After the evaluation, it was possible to identify

which competencies need to be developed to reach the required levels for each role and increase the produc-

tivity and quality of the process and the product.

1 INTRODUCTION

With the exponential growth of demands for software

products, the industry has been looking for increas-

ingly qualiﬁed professionals with differentiated com-

petencies. Conceptually, competencies can be deﬁned

as ”a collection of skills, aptitudes and attitudes to

solve a problem in a given context” (Holtkamp et al.,

2015). Competencies and skills are different things.

Skills tell us “what” professionals are able to do when

it comes to a speciﬁc assignment. Competencies show

the “how”, that is, answer what is the process should

be that the professional needs to go through to develop

new learning, therefore, they can be acquired through

training (Faria and de Castro Filho, 2014). In SWE-

COM (Ardis et al., 2014) these competencies can be

classiﬁed as technical skills and behavioral skills nec-

essary for software engineers.

Many studies (Assyne et al., 2021; Ahmed et al.,

2015; Holtkamp et al., 2015) seek to identify what

are the essential competencies for software profes-

https://orcid.org/0000-0003-3642-4772

https://orcid.org/0000-0002-6446-7510

https://orcid.org/0000-0001-7007-5209

sionals and it was concluded that is important to study

soft skills and hard skills, as both are considered two

equally critical pillars of the Software Engineering

Competencies (SEC). Researchers recommend that

studies be carried out considering different points of

view, from software professionals, educators to the in-

dustry, as it is necessary to pay more attention to both

contexts: research and practice (Assyne et al., 2021).

A software testing process aims to structure the

steps, activities, artifacts, roles and responsibilities of

testing, allowing organization and control of the entire

test cycle, minimizing risks and adding quality to the

software (Pressman and Maxim, 2021). In software

testing practice, successful testing often depends on

the skills, intuition and experience of the tester (Kaner

et al., 2011).

In view of this, this research aims to present an

experience report in the analysis of the competence

level of a software testing team in the industry, from

the perspective of managers (expectation) and testing

process professionals (reality). In this way, the skills

gaps that professionals need to develop to reach the

expected level are identiﬁed, enabling professional

training, reduced turnover, increased productivity and

product quality.

152

Maia, N., Oran, A. and Gadelha, B.

Expectation vs Reality: Analyzing the Competencies of Software Testing Teams.

DOI: 10.5220/0011853800003467

In Proceedings of the 25th International Conference on Enterprise Information Systems (ICEIS 2023) - Volume 2, pages 152-159

ISBN: 978-989-758-648-4; ISSN: 2184-4992

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

2 THEORETICAL FOUNDATION

This research addresses concepts related to software

engineering skill sets and knowledge described in the

Software Engineering Competency Model, available

on SWECOM (Ardis et al., 2014). SEC was also pre-

sented in a systematic mapping described in (Assyne

et al., 2021). In this mapping, 60 primary studies have

been analyzed, which sought to answer the follow-

ing research question: What can the current literature

tell us about the evolution of Software Engineering

Competencies? As a result, the following were iden-

tiﬁed: 2 research areas (organizational and personal);

11 themes inside 49 core competencies; 14 models or

frameworks – which, in turn, were classiﬁed accord-

ing to the context: 5 frameworks focused on personal

competencies and 9 frameworks focused on organi-

zational competencies. The framework described by

(Salda

na-Ramos et al., 2012), was chosen out of the

14, as it is aimed at mapping competencies of soft-

ware testing teams, the focus of this research.

2.1 Competency Mapping Model

The framework presented by (Salda

na-Ramos et al.,

2012) is used to mapping competencies focused on

technical skills and general skills required for the test-

ing team and was based on the main references on

software testing: The White Book (Llorens-Garcia

et al., 2009), Tmap® (Vroon et al., 2013), TMMi®

(van Veenendaal et al., 2022), ISTQB® (ISTQB,

2022), SWEBOK (Abran et al., 2004). Among the

different methodologies available to deﬁne a compe-

tency model, this framework (Salda

na-Ramos et al.,

2012) adopted the methodology based on Job Anal-

ysis: it identiﬁes the competencies from the charac-

teristics of the job in terms of mission in the organi-

zation, objectives operational, evaluation criteria, or-

ganizational situation, areas of activity and level of

autonomy and responsibility of the position.

The framework aims to reduce the skills gap re-

quired for the testing team according to the main re-

sponsibilities of the following roles: Test Contract

Manager (TCM): Manage resources, contract ser-

vices and control the project. Oversees the testing

process and improves it, and maintains and enforces

testing policies. Test Manager (TM): Coordinate and

manage the activities of the test team. It is responsi-

ble for ensuring the quality of products and processes,

in addition to checking that the documentation is cor-

rect. Tracks activities related to testing, monitoring

and estimating the time, schedule and costs required

to run the tests. Test Engineer (TE): Managing and

maintaining the test plan, designing and reviewing

test cases, overseeing the test environment and per-

forming reviews. Tester (T): Create and execute the

test cases, ensuring that they are executed with the

data according to the requirements and documenting

the results obtained from the execution of the tests.

The technical and general competencies and their

proﬁciency levels required for each role of the soft-

ware testing team, were identiﬁed through the com-

petency mapping framework. The competency lev-

els are: Low (L): Member of the testing team does

not have this competency; Medium (M): Test team

member is still learning. Competency has been par-

tially acquired; High (H): Team member is indepen-

dent. Competency has been fully acquired. Very

High (VH): Team member masters this skill and prac-

tices steadily. More details about the framework are

available in the supplementary material

(Maia et al.,

2023).

2.2 Related Works

Ahmed et al. (Ahmed et al., 2015) investigated which

soft skills are most appropriate for each role within a

software team and how the industry considers these

soft skills during employee hiring. The authors con-

cluded that the most critical soft skills for the role

of software testing (communication, analytical and

problem solving skills, interpersonal skills, ability to

work independently and organization) does not reﬂect

what the industry considered at the time of hiring.

This study suggests that the industry pays more at-

tention to the set of interpersonal skills and the ability

to work independently, given that the role of software

testing requires attention to details, at the same time

that it is necessary to report the results to the devel-

opers – which requires communication and conﬂict

resolution skills.

Given the above, this research aims to address

some gaps not presented in the works previously men-

tioned: (i) which soft skills and hard skills are es-

sential for the roles of the software testing team; (ii)

which soft skills and hard skills are less developed

and what action is needed to develop these skills to

improve project deliveries; (iii) what is the inﬂuence

factor of these competencies to increase the quality of

deliveries.

3 METHODOLOGY

This research was based on the competencies map-

ping framework described in Section 2.1, which char-

https://doi.org/10.6084/m9.ﬁgshare.21706037.v1

Expectation vs Reality: Analyzing the Competencies of Software Testing Teams

153

acterizes the levels of competencies according to a

set of technical competencies and general competen-

cies expected for each software testing team role. As

testing professionals, it is understood that they are all

those involved in the software testing process, accord-

ing to the roles deﬁned in Section 2.1. A question-

naire was applied to professionals from a company in

order to answer the following research questions:

RQ1. What technical competencies do the test-

ing professionals consider important for each role?

RQ2. What general competencies do testing profes-

sionals consider important for each role? RQ3. What

are the desired competencies levels for each role from

the managers’ perspective? And from the test team’s

to? RQ4. How do professionals evaluate themselves

according to the levels of technical and general com-

petencies?

3.1 Questionnaire Speciﬁcation

The questionnaire was divided into three sections: the

ﬁrst of sociodemographic characterization, with the

objective of characterizing the proﬁle of the partic-

ipants; the second assessment of the levels of skills

(general and technical) desired for each role of the

testing team; the third self-assessment of skills levels

(general and technical) according to their role within

the testing team. The concepts about the roles and

the deﬁnition of the levels of competency were made

available in electronic form to facilitate the under-

standing of the participant.

Three objective questions were asked, using a 4-

point Likert scale, where 1 means that the level of

technical competency required is Low and 4 means

that the level of competency required is Very High.

An open question was also asked in order to assess

whether the participants considered the questions sat-

isfactory. The questionnaire has a total of 77 ques-

tions that map all competencies according to each

role, that is, each competency was considered an ob-

jective question. It is worth noting that all participants

agreed to participate in the study by signing a Free

and Informed Consent Form (CF), thus guaranteeing

the conﬁdentiality of the data provided. The question-

naire was available during the ﬁrst half of May 2022.

More details about the CF and the full version of the

questionnaire are available in the supplementary ma-

terial (Maia et al., 2023).

3.2 Data Collection

The study was conducted at a company, which has

units in different regions of Brazil. The questionnaire

was applied through an online form, in a team that

works with software development, where the testing

team is allocated with the development team, adopt-

ing practices of agile methodology, working in global

software projects, and providing different contexts

within the development and the testing process.

The proﬁle of the participants was mapped ac-

cording to age, gender, level of education, position,

role and time of professional experience in the area of

software testing. A total of 25 participants answered

the questionnaire, who held the following positions:

interns (36%); technicians, trainees and test analysts

(52%); and project managers (12%). It was found that

most participants (80%) have a maximum of 5 years

of experience, 12% have 6 to 15 years of experience

and 8% have more than 15 years of experience.

In order to characterize the roles according to the

deﬁnition adopted in the competency mapping model,

it was asked which role is closest to the responsibili-

ties currently exercised by the participants. Of the 25

participants, 52% identiﬁed most with the roles de-

scribed in the Software Tester role, followed by 32%

Test Engineer, 12% Test Contract Manager and 4%

Test Manager.

4 RESULTS AND DISCUSSIONS

4.1 The Technical Competencies

The competency mapping model used offers a set of

speciﬁc technical competencies (TCs) for the testing

team. From this, the participants assessed what com-

petency level is needed for each role presented in

Section 2.1. We will highlight the results obtained

that were different from the baseline presented by the

framework.

Test Contract Manager (Figure 1:) TC1 and TC6

were evaluated with a High level, whereas the TC2,

TC3, TC4, TC5 competencies should be at Very High

level. Comparing the result obtained with the base-

line, there was divergence only in TC6 (knowledge

in business processes), where the level indicated was

Very High and the participants consider the High

level as sufﬁcient.

Test Manager (Figure 2:) the competencies that

must to be at Very High level was: TC1, TC2, TC4,

TC5, TC9, TC10; and the competencies TC3, TC6,

TC7, TC8 just need to be at the High level. The main

differences refer to the increase in the level of knowl-

edge expected for the Test Manager in the themes of

software engineering practices (TC1), project man-

agement (TC4) and technical programming knowl-

edge (TC7), which went from Medium/High to Very

High. There was a decreases in TC3 (knowing the

ICEIS 2023 - 25th International Conference on Enterprise Information Systems

154

Figure 1: Technical Competencies - Test Contract Manager.

Figure 2: Technical Compentencies - Test Manager.

organizational process), it went from Very High to

High.

Test Engineer (Figure 3:) only the TC3 competency

was considered as a High level, whilst all others were

assigned a Very High level (TC1, TC2, TC4, TC5).

When comparing the levels indicated by baseline it

was noticed that there was an increase in the level of

knowledge expected by Test Engineers in TC4 and

TC5, which went from High to Very High. However,

there was a reduction in the expected level for TC3,

which went from Very High to High, referring to the

mastery of revision techniques.

Software Tester (Figure 4:) all technical competen-

cies were evaluated by the participant with a High

level as sufﬁcient. Through the comparative analysis

of the levels indicated in baseline, it was possible to

verify that there was an increase in the level of knowl-

edge of TC4 (programming knowledge), which went

from Medium to High. However, there was a reduc-

tion in the level of knowledge expected in TC6, which

went from Very High to High, with regard to knowl-

edge about test automation.

Figure 3: Technical Compentencies - Test Engineer.

Figure 4: Technical Compentencies - Tester.

In general, there were no major divergences in the

baseline comparative analysis in relation to the results

found, when we consider the perspective of all partic-

ipants. The main divergences could be due to the fact

that currently the roles of the testing process have re-

ceived new responsibilities, increasing the degree of

complexity of their activities, such as: (i) the Test

Manager should have greater control of project man-

agement ; (ii) the Test Engineer needs to have more

knowledge about software and product engineering;

(iii) the Software Tester must have more knowledge

about programming.

4.2 The General Competencies

The competency mapping model used offers a set of

general competencies (GCs) that are applied to all test

team roles. From this, the participants assessed what

competency’s level is needed for each role.

In Figure 5 it is possible to observe that for

the role of Software Tester, this same competence

(GC19) is being considered at a Very High level,

that is, creativity is a very important competence for

the execution of the tests, as well as communication

(GC3), teamwork (GC07), being self-taught (GC9),

working in a variety of environments (GC16) and

commitment to quality (GC20). Other General Com-

petencies were considered High (GC1, GC2, GC5,

GC11, GC13) or tied with Very High (GC6, GC8,

GC10). Knowledge of the English language, moti-

Expectation vs Reality: Analyzing the Competencies of Software Testing Teams

155

Figure 5: General Compentencies - Tester.

vation, conﬂict resolution and leadership were skills

required as an Intermediate level for Testers.

According to Figure 6, the general Compenten-

cies for the role of the Test Engineer presented the

following Compentencies as a Very High level: com-

munication (GC3), teamwork (GC7), interpersonal

relationships (GC8), being self-taught (GC9), critical

thinking (GC10), problem solving (GC13), creativity

(GC19) and commitment to quality (GC20). A di-

vergence was noticed when analyzing GC12, GC14

and GC18, which was considered of High level for

Test Engineers and Low (GC12, GC14) and Medium

(GC18) for Testers. Such Compentencies, as they are

soft skills more focused on roles with greater respon-

sibilities within the team, justify these differences in

results, however, when considering a possible promo-

tion, they are important skills to be developed.

4.3 The Desired Competencies Levels

Based on the results presented in sections 4.1 and 4.2,

an analysis was carried out based on the categoriza-

Figure 6: General Compentencies - Test Engineer.

tion of respondents by roles. The responses of the

Testing Contract Manager roles were analyzed sepa-

rately in relation to the other roles, considering that

they are responsible for hiring the testing team.

The Technical Competencies desired for the Test

Engineer are: TC1 and TC2 were considered Very

High level from both sides perspective. There was a

divergence in TC3, where the managers considered it

to be a Very High level and the other team members

considered it to be a High level, on the other hand,

they believed that the TC4 and TC5 skills need to be

of a Very High level, whilst the managers considered

them only to be of a High. In this sense, it is desired

that the Test Engineer has mastery over the test pro-

cess, strategies and techniques. An important ﬁnding

when comparing baseline results with managers and

team members is that knowledge about revision tech-

niques (TC3) is considered Very High by managers

and by baseline, while members of the team consider

only High, perhaps because they don’t really know

what review techniques are about.

ICEIS 2023 - 25th International Conference on Enterprise Information Systems

156

Analyzing the General Competencies from the

perspective of managers in relation to the Test Engi-

neer, it is clear that the expectation has to be a Very

High level in the following skills: GC4, GC5, GC7,

GC9, GC10, GC11, GC13, GC15, GC16 , GC17,

GC19 and GC20. Whilst the testing team considers

just the following competencies on High level: GC4,

GC5 and GC13. In this sense, it is worth mentioning

that the managers have a very high expectation that

the Test Engineer has ﬂuency in the English language,

autonomy for problem solving and teamwork, while

the other members see the High level for such skills.

This result can be associated with the characteristics

of this organization, which operates in projects at a

global level. Analyzing another divergence, it was no-

ticed that GC3 and GC8 were considered a High level

by the managers and a Very High level by the test

team, which may mean that the test team recognizes

the importance that the role of Test Engineer has in

communicating and having a good interpersonal rela-

tionship. This is due to the fact that the Test Engineer

is in constant contact with the other members of the

software development process and acts as an interface

between the requirements and the developers.

The managers’ perspectives in relation to the test-

ing team diverged in almost all Technical Competen-

cies for the Tester, with the exception of TC4 (knowl-

edge about software development and construction),

where both considered the High level. This diver-

gence can be due to the fact that managers do not

know in depth the practical activities of testers, to

the point of identifying the complexity of each one

of them. For the testing team, all competencies of

the Tester role are considered High. When answer-

ing about TC1 and TC5, the managers themselves

did not reach a consensus on which level was ex-

pected, where each one answered Medium, High and

Very High, respectively. The TC3 and TC6 competen-

cies were considered Medium by the managers, which

may mean that there is not much expectation in the

ﬁeld of programming and automation on the part of

the testers.

Analyzing the divergences between General

Compentencies expected for the Tester, managers

want these professionals to have a Very High level in

the ability to adapt to new situations and a positive

attitude (GC15 and GC17), and High in the knowl-

edge of English and resolution of interpersonal con-

ﬂicts (GC4 and GC14). Such skills were considered

High and Medium by the team members, respectively.

In general, there was a consensus that for the role of

Tester, it is expected that they have a Very High level

in the following competencies: GC7, GC9, GC16,

GC19, and GC20. That is, the Tester is expected to

know how to work well in a team, be able to learn

alone, adapt well to different environments, be cre-

ative, and have a commitment to quality.

In addition to the objective questions about gen-

eral and technical skills, an open question was asked,

and it was possible to observe that 8% of the partic-

ipants answered that test automation is an essential

competence also for the Test Engineer and not only

for the Tester, as presented in the baseline.

4.4 Competencies Self-Assessment

This section will compare the levels of technical and

general competencies available in the baseline with

the results of self-assessments. This analysis only

considers the self-assessment of the roles of Test En-

gineers and Testers, given that the mapping of compe-

tencies aims to identify gaps between the competen-

cies desired by managers and the competencies that

professionals have in practice. The results from the

managers’ perspective have already been presented in

Section 4.3, but the gaps in the professionals’ self-

assessments will now be discussed.

Table 1: Technical Competencies - Test Engineer.

Baseline’s Framework Manager’s Expectation Self-evaluation

TC1 VH VH H

TC2 VH VH H/VR

TC3 VH VH M

TC4 H H H

TC5 H H H

Table 2: Technical Competencies - Tester.

Baseline’s Framework Manager’s Expectation Self-evaluation

TC1 H M/H/VH M

TC2 H VH M

TC3 H M M

TC4 M H M

TC5 H M/H/VH H

TC6 VH M M

In the Table 1, when comparing the baseline with

the perspectives of the managers, there were no di-

vergences regarding the desired levels of competen-

cies for the Test Engineer. When analyzing the result

of the self-evaluations, it is observed that the compe-

tencies TC4 and TC5 coincide both with the expecta-

tions of the managers and with what is presented in

the baseline, all with the High level. This means that

the managers have a good understanding of the role

and responsibilities of the Test Engineer and that the

professionals attend to what is expected of their role.

This alignment is beneﬁcial for the organization, as it

will not be necessary to carry out negotiations for the

development of such competencies. However, TC1,

TC2, and TC3 competencies need to be developed to

reach managers’ expectations, moving from the High

to the Very High level. The TC3 was divergent: while

Expectation vs Reality: Analyzing the Competencies of Software Testing Teams

157

Table 3: General Competencies – Test Engineer.

Baseline’s Framework Manager’s Expectation Self-evaluation

GC1 H H H

GC2 H H H

GC3 H H H

GC4 M VH H

GC5 M VH H

GC6 M H H

GC7 VH VH H

GC8 H H H/VH

GC9 H VH H

GC10 VH VH VH

GC11 H VH H

GC12 H H H

GC13 H VH H

GC14 M M/H/VH H

GC15 H VH H/VH

GC16 H VH H

GC17 H VH H

GC18 M H H

GC19 VH VH H

GC20 VH VH VH

the managers considered it as a Very High level, in

the self-assessment it appears with a Medium level.

Therefore, there is a need for investment in training

for Test Engineers to develop this competency fo-

cused on speciﬁc knowledge of review techniques, in

addition to improving the technical knowledge of pro-

gramming, testing techniques, and software architec-

ture.

In the Table 2, regarding the comparison of the

baseline with the perspectives of the managers, it is

noticed that there were divergences regarding the de-

sired levels of competencies for the role of Software

Tester. However, when analyzing the result of the

self-assessments, it is observed that the TC3 and TC6

competencies meet the managers’ expectations. The

expectation of the levels of TC4 and TC5 competen-

cies were not accurate, given that there was no con-

sensus on the part of the managers, however, the result

of the self-assessment coincided with the baseline,

whose level of TC4 is Medium and TC5 is High. The

TC2 competency needs to be developed to reach the

expectations of managers, moving from the Medium

to the Very High level, focusing on training testing

strategies, methodologies, and techniques for testers.

In addition, the greatest divergence was about TC6

(knowledge about test automation), which was con-

sidered Medium by both managers and testers, but

baseline suggests a Very High level. Therefore, there

is a need for a better understanding of the activities of

the Tester, to enable an alignment of expectations and

better targeting of demands.

According to the data presented in the Table 3

referring to the general skills expected for the Test

Engineer, it can be seen that there was a consen-

sus both between the data pointed out by the base-

line, as well as those obtained in this research: GC1,

GC2, GC3, GC12, GC20. There was a divergence

in the level managers’ expectations about what was

Table 4: General Competencies – Tester.

Baseline’s Framework Manager’s Expectation Self-evaluation

GC1 M H M/H

GC2 H H H

GC3 M H VH

GC4 H H H

GC5 M H M

GC6 M M/H/VH H

GC7 H VH VH

GC8 M H VH

GC9 VH VH H

GC10 M H VH

GC11 M H H

GC12 M M/H/VH H/VH

GC13 M H H

GC14 L H H

GC15 H VH H

GC16 VH VH VH

GC17 H VH VH

GC18 M M H

GC19 VH VH H

GC20 H VH VH

suggested by the baseline and with the results of

the self-assessments of the following competencies:

GC4, GC5, and GC15. That is, managers expect Test

Engineers to have a Very High level of knowledge of

the English language, information management, and

the ability to adapt to new situations, while baseline

considers such skills to be Medium to High, and par-

ticipants self-assessed at a High level. In this sense, it

is suggested that professionals develop such skills to

reach the level desired by managers.

The General Competencies for the Tester are pre-

sented in Table 4. It can be seen that there was a con-

sensus both between the data indicated by the base-

line, as well as those obtained in this research only

for the competencies: GC2 and GC4. Considering

the competencies GC7, GC11, GC10, GC13, GC14,

GC16, GC17 and GC20, the professionals already

meet the desired requirements for their role. Though

the GC3 and GC18 competencies were self-rated

above the desired, that is, they have communication

and leadership skills above what managers expect.

However, it is necessary to develop the competen-

cies GC5, GC9, GC15, and GC19, which were con-

sidered from a High to Very High level by the man-

agers, while the Testers self-assessed at a Medium or

High level. In this sense, Testers need to improve

their competencies: information management, ability

to learn, ability to adapt to new situations, and cre-

ativity.

5 LIMITATIONS AND THREATS

TO VALIDITY

This section presents the limitations and threats to the

validity of the study conducted with software industry

professionals. The main ones are: (i) applied ques-

ICEIS 2023 - 25th International Conference on Enterprise Information Systems

158

tionnaire; (ii) sample representativeness and size; (iii)

understanding of roles by the team. To minimize (i),

the questionnaire used was based on the Competency

Mapping Model already established in the (Salda

na-

Ramos et al., 2012) literature. As for (ii), because it

is a study conducted in a single software development

team, limiting the variety of roles to only 3 managers

and 22 members of the testing team, this study was

limited to only one context. And in (iii) we asked

the participants which role was closest to their main

attributions, they answered according to their percep-

tions, which can generate a bias of interpretation or

understanding of their own responsibilities. A map-

ping of the roles was carried out in relation to the po-

sitions, roles, and time of experience. A divergence

was perceived in terms of experience level and the

role answered by the participant. Therefore, the role

declared by the participant was considered.

6 CONCLUSIONS

This work presented the analysis of the competencies

levels of a testing team in an R&D Company from the

perspective of managers and team members regard-

ing all the roles involved in the testing process. As a

result, we noticed that the professionals believe they

have more competencies than those needed to carry

out their activities. Some general competencies the

participants already meet the manager’s expectation.

This research contributes to the industry by establish-

ing the competency levels expected by contractors

and comparing them with the professional’s results,

which facilitates interviews, career planning, and in-

vestment in training. For the academy, it’s important

to develop the themes that need to be improved dur-

ing graduation, forming professionals who are better

prepared for the job market. As future work, we high-

light the need to apply the framework in larger teams,

with a greater variety of roles, in addition to consid-

ering a better reﬁnement when assigning a role to the

participant, which corresponds to their position. In

addition, it is important to carry out a study of how

these skills can be developed, given that professionals

need to meet the expectations of managers, in addi-

tion to complying with the demands of the projects.

ACKNOWLEDGEMENTS

This research was supported by Eldorado Re-

search Institute and Federal University of Ama-

zonas (UFAM), CAPES - Financing Code 001,

CNPq process 314174/2020-6, FAPEAM process

062.00150/2020, and grant #2020/05191-2 S

ao Paulo

Research Foundation (FAPESP). We also thank the

USES research group for their support and practition-

ers for their voluntary participation in the study.

REFERENCES

Abran, A. et al. (2004). Swebok. Guide to the Software

Engineering Body of Knowledge.

Ahmed, F., Capretz, L. F., Bouktif, S., and Campbell, P.

(2015). Soft skills and software development: A re-

ﬂection from the software industry. arXiv preprint

arXiv:1507.06873.

Ardis, M., Fairley, D., Hilburn, T., Nidiffer, K., Towhidne-

jad, M., Willshire, M., et al. (2014). The software en-

gineering competency model (swecom). IEEE Com-

puter Society, Los Alamitos, CA.

Assyne, N., Ghanbari, H., and Pulkkinen, M. (2021). The

state of research on software engineering competen-

cies: A systematic mapping study. Journal of Systems

and Software, page 111183.

Faria, A. C. C. and de Castro Filho, C. M. (2014). Proﬁs-

sional da informac¸

ao: estudo dos egressos no estado

de s

ao paulo, mundo do trabalho, habilidades e com-

pet

encias. PontodeAcesso, 8(3):44–63.

Holtkamp, P., Jokinen, J. P., and Pawlowski, J. M. (2015).

Soft competency requirements in requirements engi-

neering, software design, implementation, and testing.

Journal of Systems and Software, 101:136–146.

ISTQB (2022). International software testing qualiﬁcations

board. Accessed: 2022-07.

Kaner, C., Bach, J., and Pettichord, B. (2011). Lessons

Learned in Software Testing: A Context-Driven Ap-

proach. John Wiley & Sons.

Llorens-Garcia, A., Llinas-Audet, X., and Sabate, F. (2009).

Professional and interpersonal skills for ict specialists.

IT professional, 11(6):23–30.

Maia, N., Oran, A., and Gadelha, B. (2023). Comple-

mentary material - Analyzing the competencies of test

teams.

Pressman, R. S. and Maxim, B. R. (2021). Engenharia de

software-9. McGraw Hill Brasil.

Salda

na-Ramos, J., Sanz-Esteban, A., Garc

ıa-Guzm

an, J.,

and Amescua, A. (2012). Design of a competence

model for testing teams. IET Software, 6(5):405–415.

van Veenendaal, E., Garousi, V., and Felderer, M. (2022).

Motivations for and beneﬁts of adopting the test ma-

turity model integration (tmmi). In International Con-

ference on Software Quality, pages 13–19. Springer.

Vroon, M., Broekman, B., Koomen, T., and van der Aalst,

L. (2013). TMap next: for result-driven testing. Uit-

geverij kleine Uil.

Expectation vs Reality: Analyzing the Competencies of Software Testing Teams

159