
 
consider deployment variables, for instance, is to 
only partially address the issues involved even in 
testing only a Web Service, and as such we believe 
that only a holistic approach can achieve this. Tool 
vendors, such as Parasoft and Empirix (Parsoft 
2009, Empirix 2009) have started to address the 
issues of SOA testing. Persisting problems of 
solutions proposed by tool vendors are, in our 
opinion, either a lack of integration with open-
source IDEs, or too much integration; tying 
developers into one IDE for all development work. 
Although for example, Parasoft’s approach is 
relatively exhaustive, again there is a focus on Web 
Service testing only; BPEL testing is not covered. 
The reason may be that a lack of maturity of the 
technology coupled with the problems associated 
with working with constantly evolving standards and 
protocols make testing tool development difficult. 
In this paper, we propose a testing framework for 
SOA that will provide a holistic approach. 
Successful testing needs to have as near to total 
coverage as possible of all the artefacts that are 
produced in the development of an SOA system. 
This framework will be the basis for combined 
automated and methodological support in testing 
such a system. 
Most recent development trends incorporate 
some level of unit testing and test first approaches. 
Therefore any framework would need to be aligned 
with the test-driven development process. The 
framework will need to make testing an integral part 
of the development. One reason that test-driven 
development process has become so widely accepted 
is that in addition to the natural integration of the 
development and testing, there is provision for 
automation support for unit testing. With continuous 
integration, as requested by most agile processes 
(Beck 2000, Cockburn 2001), automation of unit 
tests is a must. The framework has therefore to 
provide and identify the expected level of 
automation support 
2  THE AGILE APPROACH TO 
TESTING  
Traditionally testing has been done separately from 
the development work, by a separate team. Systems 
today tend to be developed using iterative 
development methodologies, which make the 
traditional model of testing costly, and increasingly 
ineffective. Agile approaches to testing have strong 
links with SOA, coming, as they do, from a heavily 
OO-influenced sector. The separate services that the 
SOA artefact utilises can be viewed in much the 
same way as a software package, module, or 
component, and consequently, we feel, should reap 
the same benefits from an agile approach. It is 
interesting to note how well the technique of test-
first coding, now considered to belong to the realm 
of smaller projects using agile development 
methods, lends itself to an SOA approach. The most 
likely reason for this is that although the domain is a 
vast canvas, breaking the domain into service 
components implies a modular approach, similar to a 
number of smaller projects. The overall management 
of the project can only be undertaken with 
safeguards, and in particular tests, in place at the 
service level. 
Such a comprehensive approach to testing may 
not be a problem in what is viewed as the ‘target 
audience’ for SOA. Much discussion about SOA 
concerns the reuse and exposure of Legacy 
applications as services. These are systems which 
already work. The difficulty in SOA-enabling them 
is in wrapping and exposing them via a WSDL 
specification, and then orchestrating that via BPEL. 
Hopefully, the general programming logic has been 
cleared of most (if not all) bugs.   
Developing an SOA application from the ground 
up, there are many more factors to consider, and 
therefore a more comprehensive approach to testing 
is required. It is vital to monitor and constantly 
assess the health of all newly written artefacts. 
Presumably in any reasonably-sized development 
there will be teams taking responsibility for different 
services, for the exposure via WSDL, and for the 
orchestration. Even so, those teams need to ensure 
they have testing that covers the entire range of 
functionality for their code. All these parts feed into 
the orchestration process to identify the bugs with a 
minimum of effort and frustration. 
Test design, where possible, needs to be 
addressed at an early a stage as possible; ideally at 
the modelling stage. The automation of some of the 
development activities means that it is not always 
possible to do this. For example, WSDL creation 
must wait until XML schemas have been generated. 
This means it is not possible to create full test cases 
at the modelling stage.  
In summary, there is a parallel between SOA 
development and agile development processes. The 
need for comprehensive testing in the development 
of ground-up web services, should, we feel, be able 
to be adequately addressed by an agile, unit-test-
driven approach. This approach, however, must be 
ICSOFT 2009 - 4th International Conference on Software and Data Technologies
54