WebServices dot org

The Big WS-Difference

We here at iTKO say all the time that there’s a big difference between web services and full SOA testing.

What do we mean by that? Usually we get that goofy blank stare because so many people equate web services with SOA and so from a testing point of view they equate the testing of web services with the testing of SOA.

While it is true that a number of initiatives for doing SOA are very web services centric, Aberdeen’s last research on this points out that only about 50% of the SOA initiatives at large companies are web services based. There are a variety of technologies being used to create that commoditized middleware for SOA. While we love web services for this, other technologies are very valid, possibly better for a given organization than a web services stack. I’ll maybe spend some time blogging on that topic later.

The biggest difference between web services testing and full SOA testing, even for a web services shop, is the concept of testing implementation and side-effects as opposed to just the middleware layer.

Let me give you a scenario:
Let’s say that you are trying to test the functionality of a web service that encapsulates two or three different systems, and provides one façade for those three systems back to a potential service consumer -- this is a really good use of web services. So, you have a web services testing tool invoke web services and get payloads back in the form of SOAP XML docs. Now let’s say you get something back that doesn’t look right. Where’s the problem?

• Is it in the web services stack?
  

• Is it in the implementation of that wrapper?
  

• Is it in the implementation of one of the three underlying services?
  

• Or, is it in the interaction among those three components, (four when you include the web service itself)?
  

Without complete heterogeneous testing of all of the components involved in the SOA, you’re never going to know the root cause of the problem. That is the huge distinction between the ability to test a real SOA implementation vs. the ability to do WSDL/SOAP testing of Web Services alone.

Here’s an example of how this plays out in the real world:

I was with a prospect a few months ago and going down a path somewhat similar to this with them, when someone just jumped right out of the blue and said:
“You know I had a situation just like that -- I’m a tester and I brought a SOAP document to one of my developers and said: Hey, I believe that this proves that there’s something wrong with the service.”

While that kind of brush-off might not be warranted, the developer has a point. Our tendency is to suspect the validity of the indirect vs. proof that is directly connected to the implementation. In order to really help that developer see what’s going on in his or her own component, we needed to see a test of their implementation directly.

We’ve got to test as close to the implementation as possible because that’s where the bug is. We don’t want to test indirectly. Other moving parts of the environment will cloud our visibility into the root cause and tend to blame the wrong things. It sounds like UI testing all over again only instead of a user interface, we’re using a web services interface that hides the business logic as it is programmed.

Hopefully I’ve given you some insight on one of the many differences in fact there are between web services testing and SOA testing.