The Complete ‘C’
Fundamental to our thinking around SOA Testing best practices are what we call the Three ‘C’s: Complete, Collaborative, and Continuous.
If you’ve heard a pitch or seen a webinar of mine surely this is not new to you. You can imagine, we have become collectors of stories of how important the three C’s are by listening to customer experiences of times when the lack of ‘C’ created vulnerability that was damaging to their business.
Here is such a story with enough of the actual details changed to protect identities. A software company testing lead related a story to me about what happened when they didn’t fully embrace the Complete ‘C’.
A new online software update site was produced for customers to buy new software and newer versions of their software. It was a SOA pilot and used web services as the middleware technology. Like so much of the conventional wisdom, they figured they would test the web UI with a record-and-playback tool and test the web services with another tool. After significant testing, they released…
Some three months later, a member of the ERP Order Management team asked this test lead ‘When are you guys going to go live with the on-line software site?’ Of course this freaked out our test lead because the site has been available for months but had apparently not generated a single invoice to a customer!
After urgent root cause analysis, it was discovered that the web service built to push orders into the ERP for invoicing queued the request but was failing to actually get the job done. Every OS patch and reboot caused them to loose all the orders that were captured!
This is the “verify” in our best practice around “invoke and verify” we talk so much about in the Complete ‘C’. We must be sure that the side affects we intend actually occur, otherwise we expose ourselves to unexpected issues that are almost impossible to detect.
This also highlights the need for Continuous testing in production. You know the issue could just as easily been that it worked during testing on the test ERP system but the production system requires account password refresh every 30 days; that fact unknown would cause the system to STOP taking orders quietly beginning our nightmare all over again.
To protect ourselves from these everyday events we need Complete testing on a Continuous basis, even in production when feasible.
Trackback URL for this post: http://www.webservices.org/trackback/id/84231