I originally posted this here in the membership only forum of the Integration Testing Learning Circle but realised that a more public airing would be appropriate. So here it is:
Ask “Why?” Ask “How?”
Or if you prefer the more boring title that I decided against: “Consider statements of fact in context and question them.”
This is something that I believe everyone should keep in mind when looking at any product but it is especially prudent when looking at integration testing and service virtualisation. I like to get things right which makes giving advice difficult because there’s no way of knowing whether that advice is going to be taken out of context, so I try to stick to facts but (after many years of learning the hard way) it turns out that facts are just as useless if the context is not known. So now I find myself doubting everything and asking, “Why?”; “How?”.
Vendors like to provide lists of what their products can do and in the integration testing and service virtualisation market that tends to be a list of messaging technologies; ESB platforms; protocols and message formats. But a list doesn’t tell the whole story and is dangerous to take out of context (don’t get me wrong they have their uses but…). Here’s a classic example. Customer has ESB-X and wants to automate their testing of it in isolation. They know they need to trigger the ESB somehow and they know that it dependant on other services to function (they are refreshingly current in their thinking). So they find vendors with integration testing and service virtualisation tools. All good so far, but then Customer asks each vendor, “Do you support ESB-X?” and this is where the trouble starts.
At this point everyone consults the lists and ESB-X may or may not be listed. At this point the risk for the customer is that they end up with a product that is not best suited to their needs. There is also risk for the vendor who may miss an opportunity to gain a new customer by supplying the best tool for the job. You see ESB-X may not be listed but if for example your implementation of ESB-X uses web services and JMS with XML based messages to communicate with the world then you may need to be looking for those key words. Perhaps it gets more complex and you need support for a custom protocol or message format. As a customer if the vendor says, “yes we support ESB-X”, you should ask how and enquire what benefits that gives. As a vendor you should ask the customer why they need ESB-X support and how their implementation integrates with the outside world. There are various methods to claim support for a given technology both through technical means and by adjusting testing practice. Both Customer and Vendor will be surprised what capabilities may be obscured by those lists and equally what future pains those lists may be hiding.
Of course the conversation needs to get more in-depth and this forum will include more articles investigating the details of Why and How. Please leave comments and questions to help drive future articles to expand on these (in my opinion) under-used and under-rated questions in the context of integration testing and service virtualisation.