Choosing an Integration Solution – Where do you Start?
We live in a world that has gone crazy with Categorization. In our attempts to simplify life, we have organized ourselves into a category and acronym nightmare.
In the wonderful world of Middleware, there are dozens of categories of products from which to choose and it can be difficult to figure out what it all means. Gartner alone lists 36 separate product categories in their Research Report “Who's Who in Middleware, 1Q04” [1] - 22 of which are specifically “Integration Middleware”. No wonder the market is confused.
I feel sorry for all the CIO’s and IT managers who are faced with a myriad of product categories (each with its own lovely acronym) to choose from to solve their integration problems. The list can be daunting (AIM, ESB, EII, EDI, ETL, WSB, WSC…). And you can be sure, that middleware vendors will espouse all the virtues of their specific product category and present it as the silver bullet that will solve any business problem even remotely related to integration.
Are product categories a bad thing? Not all all…but be careful not to start with them when looking for an integration solution. If you do, you risk missing products which offer features that span categories or offer functionality which hasn’t be categorized yet.
My advice?
"Don’t start your search for an integration solution by researching all the categories. Not only are you likely to miss one, but you’ve put the cart before the horse. What matters is how you are going to solve your business problem in the fastest and most cost effective way possible. So start at the beginning – define the problem. Too many people (and you know who you are) start with the latest hyped product category recommended by their favorite analyst and then look for a problem to solve with it. Don’t believe me? Take a look at your IT lab – if you have a skunk works project happening right now, then I rest my case.
"
So what should you do?
"Now we’re back to project management 101….Start with describing your problem in detail. Analyze and measure the pain, costs, time and complexity of your current situation. Then define what you want for the future (some business process re-engineering methodologies call this the “to be”). Don’t limit your vision of what “could be” by your assumptions of what categories and products are available. I guarantee that this will reduce your potential for great returns. Instead, take your vision and convert it into a detailed list of requirements that you can use to objectively evaluate what the market has to offer. Don’t forget to include the total cost of ownership (TCO) and your return on investment (ROI) needs. The business needs must drive your technology choices.
"
Only after you’ve clearly defined your technical and business requirements is it time to look at the market to see what products can deliver. It won’t take you long to find many products that can solve your technical requirements. That’s the easy part. And those products will span multiple categories, so don’t limit yourself to any one in particular.
Now the hard part starts – evaluating products against criteria which directly impact your business, such as:
Complexity
Take a look at the infrastructure that is required to for the product to work as promised. Does it need run on an application server? Does it need a separate web server, a database, an orchestration or workflow engine, a separate development environment, etc? How can you tell? If a product requires you to “call” it by using some special mechanism apart from the web-based protocols, then it is an incomplete Web Services stack and probably either assumes you already have a Web server or offers to sell you one.
Many products today either assume or include infrastructure components that are not focused on Web services or even integration. Often these are the large, “all you can eat” systems that have evolved over many years with layers upon layers of code. They weren’t designed from inception to solve your integration problems with open standards. They just try to be all things to all people. The result…they are extremely complicated to configure, costly to maintain, and nearly impossible to secure. They introduce complexity and risk to your environment and often require highly specialized experts just to maintain them.
Play Well with Others – Future Proof your Freedom of Choice
Be careful of heading down the slippery slope of vendor lock-in. What may look like a way to homogenize your environment to reduce complexity could result in your inability to choose the best and most cost effective solutions for future projects. This often leads to “vendor consultant lock-in” which can bleed you dry. As one analyst put it so eloquently, “Once you start using consultants, you can’t seem to stop – it’s like being addicted to drugs.”
So here are few suggestions:
"Make sure the product seamlessly integrates with whatever infrastructure you have in place today and in the future. One obvious question you should ask is, “Does it run on all the major operating systems and hardware platforms?”
"
"Choose a product that allows your existing IT team to build and maintain your applications as much as possible. Be afraid, be very very afraid of vendors who make a lot of money on consulting services versus license revenue. If you really feel you need to hire a consultant to help you with something specific, make sure you define the scope of the work carefully so that they create a reusable “widget” that allows you to support it going forward. If you don’t, you’ll end up needing a 12 step program to rid yourself of them down the road.
"
"Ensure the product truly complies with open standards WITHOUT proprietary extensions. I urge you to ask the vendor for sample XML or XSLT output from their system and have them prove it will run within a competitor’s product or an open source solution. You’ll be surprised how many don’t.
"
Built-in Connectivity
In almost all integration projects, you will need to access external (possibly remote) data and processes. Look for products that already have, or support, connectors for your backend systems. Make sure the product comes with support for both structured (e.g. relational and non-relational databases) and unstructured data (MS Office files, text files, etc) without tying you to a specific architecture (e.g. JCA, COM or .NET). And for those obscure proprietary systems (we all have them), look for a product that provides easy to use templates to help you create your own connectors quickly and affordably without the need to hire expensive experts in Java or .Net. Watch out for products that only offer “sample code” – that’s just another way of saying that you (or high priced consultants) will be doing a lot of coding.
Some connectors come with additional license charges, including the hidden cost of user licenses for the servers that they access. You will need to ask specific questions about these details, or they’ll get missed. Don’t assume anything!
Security
It’s the number one concern of most CIO’s these days and will be a critical issue in an integration project that crosses corporate firewalls. Ensure that your vendor is tracking security-related standards initiatives and commits to provide support for these standards as they become relevant to your needs. Also keep in mind that the more tightly integrated the various servers are that provide your solution, the easier they will be to secure. A single product that provides a full Web services stack in an integrated environment is likely to be the easiest to secure, but only if the vendor has built security into their product in an intuitive way.
Intelligent Workflow
Workflow is the heart and soul of any server that attempts to automate manual processes. Make sure the product you choose comes with an intelligent workflow engine with:
"
Logic and Business Rules: A rich workflow engine needs to have functionality that a programming language has, including the ability to loop, do conditional branching (if), launch parallel tasks, etc. Ideally, the workflow engine should support and encourage the reuse of logic between multiple projects. In many cases, long-running transactions may be required. i.e. In some cases, the transaction may have to “wait” for a manager to approve an over-limit transaction or a partner agreement, before the transaction can complete. If you have this type of need, make sure the product you choose will support this.
Parallel Execution and Dependencies: Many legacy systems or internet connected partner systems have a high latency. Therefore it is critical to be able to launch a series of activities in parallel, and minimize delay.
Exceptions: A good workflow engine should have a mechanism to deal with exceptions such as notifications of server suspension, resume and restart. It should also allow for handling exceptions where human interaction is required (e.g. when an event occurs that requires a manager to approve something before proceeding to the next step). The Server must be able to suspend the application and resume it when the appropriate action has taken place.
"
References
Of course, every company has unique requirements, so it is imperative to ask potential vendors specific questions about how their product can solve your specific business needs. Don’t believe everything you read – just because they have a documented case study, ask to speak to the customer directly. And don’t forget to talk about customer support after the product was delivered. What’s their training program like? Are they willing to let you sit in on a session? How many layers of customer service do you have to go through to get someone who really knows that they are talking about?
Don’t be afraid to request a Proof of Concept
When you narrow down your selection to 2 or 3 candidates that you think can meet your needs, don’t be afraid to ask for a proof of concept from them. Give them a challenge and watch how they perform in terms of quality, service, performance, etc. Suddenly, your decision will become a “no brainer”.
These are just a few of the things you should consider when looking for an integration solution. Obviously, each project will need to create its own checklist of things to consider. Just make sure that list is driven by your business needs, not media hype. Believe me, as you go through the exercise of identifying and documenting your requirements, you will come across some good integration products that may not be on the analysts’ or media’s current radar. You’ll also begin to better understand the standards and technologies that are important that you will want to track for future reference. And lastly, you just may begin to understand what the categories are all about!
[1] R-22-2153: B. Lheureux, R. Schulte, Y. Natis, D.McCoy, B. Gassman, J. Sinur, J.Thompson, M. Pezzini, F. Kenney, T., Friedman, M. Gilbert, G. Phifer
Trackback URL for this post: http://www.webservices.org/trackback/id/5751