Me, myself and WebSphere
Ok, so I should probably say something about the Pure System as well... During the Opening session at IMPACT they rolled this cabinet onto the stage in a puff of smoke and techno music booming... Well, kind of corny I felt and not knowing much about the Pure Systems prior to seeing it up on stage I got a flash-back to some 80's movie or something...
It would have been way cooler if they'd had implemented a HAL-9000 eye on the box!
While listening on the presentation though it got me wondering if they shouldn't have gone even further with the smoke and flashing lights... The box is way cool!
Imagine scaling down that arduous implementation project whit not half, not 60, nor 70%, but I'd say up to 95% of the time it would take you to install and set it up in a normal "server" environment, such as blade servers and regular network equipment.
First of all you'll get pre-packaged deal to bring home, e.g. DB2+WAS, etc. I guess they stole that idea from Cast-Iron... ;)
Honestly I am not that impressed by the whole "packaging" deal, that is something that should have happened years ago and for me personally being spoiled with DataPower and firmware based appliances it's "every day business". I do understand the benefits though as I spent a 2 month (!) project installing WebSphere Partner Gateway with SAN file-share running on DB2 and WAS messing around with load-balancing, fail-over scripts and IP addresses flying here and there... but now I am getting ahead of my self here because the PureSystems will do all that for you!
Yes, that is load-balancing, fail-over, disk arrays, and all those flying IP addresses while you sit back relax and just monitor the whole shaboom in a cozy Web GUI. Now that is the deal-breaker there!
Then looking at the sad part, who would buy it... Well, without a doubt there are many, many companies that would benefit greatly from having it but that would mean tossing out a lot of already installed and running software... I think that will be the main problem pushing these beasts onto the market, unfortunately as I think ROI would be quite good, even though it would mean you have to toss a few pieces out. You would also get a much lesser foot-print on the network equipment and much shorter time-to-market on pretty much anything you put on the box.
If IBM can show these numbers and the benefits of using the PureSystems in one "package" I think they will have a much easier task of getting these boxes out. The tricky part will be to actually get these numbers as they vary greatly between different companies and depending on if you have your network stuff outsource or not.
If you are lucky enough to find a company that needs a new set of hardware and new installs, well, these boxes are for them!
They couldn't do any better!
But then again, how many of those are you going to find...?
Ok, back in Las Vegas for another week of IBM technology... IMPACT is really great but this year it is rather hot outside so it is a good thing IBM keeps me busy inside...
It is three days into the the conference and I have now had my own speaker session about the implementation of DataPower XB60 and WTX for ESAB (http://www.esab.com/) and also finalized my IBM Champion interview so now I can relax and really enjoy the rest of the week without freaking out about my public appearances... ;)
What's new and exciting then?
Well, for starters those breakfast egg-cheddar cheese-bagels are quite alright... ;)
But there are a lot more goodies coming our way. Unfortunately IBM has forced me to sign a few NDA's (that's None Disclosure Agreement) so I can not share those releases with you only the public ones (for now at least...).
The most exciting for me (and my clients) would be the release of WebSphere Appliance Management Center (WAMC) as part of the support-pack for DataPower rather than a licensed (read: costly) product. The new version 5 of WAMC also have a lot of improvements and a much smaller foot-print on the running server. If you are running a multi-box environment you should really look into WAMC 5; the "multi" part don't have to be "multi" in the sense of production-production to gain benefits from WAMC but can also be a production-test setup. WAMC 5 will be GA (Generally Available) by the end of June...
Further good news is the announcement of the WTX EDI-pack support on DataPower firmware 5 (coming end June as well along with WTX 184.108.40.206 where this is included). This means that we can finally use the EDI type-trees and the compliance maps on DataPower. This comes from IBM allowing us to use the RUN() function.
For all you old Mercator people (like me) and Launcher users this means that we can start using those router-maps again. This also shows that if you nag the right people long enough things will change for the better... ;)
With this release we will also see the need for the WTX test domain go away and instead be supported out-of-the-box in DataPower, so you only need to open a port, give the IP address in Design Studio and start test running maps on the DataPower box directly. I assume this will also be the case for deploying (=uploading) the compiled maps from Design Studio, even though that was not mentioned during the session...
There has also been some talk during open sessions about new ideas when it comes to the appliance form-factor. My belief is that we will probably see much smarter solutions for connectivity for cloud and/or gateway services coming along in the form of a "connectivity" DataPower. This would be awesome for any gateway solution where you would run the server side in the trusted (intranet) zone while having a "receiver" as proxy in the DMZ, e.g. Sterling Integrator for B2B traffic. This would also mean that we would have a secure and flexible route to our trusted domain from any Cloud solution out there.
This can of course be done today using a regular DataPower, e.g. XG45, but I see that the "connectivity" model would be more limited and therefore (hopefully) less expensive.
Talking about the XG45; that's a new improved XS40 still in a 1U shape but with added protocol handling and now being able to run WTX maps as well. This clearly indicates the focus IBM has on DataPower and the willingness to listen to the demands from the market. Previously the XS40 was quite limited and you had to go that extra mile (read: show the money) to get the protocol support needed for every day use in the XI50.
Now with the new supported protocols and functions in the XG45 while considering the added hardware in XI52 you get a distinct separation in use cases even though many of the XI50 functions have been ported to XG45. The choice for a smaller customer with need only for a secure (and smart) gateway for the DMZ the XG45 is now the clear choice!
Continuing with more of the cool stuff is the IBM Workload Deployer, the snippet below explains it in more detail (stolen from IBM.com):
"IBM Workload Deployer is a hardware appliance that provides access to IBM middleware virtual images and patterns to easily, quickly and repeatedly create application environments that can be securely deployed and managed in a private cloud."
This is essentially the only thing needed for your perfect "transformation-as-a-service" solution as you can throw WTX Launcher on there and have it scale up and down dependent on your needs. I would really like to get my hands on one of these...
For anyone wondering about the DataPower GUI I can only tell you things are lightening up for us... more than that I can not reveal, sorry... (remember those NDA's?)
Ok, so the follow-up question to my post "Why SOA?" seems to be "SOA, an Executive Overview?"...
I really have a hard time explaining SOA and the benefits of it to the executive layer in the company. They don't really grasp the technology, nor do they care about technology, so how do you explain the benefits of SOA so they'd understand it without getting into technology?
Metaphors has done the trick for me, and here let me tell you a story that won't be technical at all, so all tech-nerds can jump off this train right now.
Imagine some time ago, as far back as the beginning of the last century (that is the 1900's if you lost track) where people lived spread out and managed their own farms and villages. There where not much need for integration as most of the work was done on the farm and you only occasionally had to visit the village to pick up some supplies and maybe trade for other goods. (Metaphor: Systems/Application ran isolated and where thinly spread out).
Then people started to gather together in villages which in some occasions became towns... This presented a whole new type of problems, first they realized that we need someone in charge (Metaphor: SOA Governance). Once they had someone that could make decisions (hopefully through a democratic process) for the greater mass they could start putting some structure to things. The "goverment" (read: governance) created rules for how to build your house, where and how roads would connect these houses. The also built centers, like a town-hall, which would be easily accessible to the public and that could spread information to all citizens. (Metaphor: Point-to-point integration and a broadcast model).
This was working out great until the town grew too big and the planning didn't take into account the widespread need for building houses (i.e. adding systems and applications). The planning committee needed to address this and make up for shortcomings such as deliveries not finding the right address and that the fire-wagon (yes, they used wagons in that time) couldn't get through crowded streets or would be turning into a street too narrow to get through. (Metaphor: delivery failures, performance issues and too narrow channels/bandwidth to handle the traffic).
This was maybe not too much of a concern to the "government" and their closest allies (read ERP and/or financial systems) who had a great time in the city center and the fire-wagon parked in a shed close by.
What they came to realize though was that the more houses that burnt to the ground and the more deliveries being lost the less business the town made which of course hit the town economy quite hard after some time. We can only hope that the "government" got wind of this in time before things got really bad...
Anyhow, as soon as they realized that this was not good for the town (or its economy) they started to think of how to overcome these issues. As they had gotten into a mess of roads leading in all different directions and had delivery routes going criss-cross between shops and customers as well as letting foreign customers in from different routes they had a hard time trying to figure out where to start. Someone (who probably had read SOA for dummies) came up with the idea to build a main-street, leading from one end of the town to the other. This of course improved things and even though it lead to tearing down and rebuilding a few house it came out a good investment in the end (Metaphor: Hub-and-spoke integration).
There where, however, still many houses in the outskirts that didn't benefit at all from this new main-street and some started complaining that they where left out while some other of these outskirt houses where very happy because they did not benefit from the main-street and only saw all that buzz and commotion as something negative and stressful.
These happy outskirt houses knew where they had to go for their business and had no intention of letting the "government" know what they where doing. This was especially true for the houses dealing with those foreigners roaming through town as they could conceal their traffic and benefit from not having to follow the rules and build whatever solutions they liked to accommodate the foreigners. The foreigners had no way of letting the "government" know they where unhappy or had to wait outside for several hours to find a vacant room. Eventually some got tired of waiting and gave up while others just threw in their load and carried on their journey without caring if someone ever picked it up. (Metaphor: Indifferent B2B handling and bad monitoring).
It took quite some time for the "government" to realize this "outskirts" business as they only saw what was closest, their brand new and shiny main-street and how happy their closest acquaintances where (remember those ERP and financial systems?). The "outskirt-ers" weren't too happy about being up for a closer inspection and reported back that all is well and good and that business is thriving. This is where the story normally ends (unfortunately) but some business keeps their wits and takes a closer look at their complete business area. If they do this, which is one of my main work assignments; the B2B Workshop by Enfo Zystems, we get into a new set of architecture and design of the town...
So, now they stand in front of a huge undertaking... How are they going to clear this out and manage to get control and a working governance again?
Clearly something has gone wrong since the renegades in the outskirts area has been able to roam free and create a mess. Thinking about it, their initial idea with the main-street was not bad, not bad at all actually, but they would have needed to extend it and made sure that all could have accessed it directly. That's a very clever idea they might think but it's very difficult getting every one on board and having them turn onto the main-street all the time, plus it makes a very long street, right?
Well, if it's long doesn't really matter since we travel as fast as an electron through a copper wire (normally a CAT5 Ethernet cable) so we really don't need to take the length into account, just stretch it and bend it as much as you like (yes, electrons can bend too...).
We know have decided to build a long stretch of road with entrances and exits for each and every house, reminds me a bit of a free-way, doesn't it? ;)
Now imagine this free-way with those entrances and exits extending out into the suburbs and also covering the outskirts. This means that all those foreigners also have to start their trip into town on the free-way. That, in turn, means that we could place a toll-booth over the free-way and monitor all that passes through!
Now let's extend that to toll-booths on all exits and entrances on and off the free-way, now that's a great idea because now we can keep track of who is on the freeway and where they came from.
This, along with some great and up-to-date maps, which are now easy to maintain, gives us a full overview of our town (read: integration platform).
This is really exiting to the "government" as they realized they can also add labels to each vehicle (read: message) traveling on the free-way. They soon also realize that these labels could include the worth of each traveler helping them to know exactly how much of their belongings is going up and down that road. That way they will be able to tell which travelers on the road that is bringing home the money!
Let's also add some emergency-phones if some of those vehicles get lost or breaks down for some reason, that way we know where they are and where they came from and how to help them (Metaphor: help-desk and real-time monitoring).
And just imagine how easy it would be to add a house or remove a house or rebuild a house... Just tell the free-way planning committee about your plans and how they should redirect traffic during that time and handle your ramp when you are back in business.
They still see a few issues though as some of those foreigners don't speak the language... The "government" comes up with the idea that someone who understands both languages can translate so have the foreigner drive off the freeway into a "translation center" (read transformation service) to explain their business to one of the natives and then have that native drive over to the receiver with it instead...
Now, that is a better town layout, isn't it, and doesn't it also remind you a bit about the SOA idea...?
Why SOA you might ask?
Well, it's really to reap the benefits from re-usability. Although I find that it becomes more and more common that the whole SOA "idea" is questioned, especially by "higher-upp'ers" who mostly see the added cost for their particular project.
I have realized that I, instead of explaining the benefits of SOA, more and more often am trying to persuade project managers or executives in the business area to at least talk to their Enterprise Architects on the SOA matter instead of just go with the "easy way out".
Why has it come to this...?
I would think that SOA is nowadays counted as "old-news" or the hype that never happened, and since it was only a hype, why should We spend any more money on it?
I would definitely agree that not all projects, or even companies, do benefit from a full blown SOA implementation but I think most could at least benefit partly. The main issue is really that most companies aren't looking at the big picture because most changes are driven by projects that only affects part of the business and/or integration platform.
If a company has already invested in modern "integration tools" such as WebSphere Message Broker (WMB), WebSphere MQ and MQFTE they are not very willing to continue investing in some "real" ESB tooling. Sure you can use WMB as a very effective ESB but that would mean that you have to build a lot of the technology in-house and what it really becomes is a hub-and-spoke integration platform with great routing logic...
To really benefit from SOA you need to look into the whole "Service" concept, and by that I don't mean Services from a technical point of view but from a conceptual. So, then, what is a conceptual point of view?
The first and most important thing would be to map out your IT environment, and not only the projects but the whole company's environment, and categorize the different areas. Look at what will have to remain as the old "spaghetti" or point-to-point and what can be "moved" into the "Services" view. And before you start complaining; every, yes, every, mid- to large-sized company have their share of "old school" integrations that can't be moved for one or another reason. The important thing here is not to "service:fy" them but to know that they exist and what they do!
Next start putting your integrations into a service pattern, or function reference if you will. You don't need actual "services", such as a WSDL or similar for each function, just to describe them as a service with what they do, where they are, what's the input and what's the output.
You will soon see that many of the "services" that you map are very similar and that you will have an easier task while moving forward. Once all services (on the conceptual level) are documented you can start looking at how they do interact with one another.
Regardless of the technology used to "interact" you should put it down as a "service call" and link consumers and providers of the service along with the point-of-delivery (POD). With this new documentation I can promise you that you will start making smarter choices when it comes to integration and that you can benefit from a lot of your already existing solutions, and by that I don't mean the services themselves, but the technology, code or structure of the existing integrations. Simply put; you will be able to see how you (or that other project) have created things previously and benefit from using that same pattern, and who knows, maybe even share a service! ;)
Now that is real SOA to me...
Offline 0600018ME9 1.257 Visitas
Please note that this is a very “minimum” configuration and setup, although good enough to put into production but you should of course consider backup, fail-over, testing and of course documentation as well.
I would however like you to note that this is absolutely doable and there is nothing “crazy” or strange, the tools are just this good and state-of-the-art.
So, let’s get the clock started and myself down to the server-room!
The first 10 minutes or so I will spend unboxing the IBM DataPower XB62 appliance. If you are wondering, yes, that is included in the five hours!
Just to let you know how much of a nerd I am; I always get a severely raised pulse just opening up that DataPower box and pulling out that appliance; new shiny and full of that tremendous power… The geek-o-meter couldn’t get any higher than that!
It is generally better to run down to the server room yourself than having the network-techies doing it for you. Handing a DataPower appliance to a network technician is just begging for delays… Once they realize what kind of hardware they have gotten their hands on they will spend days planning IP stack routing tables, ARP cache’s, default routes and all those other very confusing terms they use when they want to sound busy
Next up is the initialization of the XB62. That is a fairly easy task if you know the parameters needed, such as IP address, DNS server and have decided a good and strong password prior to firing up the initialization wizard.
To run the wizard I hook my computer running Putty (the SSH client) up to the Serial port on the front of the appliance using a USB-to-serial converter.
Putty will start showing some scrambled text and after the device has booted the wizard will run. After typing in all the details I restart the XB62 to make sure the new details will stick so that I don’t have to run back down again…
I always test the appliance using the configured Ethernet port through a small network hub so that I know the IP and net-mask is fine. Before running the connection test I ran the CLI command to start the WebGUI of the appliance so that I can see it when running hooked up to the box directly.
That done I can get out if that freezing server room and try to get my core-temperature back above freezing. Downing a quad-espresso on the go up the stairs normally helps a bit. (Well, yes, even though I am an IT geek of the worst kind I do take the stairs and not the elevator…).
Back at my desk I fire up my FireFox browser and open the WebGUI for the XB62 running on the default port 9090 to make sure that it works over the network.
When I have gotten this far I can forget about the DataPower box for a while. I will then start developing the WebSphere Transformation Extender (WTX) map to transform the data from EDIFACT to XML.
The message of choice for my project is an EDIFACT D.93A ORDERS message which I will transform into ebXML Order.
Here you might say that I am “cheating” to some extent as I will be using the pre-built Baseline EDIFACT WTX pack from the Baseline EDIFACT Startkit for the EDIFACT message format and the XML schema file for the ebXML Order.
The message formats in WTX are represented by something called a TypeTree. The TypeTree’s are then placed in an input-card, reading the EDIFACT and an output-card, writing the XML.
I then start creating the actual mapping between the two formats. As EDIFACT is a quite complex format to map and it requires a lot of nested objects, called functional maps in WTX I am going to spend the major part of the five hours in WTX.
I am not going to bore you with any more details about the actual mapping but after close to 4 hours the map is built and tested to make sure that at least three different EDIFACT D.93A ORDERS run through fine.
In the Baseline EDIFACT Startkit there are five pre-built maps included in the pack and I can guarantee you that those are way more thoroughly tested than the map I just built, I have to keep in mind that I am supposed to finish this within five hours, right?
Once the map is tested I change the WTX runtime of it to DataPower and save the .dpa file on my drive for later use.
At this point I have spent a bit short of 4½ hours of the five and that including the de-icing and espressos, which there has been two more of…
Now back again to the Web-GUI of the XB62!
I will trade with a partner that runs AS2 so I am going to setup the “Internal” partner, i.e. my own company which also sometimes is referred to as “Hubowner”. For simplicities sake I just am going to call my trading partner for “Partner” and my own company “Hubowner”.
Setting up trading partners in the XB62 is a walk in the park while eating a piece of cake; yes, that simple!
Before we start typing away on the partner setup I am going to create a new “Application domain”. An Application domain is like a “partition” and is isolated from any other domain. You should never ever create any objects in the default domain!
The well thought through Web-GUI makes it a no brainer (for a trained person) to remember what to put where for the partners so I type in the name, the identifier of the partners and then scoot over to the AS settings where I have to create the Crypto objects for the AS2 exchange as it should use signed messages with a MDN (Message Disposition Notification).
The certificates has previously been exchanged so all the keys and password for the private one is already laying there waiting on my hard drive. Uploading them to the DataPower box is strangely enough normally the most time-consuming task in creating the partner! :o
After the crypto’s are created I head on to the destinations where I add a HTTPS destination for the partner. My own backside connection will go to an IBM MQ Queue Manager (which has been pre-configured) so I have to head over to the Network objects of the Web-GUI to create the Queue Manager object.
Knowing the IP, channel and name for the Queue Manager the task is a breeze and I am back in the Hubowner partner setup in half a minute or so. Continuing on the Hubowner setup I set the destination to a DataPower MQ client connection typing the MQ connection URL by heart (but there is a wizard in case you need it).
Now that the two partners are configured I need to “pair” them in a B2B Gateway. Again the GUI helps us creating this in a few minutes only, selecting which partners to be included in the gateway and setting up the Front Side Handler (FSH). The FSH is the listener receiving the AS2 message from the partner. In this case we need an AS2 FSH and we need to specify the IP address and port on which it should listen.
The B2B Gateway need to know how I want to handle the B2B data and to set that I move over to the Archive tab of the B2B Gateway and set it to “Purge only”, meaning it won’t save the messages going through. Make sure you are not “purging only” in your production environment!
For the B2B gateway to run we must set the “B2B Persistence” store. This must be done from the default domain so I am going to jump back to there and select the onboard disk (raid0) for the store.
Swapping back to my “partner domain” it is now actually ready for a “pass-through” test. This is a very good idea since if the message flows through we know that the partners are created correctly and that the XB62 is up and running and all ports have been opened by the network team.
Knowing that I now can pass the message from “Partner” through to “Hubowner” means that I not only can identify the partners but also put the message un-enveloped from AS2 onto the backside MQ queue. The only slight issue now is that the message is put on the queue as EDIFACT and not as the expected XML message.
Remember that .dpa file; the WTX map you know?
It’s time to put that in now and that is done as a transformation action in a processing policy. I am going to put the processing policy on the Hubowner in this case as it will be a generic map used by several partners.
The processing policy is created as a graphical flow of the message. In the action for transformation “Transform binary” is an option and when choosing that I will be presented with the option to upload a .dpa file.
There is nothing more to adding the transformation than that, as I will use a “match all” rule for this processing policy meaning that all messages passing through will get transformed.
That’s it really, there’s nothing more to it… and the time?
Well, I was poking around in the XB62 GUI for another half an hour which puts the total to just under 5 hours where almost 4 was spent building the map.
This means that if you would only want a secure and fast performing B2B Gateway without any transformation I would be able to set that up for you in about an hour and a half…
For the more critical reviewer you might want to consider the security aspect of things since I bypassed the network team here, right?
Well… All DataPower appliances comes with all security features activated by default so just starting it up and plugging it in means you have all the security you ever going to need!
B2B Architect at Enfo Zystems, Sweden
Offline 0600018ME9 497 Visitas
I have been working within the Integration and B2B
(Business-To-Business) scope for some years now. First with IBM
WebSphere Transformation Extender (WTX, former Mercator) on to using IBM
WebSphere Partner Gateway (WPG) and now focusing more on IBM DataPower
and the XB60/62 in particular.