WebSphere Commerce Toolkit – Virtual Machine, Virtual Hard Disk and Docker

WebSphere Commerce toolkit is an essential development tool for customizing WebSphere Commerce application – to extend or override business logic, customize store flow, extend tool capabilities etc. It is built on top of Rational Application Developer which itself uses Eclipse at its core. It provides a derby database with WebSphere Commerce data model bootstrapped with WSC data and an integrated WebSphere Application Server runtime. It additionally provides plugin extensions needed to automate some of the commerce specific development tasks (JET comes to mind).

While official system requirements are found <here>, 8 to 12 GB of HDD (SDD will be nice), a fairly powerful modern CPU and 8GB of RAM is what I have come to expect – more if you want your operational activities to be zippy.  

One of the common questions asked by IT teams is how do I ensure all developers are able to setup and use toolkits that have same configuration, bootstrap data, APARs etc. Everyone is looking for quick, error free and repeatable steps for enabling developers across a team to setup their toolkit and stay up to date. In this blog I am presenting some options that I have seen or discussed, what it takes to pursue those options and call out pros and cons for each. I would like to hear about other options readers may have seen or any trouble they had encountered when pursuing these options.

Option 1: Old style trial by fire – no pain no gain

Ask every developer to install the stack in their computing device. This typically involves creating a fairly detailed document with screenshots and step-by-step instructions starting from where to download the product packages, how to run the installer, what options to choose, where to click etc. Let us face it, this is error prone. I have seen even seasoned developers miss a step and end up with a corrupt toolkit. There are always people who seem to be destined to encounter exceptions and errors more frequently than others. It is also time consuming. And if for any reason the tookit gets corrupted, it could reset the development clock.

Pros: Simple on paper – just plan toolkit setup time for each developer in your project plan. Toolkit runs on native OS – so best performance. Also, developers will need to gain expertise on setup – they will be forced to understand how things work under the hood.

Cons: Time consuming, inefficient, error prone, lot of duplicate effort wasted, backing up is tedious, troubleshooting is tedious, requires some under-the-hood knowledge.

Tips: If this is your chosen path, then you can mitigate some of the risk by creating data and configuration bootstrap packages to bring up basic environment through automated scripts instead of manual steps. For example, create a single zip file for all configuration changes needed (ACP, search configurations etc) and a single script that can copy files around, run acploads, dataload, build search index etc. Standardize the folder path and versions of packages. In fact such bootstrap script is recommended no matter which option you choose. There is also a "central repository" setup that saves individuals from having to download software from IBM Passport Advantage account and host it in your intranet.

Option 2: Traditional Virtual Machines – share with overhead burden

VM promises setup once and deploy anywhere model – and to large extent delivers on that promise with few caveats. You can have a seasoned developer or WSC administrator to create the VM package and share it with others in the team. Team can just boot up the VM and become productive. This works to most extent – but virtualization overhead needs extra processing power and memory, there may be network setup required, moving around files from VM to host and to other developers require additional hoops that one needs to jump through and there may be licensing costs for your virtualization software and the guest operating system in addition to your employees’ host OS licenses.  

Pros: Create once and deploy any number of times model, standardization is simple – just publish new VM image. Also, it allows folks running OS that does not support toolkits to run the toolkit (Mac or Linux).

Cons: Licensing overhead, execution overhead due to full virtualization – slow performance (it is slow for me even with 4GB RAM dedicated - 6GB is when it starts breathing easy), too big (~40GB). Extra steps needed to transfer around files in and out of the virtual machine.

Tips: Even here, to roll out updates, it may be simpler to publish a data or configuration load scripts and reserve publishing new VM image only for product upgrades. Beware of saved passwords of your colleagues – you may inadvertently update repositories under their proxies and cause confusion in the project. You need 4GB dedicated RAM for a VM to be functional – if it is less, it starts showing.  Interestingly, SSD drives do not help as much in speeding up VMs as it does with regular native installations – my guess is this is mainly due to VMs reserving and managing entire chunk of disk space by themselves (This is relatively speaking – it was still faster than spinning drives).

Option 3: Light weight virtualization –pseudo will do

Virtual Hard Disk is probably the most popular choice I have seen in my projects in the past 2 years. VHDs are a Microsoft virtual hard disk format that behave like a mountable drive with fixed or dynamic space, typically configured to have a max capacity. This appears as a single file until it is mounted and is typically supported in all flavors of Windows OS. For WSC toolkit, one can install all product packages needed for the toolkit into the mounted drive (taking care to ensure installation manager’s repositories and license data directories are made to point into the mounted drive) and then just distribute the .vhd file. This can be as lean as 12GB in size and can be easily shared. As long as everyone mounts with the same drive letter, this works. It gives the benefits of running directly on native OS while standardization of installations and bootstrap configurations of a virtual machine.  

Pros: Lightweight, easy to create, standardize and distribute, no extra virtualization costs or overheads

Cons: Works for Windows only, tied to the drive letter till death does it apart. Not everything works within a self contained VHD - DB2/Oracle DBs and web servers try to keep some important references in C: drives or in registry entries - so this works best for a "embedded" web server with cloudscape DB.

Tips: Drive letter management needs some foresight if you are a partner expecting to execute multiple projects and there are only 20 to play around with (assuming A, B, C, D, E and F are needed for various drives already). Installation manager keeps some information in a data config folder that defaults to C: even if you are installing the IM in another drive. This requires some tweaking of configuration to keep installation manager specific settings within the VHD mounted drive – else you may need to install some of the packages in your local computers before you can run stuff from the mounted VHD.

Note: I have been asked before – why not docker? Sure, docker works – now we have windows server containers available on docker. So theoretically one can install the toolkit and all necessary packages on such a container and share. But, we are not strictly adhering to “supported” versions as WSC developer edition is supported for professional and enterprise editions of Windows 7 only – no mention of Windows 8.1 or 10, leave out the server editions that have docker containers available. Other option is to have docker images for your DB and web server - which need work-arounds for VHD.

Option 4: Go solo – path for the brave

People who understand what they want to do with a development toolkit and don’t mind going solo – several options do exist. First a disclaimer – IBM does not recommend any of these. If you get into trouble while doing this, IBM may provide a best efforts based (kind) help but no guarantee of seeing out of any quagmire you work yourself into. Also, you may still want to use RAD if you do BOD, plugin or tool development.

Now we have docker containers for various pieces of the stack - eclipse can run in a container, so can db2 and liberty profile of WAS. Imagine this – a docker container running same DB as your QA server installations of WSC. That would make it so easy for you to extract and load all the test data for local developer use.  While doing that between *nix and Windows is “fun” (tongue in cheek), with docker, you could potential distribute same data model and loaded data to both developers and testers! Change in development environment might actually work the first time in server environment! Alright, that solves the DB. How about building source control and building projects --- these are folders that your favorite editor can be mapped to and source control scripts or plugins that can keep it in sync with a repository. If you primarily work in Stores project, then no sweat. Ok, now you can develop and have data – how about doing some unit testing. Well, how about a docker container with WebSphere Application Server running in it – and configured to connect to your database docker instance! Then your ubiquitous compile and publish tasks of Rational Application Developer needs to be replaced by a build and deploy process – similar to how it happens in a server. It is cumbersome for java classes – but it can be simple copies for JSPs and JS files.  You may also setup web server and search server containers while you are at it. All this makes it easy for you, the super developer, to manage exactly where you want to spend your resources and how you want to test it. It is fun, it can be fast and it can be daunting for not so savvy.

Pros: Fun that mechanics get taking apart a machine and piecing it together in new ways, database can be shared and exactly like server, topology can be server like, changes can be more accurately tested in local and defects more likely reproduced in local.

Cons: You need to be an expert and highly motivated, it can be daunting for not so savvy developers and standardization is piecemeal – needs a solid plan and extreme discipline – but that is similar to how configuration management and continuous integration is approached in server environments.

Tips: Works for innovation minded folks – network with some IBM folks . Your nous and your professional network are your safety nets here.

There you go folks – some options and musings around them. Hopefully got em brain cells thinking in the right direction.

**Edit1: Embedded links to installation information contextual to the blog