The other day I joked to someone that he was suffering from "The Curse of the Portal Admin". I'm sure there is a law somewhere that states something like "no matter how much time people have to get something done, people will fill up all that time." What I mean is, that if someone is given until 2 o'clock or maybe 5 o'clock to finish something, they will usually take up all the time they have. I have seen this several times on projects over the years. During the day some members of the team are busy developing new fixes or updates to the code, meetings are held, and things are tested. Then overnight after everyone has gone home, the administrators are left to finish up what is left, deploy, test, and get the system back up and running (like little shoe building elves). Admittedly some administrative tasks have to be done in the evenings or during off hours so as not to disrupt peak usage in a production environment, however sometimes these late hours can happen before you go to production, or even in testing and QA environments after your system is live. Unfortunally this can be very wearing on administrators of the system.
Many times this occurs in organizations where development and the infrastructure team are handled by separate teams, each responsible for their own domain. I fear that this happens way too often where one team completes their work, and then throws it over the wall to another team. This first team can then relax until the second team throws something back over at them. In consulting projects where the entire team is brought together to complete a project there tends to be more collaboration across the whole team, and folks tend to have more interest in how thing are working. This is not always the case in internal projects where there are clearly defined boundaries between the infrastructure and development teams. Mostly I think it comes down to leadership (If you have read any of my posts, you know I tend to think pretty much everything comes down to leadership).
In addition to the the team issues, there can also be a constant rush to get new stuff out to the users as soon as it is developed. Obviously I am not against providing new functionality to end users, but there needs to be a trade off between releasing a new component too soon, and the risk that moving too quickly might adversely affect a production portal. In many cases waiting a few days, or doing more robust testing to be sure that the changes will not have a harmful affect, is more then worth it. I suppose you can argue that minor changes will be less risky, and I am sure that more often then not, frequent minor changes will not have a harmful effect. But even if things go well 99% of the time, are you willing to accept that 1% risk that the system might fail? Is the client or end-user who is pushing for an update willing to take that risk? I would be willing to bet that most folks change their approach after a major system failure occurs from a bad deploy.
People that know me, know that when I get going on this topic I am all about process and control. A co-worker was even teasing me recently about my process fixation around build and deployments. I guess I have lived through too many bad projects, and been fortunate to see it when things really work! I could talk about this for awhile, but I am taking a new approach to shorter and more readable blog entries. More about that later, but for now I encourage leaders to take a look at their current environment and see if this is happening in your world. Think about ways to break down barriers between teams and improve processes within your organization. I think you will find more collaboration, communication, and better teamwork with a little effort in this area.
Maybe, with a little effort and discussion we can help break the curse forever!
The Curse of the Portal Admin