Millard 060001WYSU 611 Views
I chose this profession on purpose -- and in a moment of desperation.
I had started college as a journalism major after a successful run as editor of my high school’s newspaper (which won its first state-wide awards under my direction). I liked doing it, was successful at it and was attending a real School of Journalism at a decent university. And things were going well. I started questioning my choice as journalists were starting to be jailed (in the United States, believe it or not) for refusing to disclose their sources. I wondered if I could protect some criminal source at the cost of being ripped from my family and imprisoned for an unknown period of time. Then a visiting professor who had taught my news editing and headline writing class told me at the end of the semester that “you really have a career in this if it’s what you want to do”.
Chalk it up to being young and inspired to become a member of the fourth estate -- I had not really thought of it as a choice up to that moment. It was just the path I was on and what I (thought I) was meant to do. Sitting here in 2012 with many newspapers dead or dying and many news organizations generally in difficult times (or not really “news” organizations any more), I’m glad it was a choice (and one that I could remake).
As that young, inspired journalist I knew that I’d be called upon to write about all sorts of things and it would help if I knew something about them. Having grown up in a rural area in the 1970s, I knew almost nothing about computers so I decided to learn about programming during my freshman semester. I loved the class and finished at the top of it, ticked it off as “knowledge acquired” and moved on. Until that fateful December when I realized I needed to choose a different major. As I cast about my brief life’s experiences looking for something that I enjoyed doing and might support me (and the family I didn’t have yet), I recalled that programming class. Could I really get paid for something that just seemed fun to me? Would someone pay me to solve technical puzzles and build software? [Those sound like crazy questions now, but back before personal computers and the Internet and smart phones the answers were not as obvious.]
Thus began my journey to this moment, this Sunday morning in early 2012, with decades of solving technical puzzles and building software behind me (and hopefully many more in front of me). Knowing that my career was a choice and I made it well. Waking up most mornings wanting to go to work despite having done it until bed time the night before. Using those skills to fuel hobbies and community activities and house and feed and support my family.
Also at this moment, I find myself in Hillsboro, Oregon working at the IBM Lab in Beaverton helping develop the next version of IBM Rational Team Concert -- a tool that helps people and companies (including me, my team and IBM) build software better. Given how I feel about my career choice, I can’t imagine a better job -- building software that helps people build software (and getting to use that software myself for that purpose).
Thanks for reading this first post. Hopefully you’ll find value in the ones that follow. I expect to write a bit about software development in general and sometimes about RTC in particular. Because of my journalism background, I can chase tangents and write for hours -- doesn’t mean anyone would want to read it. So unless the topic requires deep explanation or lots of pictures, my personal post size limit is one page (and this one just breached).
Part of what makes software development a perfect career choice for me is that it never stales. At least during my lifetime so far, technology hasn’t sat still for more than a few years at a time. The next layer of capability, the next opportunity, the next whatever is always incubating. I can safely say I’ve rarely had the same year of experience twice.
And I love that about it. I love that I need to keep learning and reinventing myself as a technical professional, that what I know is only a little bit more important than what I’m learning. We are served well if we can maintain a “Beginner’s Mind”. [I first learned of the Beginner’s Mind from Zen Guitar by Philip Toshio Sudo but a quick search will provide many references.]
One of my favorite stories (perhaps even a parable) related to the Beginner’s Mind (cobbled together from a couple different versions) goes...
A university professor comes to visit a famous Zen master. While the master quietly serves tea, the professor talks on and on about Zen. The master fills the visitor’s cup to the brim and keeps on pouring. The professor watches the overflowing cup until he can no longer restrain himself. “It’s overfull! No more will go in!” he blurts out. “You are like this cup,” the master replies, “full of your own opinions and speculations. How can I show you Zen unless you first empty your cup?”
Following up a bit on the last topic (where I talk about how much I love the field of software development because I always have to be learning new things), a college professor friend of mine who teaches software development shared a disappointing story...
As he’s chatting with some students after class, they ask him which language or technology they can learn so that they don’t have to learn anything else once they get their degree.
This blew me away on several levels: (1) These are university students participating in a largely optional learning activity (college) and they are already trying to figure out how to stop learning?! That doesn’t bode well for their lives in general, much less their future as software developers, (2) They are studying a technology field and are old enough and smart enough to get into college, yet they’ve missed the fact that technology has done nothing but change their entire lives? There’s a chance that some of the specific technologies they learn in college won’t even be relevant by the time they graduate, (3) They think they’d actually want the job they are describing. [That last one is wrapped up with some personal bias, for sure -- I can’t imagine any period in my career where I would have been happy doing just that thing for the remainder of it.]
Yet, somehow I fear I may be the odd one here. One of my favorite interview questions (sorry, no bizarre thought puzzles like “how do they make M&Ms”) was “What was the last book you read and what magazines do you read regularly?” I rarely got a decent answer. Often I’d get quizzical looks and would have to eventually follow up with “Software development is a rapidly changing field. You’d like to come work for us as a software developer. I’m trying to figure out how you go about keeping up with change and staying abreast of new technologies and practices.” Unfortunately, they often now understood my question and still had nothing satisfying to say.
BUT, here’s the answer that I hated the most: “Yeah, you’re right, doing that’s a good idea but my company doesn’t pay for books or magazine subscriptions.” So you know it’s a good idea, that it would be valuable to you as a software developer and by extension make you more valuable as an employee, but unless someone else pays for the book or magazine subscription, you’re not going to do it? [imagine several more paragraphs of crazed ranting here -- it’s harder than I thought to not include them]
I’m afraid to ask this question any more. It’s so easy to follow folks on Twitter or Google+ or read their blogs or get entirely free magazines sent to you in email or buy even cheaper e-book versions of books that the interviewee’s quizzical stare would cause me to tilt.
To help fight this phenomenon I will occasionally curate a useful set of links or list of books that I personally find valuable. If you know of equally good or better books or links, please post in the comments -- I can’t read everything and I won’t post about something I haven’t actually read or used and found value in.
Getting StartED with Dojo by Kyle D. Hayes with Peter Higgins. A gentle yet useful introduction, nicely organized and presented.
Dojo: The Definitive Guide by Matthew A. Russell. As I searched for a really good Dojo book, this one kept appearing at the top of people's lists. While based largely on dojo 1.1, it focuses on fundamental aspects of Dojo that I expect remain the same today. I understood things after reading this that I only thought I understood before reading it.
Agile and Scrum
Implementing Lean Software Development: From Concept to Cash by Mary and Tom Poppendieck. This book "made it click" for me, providing a vivid description of how you can deliver high quality software of value to the customer while eliminating waste and constantly improving your team.
Succeeding with Agile: Software Development Using Scrum by Mike Cohn. I very much enjoy Mike's writing style and I give this book an edge over Ken Schwaber's original treatment, but either will suffice.
Agile Estimating and Planning by Mike Cohn. If you want to really understand how Story Points are meant to work and how Agile Planning seeks to mitigate risk, this is the book. If you're more into Cliff Notes, Mike provides lots of useful material at his web site: http://www.mountaingoatsoftware.com
I personally have a Safari Books Online (http://safaribooksonline.com/Corporate/Index/) Individual subscription. For $23 a month I can read any book I want, up to 10 at a time on my personal "bookshelf". You also earn credits that allow you to download books permanently to your computer, often in a variety of formats (PDF, .mobi, .apk, etc.)
You don't have to look around much to find articles decrying how few American students are choosing engineering or technical professions. It has been feeling like an annual set-piece for years. Hiring managers wonder why more American students don't choose these high paying and professional career paths. Is it because they aren't properly prepared by our education system? Are they too lazy? Too interested in video games or rap music or marketing?
I'll suggest it's our fault. You and me.
I don't have a lot of data, but I have some that's poignant and personal.
My son is a smart kid. He does well in most subjects if he cares to. He's in college and certainly could be studying a technical discipline if he wanted. I've been a software developer my whole career (and as you could tell by past posts, I enjoy it). As a profession it has supported our family well including some private school education for the boy, quite a few glorious vacation experiences and many of life's amenities that we could have done without (and many do).
While enjoying one of those fun-to-do-but-somewhat-expensive outings with my son (and a couple co-workers), I had to step away from the group to pay for parking. [I learned of this discussion later, but confirmed it with all parties involved.] My teammates were politely chatting with my son about his plans now that he was in college. And the somewhat obvious question came up: "Have you ever thought about working for IBM and doing the sort of stuff your dad does?". Without needing to reflect much on the topic, my son replies "No. I see how much he works -- all the nights and weekends. That's not the sort of life I want."
I feel the need to defend us momentarily. By all accounts, I've been a very good father. I never miss an event (and I'm not late to them nor distracted at them, either). I read to him at bed time every night until he was almost a teenager. We did Tae Kwon Do together. I've been involved in all sorts of volunteer activities in support of him (I was a Cub Master and Assistant Scout Master for many years and ran fund raising for the wrestling team). I was his Statistics tutor. In my son's defense, he's not lazy nor unmotivated -- he's an Eagle Scout and has worked tirelessly and without complaint many times and in many situations. We work side-by-side in support of our local food bank and Habitat for Humanity organizations. He's been on the Dean's list or Honor Roll more than once and his first full year of college isn't quite over yet. And I know he thinks I've been a good father because he brought a large room full of people to tears (including me) during the speech he gave at his Eagle Court of Honor recently.
So what gives?
It's not me complaining about my job (most of the time) as I'm the first person to tell you I love what I do for a living. [I'd rather play bass for Lynyrd Skynyrd, but that job was taken.] It's not him being incapable of doing the work. As he's playing XBox on the large HDTV with explosions coming from the surround sound speakers while posting to his Tumblr on his Mac Book, it's not because he's unaware of what earning a good living makes possible.
Part of it is politeness on my part, failing to provide a complete context. At least some of those evenings or weekends I was working were caused by not-working so that I could attend the school play or teacher's meeting or wrestling match or drive a car load of kids to a campout on a Friday. Fortunately, the work I do makes it possible to have that flexibility. But you don't want to say very often "Daddy's working tonight because he took time to be with you today." These activities were never a burden and I refused to give him any reason to think they were.
Part of it is a personal achievement complex I've never been able to shake. I don't want to do good work, I want to do great work whenever the situation allows. If I can swap a couple hours of relaxing in the evening for an impressed customer, I'm usually happy to make that trade. I'm a maker at heart and want to make things worthy of my time and effort (and other people's attention).
But I think the larger part of it is how we run this business of software development (not necessarily at IBM, I've worked a few other places during my career): Plans that were never achievable unless everything went perfectly (and when is the last time that happened?). Customers that want more because they know you could give it -- and if you don't maybe they won't give you the next job. Teams that want to do more than they promised because that's just how they roll. And all of this comes at a personal cost to those involved -- and it may be convincing their children to look for other kinds of work.
I hadn't heard of Agile Fluency until a few days ago. I had recently joined AgilePDX (our local Portland group of folks interested in all things agile) and the email reminder came in about the monthly evening talk. Jim Shore and Diana Larsen were going to discuss their Agile Fluency model as part of the group's annual Agile Tune-up where everyone comes to think about what good goals might be for their teams for the coming year.
The fog was thick outside. As bad as I'd ever seen it (and it proved to be as difficult to drive in as I feared). I thought about not going to the meeting, but then I googled “agile fluency” and started to read a bit about Jim's background (I hadn't realized InfoQ had named him one of the most influential people in agile for 2012: http://www.infoq.com/news/2012/04/agile-influential-people). Then I read the article. Then I realized I really needed to go – this was going to be interesting.
I won't recap the article, you can read it here: http://martinfowler.com/articles/agileFluency.html. I won't recap the meeting, Moss Drake did a great job in his blog the next day: http://mxmossman.blogspot.com/2012/01/agile-levels-of-fluency.html (though I'll mention to him that it's really 2013 now). I'll presume you've already popped those links into a new tab and skimmed them. No? Go ahead, I'll wait.
Okay, just in case you didn't go read, I'll recap a little bit (most of this is in the article, some of it embellishments from the meeting): Jim and Diana discuss a team's path through agile fluency (based on their extensive experience in the space and with many teams), giving each level a star:
What struck me most about the article is the use of the word “fluency” and how it is defined by Shore and Larsen. They draw a bright line between proficiency and fluency. Proficiency means you understand the set of practices and are capable of following them. In their words, “Fluency is how a team develops software when it's under pressure. Anyone can follow a set of practices when given time to focus...true fluency is a skillful, routine practice that persists when your mind is distracted with other things.” Like broken builds and disgruntled customers (or, here in the Pacific Northwest, the first sunny day in weeks).
I'm reminded of a saying about musicians: Amateurs practice a piece until they get it right; professionals practice a piece until they never get it wrong. For me that sums up the difference between proficiency and fluency in this context.
Having been involved in several team's attempt at agile transformation, I've seen a lack of understanding of the need for fluency disrupt the transformation. The team has had some training and faltered in their early sprints (as is common and generally expected) but they are using the retrospectives and honestly looking at what needs improvement. Then, before they reach a point of fluency, something distracts them, some disruption in the force tries to throw them off their game. And it is successful. Before they can get back on track, their manager decides (problem #1) that the retrospective meeting is worthless because nothing seems to get better – and it just sort of crumbles from there. They are now a Scrum-but team and may always remain one. Had they been given the space to reach fluency at their current level, the could have weathered this storm. Lacking that fluency, they could only return to what they used to be fluent at.
Reminds me of another saying from a different field: Under stress, you will not rise to the occasion, but default to your level of training. So your level of training needs to bring you to fluency at the things you care about and your product and team depend on.
Models like CMMI tend to focus on practices that you adopt – so perhaps you are proficient. But the Agile Fluency model's focus on fluency, not just being capable of the practice but having adopted it so completely that it's just what you do makes a lot of sense to me. I also like the model because it is not about some set of practices you have adopted or a particular metric you've hit but about who you are, how you behave. Your fluency allows you to “be” at that level – it's what your team has become, how it's focused, what behavior feeds its soul.
The article includes a nice diagram of the path as well as a discussion of the benefits, investments and core metrics at each level. It's frank about the type and size of investment required at each level. Two stars can be hard because of the breadth and depth of skills required and the level of practice to become fluent. If you work in an organization that cares about agile development, go read and share the links (and this blog if you are so inclined). The ideas are valuable and I expect you'll hearing more about them.
It's mostly my fault for being optimistic and hoping that this time will be different, trying not to let a bad experience with someone else in a different context prejudice me against you and your situation. But it has become such a predictable pattern that the only thing to do was either
Only once in recent memory has this worked the way it should:
This happened with an early draft of The Practical Guide to Distributed Scrum. Elizabeth Woodward, the lead author, asked me to review it. I didn't particularly care for the structure of some of the content -- the hardest sort of thing to fix. But she asked early enough and we worked through the issues and the book turned out much better because of it.
Usually, it works like this:
When I agree to review something, I accept partial responsibility for the end product. I become a partner, even if a minor one. I will not skim the material and give you a winking thumbs-up. I will ask questions, some times lots of them. More than one person over the years has been shocked, I think, to get a detailed set of suggestions (from outline adjustments to copy-editing) rather than a "looks fine to me" reply.
I do this for one simple, perhaps egotistical, reason: If you are going to tell people that I reviewed it, then I want it to look like something I would publish, present or approve of. I'm going to treat it as partly mine and put the same level of effort into reviewing your work as I would put into reviewing my own. But that seems to almost never be what you want (or at least not something you can use). You want me to say it's okay so you can be done with it and claim to have had it reviewed. I'm never going to do that.
Just like changing code, fixing demonstrations and tutorials after they have been written is costly. Just like code, you are probably up against a shipping date and you've waited too long. So let's promulgate proper planning for handling reviews, particulary if you are using reviewers outside of your team who have their own projects to worry about:
If this seems like too much effort for your project, then don't do your project -- it's probably not worth doing. My father (and I'm sure many parents) always said "If it's worth doing, it's worth doing well." If time is too tight, talk your team or management out of doing it. If they think it's too important to drop, ask them why it's not important enough to do well.
Before I get to the closing fun, it's important for me to state clearly that I LIKE reviewing things. I like being able to add value to something I didn't have to build from the ground up myself. I like that someone trusts my opinion enough to ask for it. I just can no longer afford the time, effort and frustration it costs me if you aren't actually going use the feedback (part of the Lean mantra: Identify and remove waste).
I have a new set of questions I will be using before agreeing to review anything:
ME: I'm imagining it's fine, go ahead and use it.
YOU: But you didn't even look at it!
ME: You just told me you wouldn't have time to fix it if I found something worth fixing. What's the point of me looking at it?
YOU: But maybe you won't find anything big that needs fixing...
ME: Maybe not. But given the rush you've put this together in, it's almost certain.
YOU: Well, why don't you review it anyway and we'll fix the big things in the next version.
ME: Why don't you just publish it and when you are ready to start work on the next version, I'll review it then?
YOU: Because I need at least one reviewer to sign off or I can't publish it.
ME: Sorry. Let me refer you to this blog post I wrote.
I'm not being difficult. Like most of us, I can fill my days (and evenings) with my own work. I've read that learning to say "no" can improve your productivity by avoiding wasted effort and distractions. This is how I start.
Case Study: How social interactions changed my life (or how seeing rock god guitarist Eric Johnson made me a bluegrass fan)
Millard 060001WYSU Tags:  medicine music old media adrienne connection show ketch secor crow bluegrass social young 2,311 Views
For me (and I imagine many others), social media and the opportunities it brings enrich my life. Everyone is more interesting than we probably give them credit for and if you listen to their story it will be hard not to like something about them. But that's not where this story begins...
Shortly after seeing Old Crow Medicine Show (pictured above sans one member too far left to fit in the frame) at the opening of the Oregon Zoo's 35th annual concert series, my wife and I were on a short road trip, reminiscing about the concert and the series of events that brought us there. I grew up on 60s hippie music (wearing out that Woodstock 3 record set) and came of age listening to 70s progressive rock of Yes, Emerson, Lake & Palmer, early metal from Black Sabbath (no one really appreciates Sabbath bloody Sabbath as much as it deserves) and the hard-edge rock of early Grand Funk Railroad. I've played in a band the singer not-very-jokingly referred to as "post Roman death metal".
So I'm still surprised that I now seek out bluegrass and Americana and these are the live shows I'm most likely to attend. What the heck happened? It all started at an Eric Johnson concert. [Eric is considered one of the most respected guitarists on the planet, though I'm not sure he's strayed into my new genre much.]
This is where it gets social, where one thread leads to another until you are no where near where you started. I find this sort of thing happening a lot when I have the time to follow links from one place to the next -- though not always with such dramatic life changes. My wife and I put this chain together as we were riding along the Columbia river.
Tony Furtado opened the Eric Johnson show. I never heard of him before that night (but I'm one of those people that comes for the whole show, not just the headliner). He was amazing (and the main thing I remember from that evening). I went home and searched for more about him because I wanted to share it with my wife. I found a link to a performance he did for the Woodsongs Old Time Radio Hour. Given the name and my musical background, I was a bit dubious, but watched it anyway. Also on the same program was Adrienne Young and Little Sadie. I fell in love both with one of the songs they played (Home Remedy) and their upright bass player's amazing smile.
This led to two subsequent actions: purchasing the album the song was on (the Grammy nominated Plow to the End of the Row) and attending the Live Oak Music Festival near Santa Barbara, California where they would be performing. The Live Oak Music Festival turned into an annual trip, introducing us to many other artists along the Americana, bluegrass and Celtic path.
My wife gets all the credit for the next link in the chain that brought us to Old Crow Medicine Show (OCMS). While we were enjoying the song Conestoga from the album, she heard a guy singing background vocals whose voice was interesting to her. A bit of research in the liner notes turned up the name Ketch Secor. Along with being a generally talented multi-instrumentalist, he's also the front man for OCMS (he was also Adrienne Young's banjo teacher). This, of course, brought us to OCMS who we saw for the first time at Stubbs BBQ in Austin, Texas (wonderful outdoor concert venue if you ever get the chance). Pretty sure this video was taken at the show we attended. Yeah, there were a lot of people there -- a lot of young people very into the high energy music of OCMS. We were hooked.
What has kept me hooked, weird as it may sound, is the teamwork of a good bluegrass band. Many bluegrass groups work around a single microphone, moving in and out to raise their volume when it's appropriate. You can tell that they are listening intently to each other, feeding off clues, reacting to what they hear. There's a case study in teamwork there that I may chase down some day. On the Americana front, the often deep and meaningful lyrics of the singer-songwriters keep me listening and wanting more. It's a shame everyone hasn't heard of Slaid Cleaves (we own all of his albums and have seen him at least a half-dozen times).
I'm hooked on social media and the opportunities it opens for everyone to find new things to learn about and cherish. Follow lots of different people, listen to their stories and see where it takes you.
This time I'm going to make your life better in a most delicious way (unless you are vegetarian/vegan). There's going to be some useful but gratuitous technical design stuff and some links, but the meat of the post is after that.
Importance of simplicity
KISS. Keep it simple,silly.
But why? Because people don't like to learn and won't bother to when they can avoid it: http://www.nngroup.com/articles/stagnating-expertise/ If you force cognitive strain on your users you will lose their trust and this has many downsides: http://www.nngroup.com/articles/navigation-cognitive-strain/.
When we look at great designers who have done simple well, it's hard not to see Steve Jobs at the head of that group (these quotes are from http://blogs.wsj.com/digits/2011/08/24/steve-jobss-best-quotes/, which includes many others if you like this sort of thing):
“Simple can be harder than complex: You have to work hard to get your thinking clean to make it simple. But it’s worth it in the end because once you get there, you can move mountains.”
“Look at the design of a lot of consumer products — they’re really complicated surfaces. We tried to make something much more holistic and simple. When you first start off trying to solve a problem, the first solutions you come up with are very complex, and most people stop there. But if you keep going, and live with the problem and peel more layers of the onion off, you can often times arrive at some very elegant and simple solutions. Most people just don’t put in the time or energy to get there. We believe that customers are smart, and want objects which are well thought through.”
Simplicity tells your audience that you care. Even if they are smart enough to use your complicated product and even if they are inclined to do so, they won't forget that you didn't try harder to build them what you should have.
If you are jonesing for some more technical discussion of why simplicity matters in software design, here's Rich Hickey's 2012 Rails Conf keynote: http://www.youtube.com/watch?v=rI8tNMsozo0. Some folks prefer his talk from Strange Loop the year before: http://www.infoq.com/presentations/Simple-Made-Easy Either way, you win.
Winner, winner, chicken dinner
We cook at home often. After a while, we got good enough that the results were better than going out in just about every way (except the cleaning up part afterwards). This of course leads to experimentation, different styles of cooking, using different approaches, etc. Some times these can get quite involved and use small amounts of difficult to find or expensive ingredients (and you always need to find the recipe to recreate it).
This post is a result of a recent chicken dinner at home. Everything I wrote above was just to get us here. I was surprised that such a simple recipe regularly produced some of the most delicious chicken I'd ever enjoyed. But as I pondered that, I realized that the simplicity may be the reason it was so reliably good.
Having prepared chicken many ways over the decades, we've settled on this recipe: http://www.epicurious.com/recipes/food/views/My-Favorite-Simple-Roast-Chicken-231348 because it is simple, easy to follow and yields the same delicious result every time. It's literally as simple as (1) rinse and dry the bird, (2) salt and season the cavity, (3) truss the bird, (4) rain salt on the outside, (5) cook it for about an hour. Sit down to a delicious dinner -- if that sounds too easy, just try it.. If the trussing-the-chicken part has you worried, it's also dead simple -- here's a 90 second video that will have you doing it in no time: http://ruhlman.com/2010/07/how-to-truss-a-chicken/
Simplicity isn't just about the quality of the result but also in how you get there. Thomas Keller is a phenomenal chef and could have written pages worth of directions (that people would have slavishly followed), but they weren't needed. He had a simpler way that worked perfectly every time. After you have done this once, maybe twice, you won't even need the recipe any more -- the simple steps will just stick in your brain. That's perfect instructional design. Enjoy!
Addendum: "Winner, winner chicken dinner" was stuck in my mind and I couldn't remember where it came from. So I had to Google it: http://www.urbandictionary.com/define.php?term=Winner+Winner+Chicken+Dinner
Millard 060001WYSU 5,361 Views
A while back a friend and fellow IBMer @ProfAvery turned me on to Sahat Yalkabov's hackathon-starter project on GitHub. At the time I was just trying to learn some more about node.js and the project provided some solid examples for using various APIs and over time with contributions from many folks has become what I expect would be a solid base, at least for a hackathon effort and possibly for more serious, long-term development. And as I have watched it develop I've been pleased to see how Sahat has stuck to his guns about keeping it reasonably simple and approachable -- he has politely turned down a variety of contributions that may have added some value, but would have also moved the project away from its purpose.
As IBM Bluemix™ and its DevOps Services started to develop, I realized it would be a great match for hackathon-starter. While it's easy to stand up hackathon-starter in many environments (including Bluemix -- more on that in a moment), DevOps Services integration makes it simple to collaborate and provide a continuous delivery pipeline automatically deploying each delivery to Bluemix. With a shared Git remote at DevOps Services, you could easily work together on a hackathon from distributed locations.
I think that's the best way to stand up hackathon-starter: using Bluemix and DevOps Services together for a collaborative, continuous delivery environment so I wrote a developerWorks piece about it, complete with screen shots, some sample code bits and a short video.
Another way that also works -- but without the DevOps Services integration -- is to just use the Cloud Foundry command line to push it to Bluemix, configuring the required database service and a process environment variable to connect to it. While I consider this approach less capable from a teaming perspective, it can certainly work if you just want to stand up the example and poke at it yourself. I'll do it two ways: the first feels like the easiest to me and takes you to the Bluemix web interface where you can explore other things about your project; the second is a completely command line approach (#protip: reading through the first will provide some useful context for the second one).
For both approaches, the basic steps are very similar:
Note: On June 23, 2014, the IBM BlueMix service dropped the ng.bluemix.net domain in favor of mybluemix.net. While I have fixed the text, some of the screen captures still show the previous name. Please be aware of the difference while working through the material.
The first four steps are pretty basic stuff and I won't explain those further. Step 5 is the same for both, so let's do that first.
Since you've installed the Cloud Foundry client, it should be on your path. Make your current working directory the one with hackathon-starter's package.json file. From your command line window, execute cf push your-app-name -m 512m. Obviously, replace your-app-name with whatever you like.
You'll see a lot of activity in your command line window as Cloud Foundry processes the package.json file and pushes your application to the cloud. It should start something like this:
and end something like this:
But if you believe it and go to your application URL, you'll see this (because there is no MongoDB service configured for the application yet):
There are two paths for fixing that, let's start with the cuter one.
hackathon-starter and Bluemix
Log into the Bluemix web application, go to the Catalog and create a MongoDB service. Where it says Add to:, insert your-app-name, same as how you named it for the cf push command earlier. You can leave the name as the generated default or change it to something that is more meaningful to you:
You will be prompted to restart, but don't do it yet - there's no point until we complete the next step.
Click on your application from the Dahsboard page and then on the Runtime section. Scroll down to the Environment Variables section. Under the VCAP_SERVICES section you'll see some JSON that describes your MongoDB service. Copy out the text of the "url" attribute towards the bottom of the section -- be sure to get it all. It should look something like this: mongodb://68638358-a3c6-42a1-bae9-645b607d55e8:firstname.lastname@example.org:10123/db
Now switch to the USER-DEFINED Environment Variables section -- it is likely empty. Using the string you just copied from the VCAP_SERVICES MongoDB url, create a new environment variable called MONGODB and set that string as its value.
Your application will automatically restart after you save the change and after a moment, if you visit the Route shown at the top of your application's overview (your-app-name.mybluemix.net, you should see hackathon-starter. Note that configuring the database this way is a touch fragile as if you needed to make any changes to the MongoDB URL (you dropped and recreated the service, you wanted to use a different service that works with the MongoDB wire protocol, etc., you'd need to come fixup this environment variable to match.
Command line only, please
We can configure a service for our application from the Cloud Foundry command line as well as interrogating the environment variables to get the values we need to set our new user-defined variable. For the command line purists in the audience, I'll step through those parts next.
I'll leave most of the investigation to you, but here are some basic commands that will help. Use cf marketplace to see a list of available services, cf services to see a list of your services, cf create-service (add -h to see help for this, including an example) and cf -h or cf command -h for help at any time.
1. To create your MongoDB service from the command line, use (the last part is a name you choose): cf create-service mongodb 100 your-service-name
2. Bind your new service to your application: cf bind-service your-app-name your-service-name
3. Use cf files your-app-name logs/env.log to see the content of the VCAP_SERVICES environment variable.
4. Copy out the url attribute value in its entirety.
5. Create a new environment variable for your application with: cf set-env your-app-name MONGODB your-url-value
It will look something like this:
cf set-env bm-hackstart MONGODB mongodb://68638358-a3c6-42a1-bae9-645b607d55e8:email@example.com:10123/db
6. To see the currently configured user-defined environment variables for your app (to verify it worked), use: cf env your-app-name
7. As the TIP in the cf set-env command output suggests, use cf push (see full command example earlier) to make sure the changes take effect. Using just cf restart your-app-name might work, too.
Go visit your app at your-app-name.mybluemix.net -- it should be running and ready to go.
I've started looking at Docker™ seriously as a way to make all sorts of things easier. We're considering it for our team's next generation development environment work, incorporating it into our build process to make testing easier and perhaps even as a way to reguarly push new versions into production. I have a lot left to learn (and working on Windows, at least for now, it's all just a little bit harder since the support is not yet native -- though this seems to be changing http://azure.microsoft.com/blog/2014/11/18/docker-cli-for-windows-clients/).
To help with my learning, I try to answer relevant questions from a "How would I do this with Docker?" perspective. I needed to standup a standalone web server to test something recently. I could have downloaded and installed Apache or IBM HTTP server, but that would lead to some cruft as I may not need it for long and now there would just be more stuff in my path and on my system that didn't really need to be there.
Of course Docker could help with this, no? Turned out to be a piece of cake (though largely because of past research and efforts). Here's what I did with some background notes in case it helps you do something similar (or increases your interest in Docker). Most of this is easier if you aren't on Windows, so I'll focus on the Windows parts.
Steps to stand-up a standalone web server mapped to your local system are below. This is not a complete step-by-step tutorial for getting started with Docker. You can learn more about Docker (and using it with Windows) at www.docker.com.
Mostly just that boot2docker has been installed (see http://boot2docker.io/) and works (you can at least do the helloworld-style tutorial). That gives you the infrastructure for the rest of this.
0. Start boot2docker
1. Get Apache's httpd
docker pull httpd -- will download the official Apache image from Docker hub
2. Start the image, exposing the port and mapping to your local (Windows) file system
This is the interesting step and where some of the background information comes in handy. As of late 2014, Docker will use VirtualBox Guest Additions to automatically map your Windows C:\Users folder for sharing with the VM running docker (see https://github.com/boot2docker/boot2docker). What this means is that C:\Users will be known as /c/Users to the OS running in your docker container. This is critical (and cool) because you can edit files on Windows using your normal tools and when you save them, they will be available via the web server immediately.
The example startup from the docker hub image page (https://registry.hub.docker.com/_/httpd/) is:
docker run -it --rm --name my-apache-app -v "$(pwd)":/usr/local/apache2/htdocs/ httpd:2.4
Rather than use whatever directory you are in "$(pwd)", use a mapping that works for Windows and the VirtualBox Guest Additions. For me, this changed the -v argument to:
This maps my C:\Users\Public\project\webserver directory to the top-level where Apache expects to find web files.
The other piece you need to add to the startup is a port mapping. I'm pretty sure this is another boot2docker thing and wouldn't be necessary if you weren't running under Windows. While the image exposes port 80 on the container, I need to map that to my host system to make it visible. So I have to add a "-p 80:80" to make the port visible (if your port 80 is already in use, you can use this mapping to set it to something else). I dropped the ":2.4" from the image to run since just running the latest version of the one I just downloaded was fine. My final startup command for boot2docker is:
docker run -it --rm --name my-apache-app -p 80:80 -v /c/Users/Public/project/webserver:/usr/local/apache2/htdocs/ httpd
By quick explanation, "-it" says to run interactively rather than push it into the background and "--rm" says to automatically remove the container when it closes. This keeps startup simple -- it's always this same command. Otherwise, you'll need to use stop and restart for the container and if you want to change the arguments, you'll need to provide a new name (or remove the existing one first). See the Docker docs for more details
3. Create or copy your html to your mapped directory
For my initial test, I just made a simple Hello World index.html and put it in a directory named helloworld under \webserver (the mapped top-level location for web files).
4. Hit the new web page at the super secret IP address
Okay, it's not really a secret, but you will need to figure out what it is. From a command window, enter boot2docker ip. This will likely show you that the IP for the Docker image is 192.168.59.103. But it could get mapped to something else, so it helps to check if things don't seem to be working.
So I visit http://192.168.59.103/helloworld/index.html and see:
Now I can just add folders to my /project/webserver folder for other web sites. I can get updated web server software by just downloading an updated Docker image -- no need to run updates on my local system. And when I'm done with this side project, there's nothing to uninstall and no cruft left behind. Docker FTW!
How will you use Docker?
To live the Agile/DevOps lifestyle (being agile is more attitude than practices), one thing you need to do is drive out manual process steps wherever you find them in your project or team. [Well, almost every place -- I'd prefer we didn't automate away the team lunch.] And there will be times when you have to grit your teeth, stick your nose in a bunch of books and blogs and just make it happen. This is the story of such a time...
This quite excellent article describing modern Java development practices (http://www.jamesward.com/2014/12/03/java-doesnt-suck-youre-just-using-it-wrong) opens with 10 Page Wikis to Setup Dev Environments Suck (I very much agree with this and many other points in the piece). I recently worked with a team that had a 50+ page document -- not even a wiki yet -- and it was generally accepted that if you got through it in a week, and things worked, that you had done well. I was aghast, but being the new guy, I just ground my way through it vowing that some day this had to be automated.
The biggest part of the grind -- and the most error-prone -- was configuring WebSphere Application Server for the project. There were pages and pages with instructions and screen shots and tables of data to enter. And if you messed up part of it, you might not realize it right away. This actually happened to me. Months after completing this stage, I was working on a different part of the application and I was getting errors no one else was seeing. Hours of debugging later, I realized that I had transposed characters in one of the JNDI names during the setup (months earlier). It just wasn't until I worked in this particular part of the code that it was actually a visible problem.
Backing up in time a bit, when I was first handed this doc, I mildly protested: "You don't have any automation or scripts for this?" The answers seemed to fall between "Never had the time" and "No one knows how". I didn't know how, but I knew that it could be done -- I had worked with some folks that wrote WAS admin scripts for a living. I had done some work in Python before and the WAS admin scripting was done in Jython (Python for the JVM), so...
When we started working on the next generation of our product (not just the next version, but a brand new, nearly green-field, newly architected version) I knew the time was now. If this were ever to get done, it had to be part of our build-first, DevOps-oriented reinvention of ourselves. So I lobbied for this as a high priority item and even agreed to do it because it needed doing. No sensible, sustainable, Agile-reinvention was going to begin with a new version of the 50+ page Dev Environment Setup Guide -- not even if it was turned into just a 10 page wiki.
Few people are good at everything and I certainly had no experience with Jython or WAS administrative scripting (and am certainly no expert at Windows batch scripts). And, frankly, I didn't really want to do this (when would I need this again?). But somewhere between the concepts of Whole Team and Collective Ownership, someone had to do it. When I'm not good at something that matters, I tend to lean in. I own my ignorant suckiness and fight through it (with lots of help from other people's blogs and StackOverflow.com). The rest of this blog is a description of what I built in hopes that someone else fighting a similar fight can cop a short-cut on me.
Get on with it already
TL;DR -- It worked. We took a multi-page, multi-day, error-prone process and replaced it with a page or so of setup instructions and a couple scripts that take maybe 15 minutes to run. We made the WAS profile a throw-away item that can be recreated in minutes, not something to be cherished and protected. That is the DevOps Way.
Enough story telling, time to talk real WAS automation.
What we were trying to do isn't all that fancy:
1. Create a couple JAAS Authentication entries to use with data sources
2. Create a JDBC driver for DB2
3. Create a few data sources used by various applications
4. Add a collection of SSL certificate entries
5. Create a collection of Work Managers
6. Create a collection of Cache Managers
7. Save changes (so that we can)
8. Test the data sources
9. Happy Dance.
What we weren't trying to do is make it bullet-proof, so you won't find scads of error handling in the code. For the most part, if the script fails, the save operation is at the end, so nothing gets saved. You can figure out when went wrong, fix it up and run it again. If that doesn't work, crush the profile and start over. This is our Minimum Viable Product and it proved sufficient.
I'm not going to explain WAS scripting beyond saying that you get to write Python (technically Jython) code that runs on the JVM and uses libraries provided by WebSphere to complete most tasks. The real work is done by starting the wsadmin tool (which sets up the Jython environment and pre-loads some libraries you'll need). The WAS docs do a reasonable job with the details and some searching will fill in the gaps. Most of the action is at the command line (all examples use Windows) so a text editor that understands Python syntax is helpful. I'm also not going to explain the details of how the scripts work -- if you've read this far you clearly have an interest and enough background to understand or work it out.
What I do need to explain is something that it took me waaaayyyyy too long to notice: The WAS Administrative Console is your friend and will help you write your scripts. That little link off to the right is always there and I was almost done doing this before I realized it. DOH!
If you're not sure how to do something with a script, do it in the UI (you don't even need to save it), the click that link. In the image above, I created test data source then clicked the link and the window came up showing the scripting command.
The code is available at GitHub: https://github.com/millarde/wasadmin-automation.
CAVEAT: Some of the things in the scripts are very specific to our environment. They aren't meant to be configurable to your environment. Make a copy, learn what you can, change what you need to. I have trimmed and changed some of the data for brevity and privacy.
Setting the Environment (literally)
To coordinate between the scripts involved, we needed some environment specific information. We don't need it permanently, so rather than create new permanent environment variables, the user establishes some temporary variables in setmywasvars.bat and runs it in a new command window. There's a little bit of error checking to make sure they didn't leave out something that will cause an issue later. All subsequent commands should be run in the same command window since they will rely on these variables. Depending on your WAS configuration, you may need to update the port number in wsadmin.properties and you'll definitely need to add the WAS Admin credentials.
Then the user creates a WAS profile using makemyprofile.bat. Since later steps will require the server to be running, the script will offer to start it. At this point, the user may need to update their WAS Node name in setmywasvars.bat if this was their first profile. I'm not aware of a reliable way to determine that value ahead of this.
The last step is to run configuremyserver.bat which will process all of the steps described above (except the Happy Dance, that's for you to do). After creating the JAAS Authentication entries, it has to save changes and restart the server to get them picked up before it can continue. After making all of the other changes, it has to save them before it can successfully test that the data sources work.
Some task details
There is a subfolder for sslcerts. When the script runs, it will pick up and add each certificate to the system. If you have certificates to process, copy them into that folder.
For the Work and Cache Managers, which don't change often, the details are captured in the script. This could be externalized (similar to the sslcerts) if you need more flexibility.
Some versions of Windows are not properly handled by the Jython scripting. If you run the automation and see something like “KeyError: WAS_PROFILE_NAME” or “Failed to get environment...”, then you are running one of those versions. This issue and the fix are described here: http://blog.oliver-mueller.com/-216. The fix is to create a file named %WAS_ROOT%\optionalLibraries\jython\registry and put this one line in it: python.os=nt. Now Jython can properly detect you are on Windows and will be able to process environment variables and complete your setup.
The batch files
There are quite a few small batch files that do little things -- some meant to be used by the user, others are more internal and some to save me time when I get pinged about something not working (e.g., showmywasvars.bat exists solely so I can ask someone to run it and confirm their settings). See the GitHub file list (or the file's comments) for more description.