Rational Software Development Conference 2006 June 6 keynotes
Erich Gamma, IBM Distinguished Engineer
John Wiegand, IBM Distinguished Engineer


ROGER OBERG, VP OF MARKETING, IBM RATIONAL (Conference Maestro): This morning we're going to get to jam with some serious
visionaries from IBM, not a one hit wonder in this crowd, and we're going to start with a pair of IBM
Distinguished Engineers noted for their contributions to Eclipse.

You know, the Eclipse development team has consistently hit its projected delivery dates with precision and
quality, release after release.

And some have said that Eclipse is one of the world's most successful software projects ever.  And here to
speak on the realities of what makes the teams successful with the lessons they've learned from Open Source
development, please welcome two of our favorite living legends, Erich Gamma and John Wiegand.
[MUSIC]

JOHN WIEGAND: Erich and I talk about various aspects of Eclipse.  And this morning we thought we'd try
something a little bit different than we've done normally, which is we're going to step back, we're going to
talk about some of the things we did pre Eclipse.

And then transitioning from the pre Eclipse, watching what happened as we started working in open
developments, our experiences, and then comparing and contrasting both the closed style of work we had prior
and the open development that we're doing now.

So just a little context.  Eclipse has been around for a while at this point.  We're up to six years of
development in Eclipse.  It wasn't all done in the open.  So this is just going to talk about that time box
and what you can see here is that we spent a year or so getting to a place where we had something interesting. 
We'll touch on that just a little bit.  And then beyond that we focus on the regular rhythm of Open Source
development.

So what makes something interesting?  The normal test here is it's something you can talk about, something
tangible.  You can actually put your hands on it.  Our test in Eclipse was we wanted to get to a place where
we were self hosting on the technology.  We were using it ourselves and we were ready to get the feedback to
help make it better.  And so that was the point where we went Open Source.

ERICH GAMMA:	So what's behind that actually is that we always did all the work in a distributed way.  We
never had the luck to all sit together in the same time zone.  So we just always had to be globally
distributed.  So we had no choice and we always liked to stay in our favorite hometowns and do cool stuff.

How did we start?  So for us it was a big step from where we started to where we ended, because we just did
not make closed development, we did as closed as you could get.  I'd like to call that the Swiss bank account
approach of self development.  Each project just had a number, was put in a safe every evening.  No one could
see it unless it shipped.  It was really hidden until the point it shipped.

And we also had a very strong firewall between the developers and our users, or our customers, which means we
never got a buck directly from them.  Until they reached us it had to climb at least three lines of defense.

And we got really filtered down, I guess we couldn't even trace down where it came from.  So it was really no
disconnect.  So that's why I say it's as closed as it can get.  However, we had one value even when we did
closed development, which was key to our style of development.  That's shipping.

JOHN WEIGAND: So just a funny story here.  I was an early customer of OTI.  So I was on the customer side of
things, and worked, this is 20 years ago at this point, and I worked closely with the OTI team bringing
forward my requirements for team development tools and for packaging technology.

And worked very closely, and I thought I had a very close relationship with the team.  And really enjoyed the
experience, actually.

However, let's see, probably about six years later I joined OTI and found out that I only knew about 20
percent of the people in the organization.  I thought I knew what I was getting into, but actually I was
firewalled even in my situation as a starting point.  But that worked very effectively, because inside, once
we were inside that firewall, our focus was shipping.  Shipping is what matters.

ERICH GAMMA:	And also when we ship, there is focus on one approach at one time.  And shipping was our main
goal.  And we had a very simple litmus test if something is good or bad and the litmus test is, ‘does it helps
with shipping?’  So is it indeed shipping product, which means code is in the shipping product.  High value. 
We put a lot of emphasis on it.

All kind of other artifacts we produce, which are not in the final product, don't contribute to shipping, so
they have less value to us.  Similar to other stuff we do like meetings, organizational structure, right?  So
we have this clear separation.  Shipping -- good, not shipping -- bad.

This is also reflected in our way how we value our own team members.  And we had the three quotes that we like
to share.  We've captured this very well.  And I could summarize them as ‘ship happens.’

So the culture is if you ship then you may speak.  The question is, if someone comes in we don't know him, did
they ever ship anything?  Right?  So, and then in the end, no, they never shipped anything.  So that was a
simple way to capture our value system.

JOHN WEIGAND: So we apply that everywhere and the results show.  So here's a reflection over things that the
team that we reflected on at the start has shipped over the last 10 years.

Eclipse is the current shipping piece, but there was MB small talk before that.  VisualAge for Java and
completely independent product even though similar name, VisualAge for Java Micro Edition.  So shipping in a
regular pattern.

ERICH GAMMA:	Now came the big step, right?  It was decided Eclipse will become Open Source.  And I would
expect that everybody jumps up and says cool, right?  We want to go there.

You will see some reactions from the team.  But this required some groundwork before we could actually make
these transitions, and what we learned and what we, of course, also learned from other Open Source projects,
the first thing you had to do was you really establish the rules and make them explicit and capture them in
the charter.

JOHN WEIGAND: So the question is, how does the team work together?  And the interesting thing here is we had
these characteristics inside of our teams prior, even though we're working in a closed way, we just hadn't put
them in writing.

But answering questions like ‘who can actually change the source code?’ so we defined a set of committers --
those were the people who had the permission to make changes.  Others could make contributions but they didn't
have the right access to the repository.

Next question, who is responsible for deciding when things are good enough.  The teams themselves.  And so we
structure ourselves into component teams.  We distribute that responsibility across.  And then we have, in
addition, a project management committee, the PMC, which is responsible for the final sign off.

So the rules we applied, in this case, we applied them fully across the board.  So when we bring a new hire
in, for example, to start working on the project, just because we hired them they didn't get commit rights...

Again they would need to prove that they understood what was going on in the project.  They needed to deliver. 
They needed to ship just like anyone else.  And then after they had proven that they could do what, do the
job, then the commit rights came.

ERICH GAMMA:	So here we see now the reactions when are spreading the word, ‘you go Open Source.’

JOHN WEIGAND: ‘You want us to do what?’  It was definitely a change.  A different way of doing business, and
the reaction was a range of dubiousness.

ERICH GAMMA:	Code in public.  At some point you were always choking, people want to have more releases from
us, why don't we just put our repository on our Internet and everybody can take down the release.  So we were
choking at that.  But now they're telling us we should make this do for real.  So code in public.

ERICH GAMMA:	Have technical discussions in public that's even more a thing, right?  We make ship happen. 
So we code.  What, we should have discussions that's good but do them in public using our language, right?

JOHN WEIGAND: Yes, and a concern about the quality of the interaction.  Answer all those questions?  So there
was a concern that we were going to end up being that first line technical support that we were so well
protected from.  And the concern was that might not be such a happy place.

And then one of my favorites.  ‘Why are we doing this again?’  So the team was seriously, we were focusing,
‘what's the value, how is this helping us ship?’  That was the true reaction from the team.

And it was a learning experience, right?  So we were in this transition period and opening up and we needed to
do things and we needed to practice being transparent.  That was the only way we were going to make this
happen.

So we basically went through the steps of increasing visibility.  And the first thing we did was on this
question answering front, which I said ‘we're going to answer questions as a primary deliverable that the team
has.’

ERICH GAMMA:	Maybe to go back now, to be more transparent, there is something which I guess helped us to
make this step and something which was a problem. Provide visibility in the process.

We always are kind of proud in our code, right?  Having someone else and reading our code, that's cool, right? 
Because that's how we write code.  We want to make it, others can read it.

However, when it came to the other process elements like no people reading our bugs, we were a little more
skeptical -- ‘what does that mean?  We had sometimes strong language when we discussed things.

So this was an area where we had more learning steps to do.  And in fact what we had to do before we made our
bugs public, we had to run them through a word filter to take out the words which we don't want that others
can read when they will be able to read it.

So you see it was really a learning experience for us.  It does refrain from framing for us, but actually at
some point we felt having others also read our bugs, writing bugs and discussing bugs in the same way we write
code with the same pride is goodness, right?  Because our bugs become better, better to read and everybody can
understand what the point is.

JOHN WEIGAND: So we transitioned from this world where we were afraid of the community, we didn't know what it
would do to us, to realizing the community is giving a lot to us, and it's not a one way street.

Rather, the community actually gets to a point where they're answering questions for each other.  They're
reporting and finding bugs that we were spending a lot of time trying to track down.

It was a lot of work for us to find a obscure case compiler defect we just couldn't quite get our hands on
sometimes.  These obscure things would come in through the community and they would track down these problems
for us.  They would put our technology to use.

So, in the Eclipse context they were building plug ins, they were trying Eclipse out in new environments, new
platforms, giving us feedback, making our technology more robust.  Submitting patches in some cases and other
places writing articles, but doing all kinds of things to make the technology better, so a big advantage for
us.

ERICH GAMMA:	So also what the community gives us is not only always technical things.  Sometimes they give
us appreciation.  Thing say thank you for this bug, that you fixed this bug.  And that's what our developers
also appreciate.  So encouragement.

JOHN WEIGAND: But it's a two way street, so as we get these things from the community there's an expectation
that the project of course is delivering things back, and it only works because of the feedback loop.  So if
the project isn't listening, the community isn't going to be providing feedback.  That's the one thing we
learned over and over again.

The other thing is the community needs to see progress.  So a release once a year is not progress.  It's
ongoing releases, ongoing answers to questions, ongoing interaction, discussion about how should this be done,
or ‘why are you doing this?’  Those interactions need to happen all the time.

ERICH GAMMA:	Always here we have to understand the community is demanding.  This is not always balanced. 
It's real life give and take.  If you have a community who is demanding it's also encouragement again because
you want to do better.  You want to improve.

And I guess that's particularly with the Eclipse community we have seen they are very demanding because they
know all the competitive products.  They do for us real time competitive analysis and give us this feedback as
we go.

JOHN WEIGAND: So there's many different kinds of interactions that take place.  Here's just an enumeration of
some of them.  But the first kind of forum for interaction is the Eclipse Web site itself.  We can go, find
out about the project, get access to other people working on the project, hear what they're doing.

But then you need to be able to answer questions.  So we have news groups for that purpose.  There's a wiki
now at Eclipse so you can learn more in that context.  If you're trying to build on the technology, then
people want to not just build on but in some cases they want to contribute to the technology itself.

So the mailing lists give you the next level of transparency, looking inside the project to see what's going
on on the inside.  That's actually used for the component teams.  The component team communication and the
inside of component team communication.

So those are basically, using the media that's available today, but face to face meetings are still very
important.  And one of the things we found is very effective to get people to the next level of productivity
is get together.

So we'd go show up with a project team that was trying to build on Eclipse and we'd work with them on their
problem.  See what we could do there.  Or what we call sprints -- we would get together with other projects.

We'd bring, we have a globally distributed team as we flagged.  We'd get together and actually focus on
specific problems working together face to face.  Conferences are ideal for that, and EclipseCon of as an
example of one conference where there's lots of people doing that.  But our Rational Software Developer
conference here, again, many people with common goals getting together.

ERICH GAMMA:	So I think it's easy to get initial attention to a project.  You can make big announcements,
donate all this stuff.  The big challenge is how you can keep your attention from the community.

It's not a one point in time in thing.  If you have want to have a committee and keep it, you have to
continuously entertain the community.  And by entertain I mean you have to give them continuously things they
can give you feedback on.

So that's what we call life betas.  It's not a beta program that starts one month before you ship.  The beta
program starts at the point we start developing.  And of course the beta program has to be interesting and the
way you make it interesting is, of course, by having continuous improvement.

And what improvements do we put in there?  We listen to the community.  We listen to what they would like,
react to that, and by reacting to that they give us more feedback.

This is how you can keep the attention of the community.  It's not a one point in time but actually you want
to grow that over time to become more and more excited, more and more passionate about what you build.

And I think being passionate about your software, that's the ultimate goal we want to have.  I want to have
passionate users of myself.

JOHN WEIGAND: So it's not just a statement about new features, which is part of it, but there's this
observation of project health where if you want people to give you feedback and be listening all the time, the
thing you're producing, the project, needs to be continuously consumable.

And this fitness chart we have here shows the situation that you need to be in.  So, every milestone, so every
time there's an iteration, the technology needs to be at a place where it's interesting so that people can
consume it and put it to use.

It's the observation it's not just useful on occasion, but good and fit all the time.  We'll go into that in
greater detail later today.

So here's an observation of what we saw in how the community evolved ultimately in size.

You can see the green bar here.  It shows our participation in the project.  We needed to make a big
investment to prime the pump in terms of our development investment to get the user community off the ground
and to enable them, to enable the developer community, people building on top of it.

So that was an up front investment where we weren't as productive in the development sense but very good in
getting people enabled.

But we got to this place where you can see where the community was self sustaining.  The user group
themselves, they could answer their own questions.  They would do further training, and things really took off
at that point.  And you can see that threshold there that occurred to us a year or two into the project.

At that point all kinds of good things happened and all kinds of unexpected things occurred.  If we look at
the numbers, there's two that I think are worth drawing attention to.

One is, if you want to just patrol the bug system; anybody can do that.  But if you want to report a bug you
need to become a registered user.  And in the Eclipse project there's 25,000 registered bugzilla users.

There's 25,000 people who went to the effort of reporting bugs to help us out.  And ballpark numbers, on the
order of two million users of the Eclipse technology.  We're getting feedback from lots of people and lots of
different contexts.

ERICH GAMMA:	For us it was real happiness, right, because the silly questions we no longer had to answer
ourselves, but the developers could really answer the interesting, the deeper, questions.  That's why the self
supporting thing is so critical to getting to this point.

And also initially we really had to kick our developers to participate in this community and say, you can
devote up to half of your time to the community work, meaning answering questions and writing articles.

So now we double clicked and looked a little bit inside into our development process, which is kind of here is
capturing in a mind map.  We don't have the time to really go down deeper into this session, but we have a
session around noon where we double click on this mind map and we look into all these practices we have built
up.

So we found this practice is known by trying things out, things we like we kept on doing.  And if you reflect
on all these practices, then you see you have three main themes, right?

We have practices which allows us to get feedback, continuous feedback.  We have practices in there which
ensure transparency, and of course we have practices which help us to be predictable.

So all in all what you get from that is you know where we are.  We can always be honest and know where we are
in the project, giving all this mechanisms we have in place.  So if you want to find out more about that join
us in the session at lunchtime.

JOHN WEIGAND: So let's reflect on just a few of the things we talked about this morning.  So the first is this
transparency helps on so many fronts, but one, it gives the community access to what we're doing, it enables
them to provide feedback. Without knowing what we're doing you can't provide feedback. 

But the other thing is it provides transparency within the development team itself. So we're not longer in a
situation where because of the globally distributed situation you don't have information flowing between
component teams.  It's all there.  So the same mechanisms that we provided for ourselves we actually share
with the community as a whole.

There's costs.  And let me take the cost point on this one.  There are costs with this.  And it's a real point
that it takes longer to listen to the feedback and reply to the feedback but there's true value in it.  So
it's a trade off but one that really pays off.

ERICH GAMMA:	I just want to make one point.  One quote I got from one of our developers, “thanks that we do
this out in the open, you have all this community focus, this Open Source portal, all our communication
channels.

Before we did this all in the open we never knew actually exactly what they did. But since you have opened up
and have all this communication in place our global distributed development has even improved because we know
what's going on.  There are no more secrets anywhere.  Everything is in the open, fully transparent, which
helped our own project to be more effective.”

So putting this all together, there is one phenomenon that we observed, and I like to call it the village
effect.  I'm from Switzerland and there we have villages.  And one strange thing about villages is people
greet each other.  Little bit strange, if you get there and I forget sometimes too and then I get told,
actually here you greet each other.

But what we find that, if it is open transparent development, what you can get is really this kind of village
effect that people start to greet each other.  They know each other.

And that's also what I hope achieve with Eclipse, you know our developers.  You know some of the faces, names
at least from all this discussion.  So that makes a larger organization act like a smaller one.  And that's a
very appealing thing.

Because when you act like a smaller organization we get the next interesting point and that is kind of like
visible accountability.  I know you, I know what you're doing.  I know what you're supposed to do and if you
don't do it I think I go and ask you, and that's not the point about blaming, right?  I want to find out what
you're doing to understand better what is going on.

And this really ensures that the communication flows become visible and this helps you to react much easier to
change.  But I think it's really important, I think, that you get this information flowing.  You get this
communication, this collaboration going.  That's why I like to live in a village and I actually live in a
village right now.

JOHN WEIGAND: So this transparency we get actually improves the health of the project.  It's not all good
news, however, and one of the things we learned is that when you find something that's unpleasant, actually
being transparent about that is helpful as well.

I'll pick one example to draw attention to,but we were doing a performance push the Eclipse 1 development
cycle, and the results weren't very happy.  We were doing things in an ad hock way and the results weren't
good.

But what we did is we started measuring and transparently showing where we were with our performance.  We
produced graphs daily.  We actually published the results as we got them -- they went out with the integration
build, so it showed up every week.

And we'd look at the results.  We would see places where things going well and other places where things were
poor.  Our original reaction was, that's terrible, we can't let the community see that we have bad
performance.  But then we actually learned.  We had bad performance.  Who are we fooling?  We have work here
that we need to do to make things better.

So by being transparent about this it actually focused our effort.  We actually brought the community into
helping us improve performance.  They provided additional test cases, they found areas where they had others
were we missed performance situations, and overall we ended up in a much better situation than we had ever
been before.

ERICH GAMMA:	Another nice story on this was when we did all this to make our bugs public, we were a little
concerned because we're in the tools based IDEs.  That's a highly competitive area.  If you expose our bugs,
we're afraid will we get abused because then our competition can see where we're weak, for instance?

And this has happened actually.  But was interesting though, the competition has pointed out bugs where we
look were really bad.  But by pointing them out they gave us the great feedback to really address those bugs. 
And what has happened over time, these bugs got really fixed.

So the competition thought they can weaken us by pointing at our flaws, but in fact they motivated us and
encouraged us to fix these flaws.  And if you go the list of what our weak points are, somebody pointed that,
I think nine out of 10 are fixed by now.

JOHN WEIGAND: So Erich's comment is right there on this second point, which is the transparent development
ensures that we're building the products that the community wants.

We're engaging with the community all the time.  When the community finds problems they raise them with us. 
When there's feature needs we work together to figure out what we should do.  So this transparent development
ensures we're in synch with the needs of the people that we're building for

I don't want to make things sound too happy.  This last point is another statement of reality.  We're
improving all the time.  And we're learning all the time.

When we started off we were in a situation where our project plans, for example, we said we'll do our bug
tracking in public.  We'll use the news groups, but project plans we couldn't tell someone our plan because
then we might miss it.

But we learned again let's be transparent in our planning.  We came up with a way to do that that worked well
with the community.  We could establish the feedback loops there.  And just continue to improve it.
[END OF PART 1]