Open standards, open source, open minds, open opportunities

People stop me all the time and they ask, "Bob, what is an open standard?" And I tell them, well it depends, but if you read my blog you would have an idea of some possible answers. They then look really scared and I never see them again.

All kidding aside, I have tried to force some examination of what the word "open" means and warn against its use as a marketing term. I've also put forth the idea that we should consider "closed" and "open" as two ends of the spectrum. Here "closed" means "it's mine, you can't have it" and "open" means "here take it, do whatever you want with it." Most standards fall somewhere in between, with very few things being all the way at one end or the other. This language is still imprecise, so let's take some time to look at what we might really mean by this use of the word "open" in the extreme sense of "purely open." To be clear, I'm not the only one who has looked at these types of things. In particular, my IBM colleagues Tom Glover and Chris Ferris have thought and written quite a bit about this, so I'll attribute the really good ideas to them and save the bad ideas and mistakes for myself.

I think there are five areas we need to look at:

1. Development
2. Maintenance
3. Accession
4. Implementation
5. Modification by Others

Let's consider them in turn. Remember, this is meant to describe the extreme meaning of "open." After this initial discussion I'll return to some things to be considered when you decide how open you need a standard to be.

Development: A purely open standard is created in a manner that allows anyone who wishes to participate to do so without discrimination. All proceedings and records of proceedings are public and decisions are made by consensus. No one can prevent anything that actually happened during the development of a standard from being recorded in the minutes. There are no secret agreements. No one person or organization can veto a decision.

Maintenance: Once created, a purely open standard is maintained by a community under terms similar to that which developed the standard. In addition, the community has a responsibility to maintain backward compatibility if it decides to do so. That is, openness gives the maintaining community the right to make significant modifications, but since membership is not restricted, the community may get enlarged at times by a sizable group of people who may either want to force the changes or maintain the status quo. (This is just democracy in action: if you get more people to vote for you, you win.) So while significant changes may be made in the evolution of a standard, it is not done at the whim of a single vendor unless that vendor can garner significant independent support. As in the development of a standard, all votes or positions taken during consensus creation are recorded for all to see.

Accession: Anyone who wishes to have a copy of a purely open standard may have one at no cost. When possible, the standard is available online and the document format used to represent the specification is an open document standard (I know this is recursive, but you know what I mean).

Implementation: A purely open standard can be implemented by anyone in any way desired with no payment of royalties.

Modification by Others: A purely open standard can be mined for its good ideas and used in part or in whole in other standards. In particular, a profile can be created that states which parts of multiple standards should be used together to achieve a certain end (for example, a subset of a set of standards for use on mobile phones).

What are some of the problems with these criteria?

• Standards development organizations (SDOs) have expenses. They may need to pay for permanent staff, lawyers, websites, conferences, offices and other ongoing expenses of formal organizations. Where is this money to come from if there are no membership fees or costs for acquiring standards? Perhaps an SDO survives by accepting voluntary donations from participants with a "give if you can and want to" model. Giving more should get you no more privileges than if you give nothing. If this still doesn't generate enough money, then the SDO may go out of business. It also begs the question: if no one is willing to support the effort, why does it exist in the first place?
• If you can't get a copy of the standard for free, how much is too much? If it costs you $5, is that ok? How about $500? What if you have 2000 developers that need a copy? I tend to think free is really the best answer here, at least for an online version.
• Standards efforts need to be sustainable in order to maintain specifications. This brings us back to the funding question, but also makes us think about how one organization might pick up an effort when another decides to no longer pursue it. Note that because of the community nature of purely open standards creation, it is likely that some group can be found to maintain the standard, whereas a closed de facto standard might vanish if the owning vendor goes out business, gets sold, or simply decides it wants to something different.
• If membership is completely open, you have to make sure you have rules that prevent an organization from "stuffing the ballot box," that is, sending many people to vote for or against a proposal.
• Can you really be sure that no one can put up a barrier to implementation of a standard? In a purely open standard, the creators of the standard can presumably be prevented from putting up such roadbloacks, but what about others? Do you allow legal concepts such as reciprocity and defensive termination to be in license agreements for purely open standards? Can they actually help make it purely open?
• In a purely open standard, license agreements do not have tricky or cute language that prevent open source implementations. Should a license just allow implementation of the standard or should it also allow arbitrary derivative works that go beyond the scope and intent of the standard?
• In order to maintain control of a standard it creates, an SDO may copyright the specification and so prevent subsetting or extension. However, certain permissions may be included that liberalize this. Which conditions are acceptable?
• You need to maintain some sense of an official version of a standard, so you don't want to allow such looseness that someone claims to implement a standard if only a small subset is actually handled. You also want to prevent people from unilaterally extending a standard, thereby preventing interoperability and allowing undo advantage to one implementor. Here an open source reference implementation will help and do two things: 1) provide a test case for compatible implementations, and 2) allow people to quickly use a standard without waiting for proprietary implementations.

If you have read this and conclude "ha, we can't really get an open standard so we'll muddle through and do what we want", then you have misunderstood what I am talking about. Since most standards fall between closed and this "purely open" characterization, you need to decide what is important to you. For example, a government may decide that it really cares about Maintennace, Accession, and Implementation and will try to use standards that maximize those characteristics while being willing to concede a bit on Development and Modification by Others.

For some kinds of standards efforts, it may be easy to obtain patronage for the effort that therefore allows anyone interested to join. The nature of the effort may preclude troublemakers from taking advantage of the rules. That said, you need to be careful and precise in the membership and participation rules and always think about how someone might abuse them. To paraphrase Bo Diddley when speaking of the music business: "Trust no one, except your mother. Even then, keep an eye on her." Ideally, standards would be created in an environment of trust. You just sometimes have to legislate trust.

In practice, I think it is probably easiet to tell when a standards effort is less open than you think it should be. Here are some danger signs:

• control by a single vendor,
• overly complicated license agreements,
• license agreements that reserve certain special rights to individuals or vendors,
• license agreements that prevent some kinds of implementations,
• overly complicated procedural rules that can allow people to be less democratic than they should,
• a history of disregard for backward compatibility,
• costs of participation that exclude individuals or small organizations,
• high costs of obtaining copies of the standards,
• standard specifications not being openly available online,
and
• for XML-based standards, allowance for proprietary, vendor-specific extensions.

Some of these are worse than others: you will need to decide for yourself how open you need the standards you use to be. As I've suggested in other entries, I hope that we eventually come up with a grading system that allows us to compare two specifications side by side and see which one is more open. I hope that what I've written here is a start on such a system. I believe we're getting more and more open every day. Such a grading system will help you tell whether a particular standard is more open than it was last year and whether it is on a path to be more open next year.

---

I consider this a work in progress and may update it from time to time. RSS

[Read More]

In an opinion piece published yesterday ("State's Open Document Dispute Raises Legacy Questions"), eWeek's Editorial Board has come out in support of the decision by the State of Massachusetts to support the OASIS OpenDOcument standard. The piece concludes

Government is trusted with preserving for its citizens the integrity of their public records. Because most, if not all, permanent records will eventually move from that universal data formatapero electronic formats, IT bears responsibility now and in the future to preserve access to those records. Consistent with that responsibility, we think Microsoftnd all software vendorshould support accepted and ratified open formats.

We think Massachusetts officials are right to take a stand.

[Read More]

Stephen Walli, former Microsoft exec, has posted a very interesting and thoughtful blog entry "Microsoft, Massachusetts and a Standards Primer". Here's a sample

Proprietary specifications of de facto technologies are NOT standards. They don't exist to encourage multiple implementations of the technology for the consumer, but rather to encourage multiple complements to the technology, therefore increasing the value proposition of the de facto technology for the vendor. They serve a very different economic purpose. All of Microsoft's claims about being "the standard" are the antithesis of the standardization efforts the Commonwealth want to support.

[Read More]

The story in Massachusetts just keeps unfolding in new and interesting ways. See ZDNet's latest blog entry "Carr gives Microsoft a taste of its own OpenDoc medicine (and I pile on) " where he talks about Harvard Business School's Nicholas Carr take on the goings on.[Read More]

It is Saturday morning as I start this, and I am sitting in the Qantas departure lounge in Melbourne. It is raining, just as it was when I arrived in Sydney last Sunday (I think there is a song in there). In between, the weather has been fine but variable, if only because I've visited four cities this week.

That's a lot of time spent in airports but, with only a couple of glitches, I got everywhere I was supposed to be. I had virtually no time for sightseeing, though I did manage to walk around a bit and visit bookstores in three of the four cities. I am tantalizingly close to the end of Harry Potter and the Half Blood Prince. It is too big a book to carry around, but I did manage to read a couple of chapters while browsing. It will probably take me about thirty minutes to finish the book when I get back. This is the sixth book in the series and when I'm done with this one I really need to go back and read number five again. I read it very quickly and don't remember very much of it. As a rule, I don't like rereading books I've already been through since I feel I only have a limited time in my life to read books and time will be lost if I keep reading the same ones over again. One that I have read multiple times is Crossing to Safety by Wallace Stegner. The combination of the story of the friendship, the summer by the lake, and the academic life makes me nostalgic for the story every couple of years.

Most of my casual reading this week was taken up by the book The Living Great Lakes : Searching for the Heart of the Inland Seas by Jerry Dennis. While it's odd that I would spend so much time reading about the North American Great Lakes while in Australia, it was a great read. I've read several of these boat travelogues and I'm still not sure whether it's because I would like to take such a trip or whether I just prefer to live vicariously. Certainly none of them describe the trips as very easy and, indeed, this one describes a near-death experience in Long Island Sound right near the end of the trip. I've tried to convince my family of the possibilities of renting a boat and doing a trip on the Erie Canal some summer, but I've not gotten any takers.

In my travels this week, I started to think about how I would answer the question "On average, where have you been?". Even forgetting the rotation of the earth around its axis and around the Sun, and forgetting its general path within the universe, there is some sense that at every single moment you have been physically somewhere. So if you "average" all those positions, where were you? What answer do you get if you ask the question with a starting point of when you were born, or when you became 18, or since you got married? Similarly, you can ask for this average position during your college years, during the time you lived in your first apartment, while you had a particular job, or while you were dating someone. I'm sure you get the point. Except for trivial cases, you really can't compute this, but it's fun to try to estimate it, maybe with surprising results.

To finish off these ramblings, I'll let you know that I am 9 hours and 41 minutes from LA, 2181 miles from Melbourne and we should be crossing the International Dateline within the hour. I'm listening to Coldplay's A Rush of Blood to the Head album. I'm using Eclipse 3.1 to directly write the HTML for this text (gee, an open source application creating a document using a royalty-free W3C open standard). Though it is only 4:21 PM Melbourne time, I should get some rest since I have two more flights in order to get home late Saturday night.

[Read More]

See the Boston Globe story "Microsoft fights bid to drop Office software".

Remember, the Massachusetts ruling on open standards does not rule out the use of Microsoft software: they have the choice to implement them. Note that Microsoft decided to copy the governor: perhaps Massachusetts citizens who support the decision to use open standards will do the same.

[Read More]

Via David Hornik's blog post "Pandora and Persistence", I discovered Pandora, which generates custom radio stations based on the structure and characteristics of music you like. Give it a try.[Read More]

See "Massachusetts Back-Room". Nice job, Tim. This is required reading.[Read More]

This is a draft, I'm not supposed to cite or quote it, but it is from the United Nations Development Programme's Asia-Pacific Development Information Programme: "Free/Open Source Software: Open Standards". They discuss the OASIS OpenDocument Format along with many other things. Interestingly, the report itself is available in both PDF and OpenDocument.[Read More]

"Technical neutrality" is a term used these days, usually by government agencies though also in private enterprise, to indicate the commandment "Thou shalt state no preference between open source and proprietary software." When used in the most well-meaning sense, the usual intent is that ideology, politics, economic and market theories, marketshare, and de facto product standardization should not influence your choice of the best software for the intended use, now and in the future.

It is not separate from conditions that might state that the software should fully support open standards, because factors like real, broad interoperability (vs. interoperability with a particular vendor's software) are absolutely part of the criteria in determining what "best software for the intended use" means.

There are some variations to the statement as I've given it that I would like to point out in case you hear them. These are a little over the top to make my point, but I think you will recognize many of the arguments. In any case, I want to make clear that these are all my personal opinions based on my own experiences with customers around the world.

"Thou shalt state no preference between open source and proprietary software"

Variation #1: "Thou shalt state no preference for open source over proprietary software"

This is usually used by critics of open source software, who usually coincide with the group of strict proponents of proprietary software. While seemingly neutral, it nevertheless leaves you with the impression that protection is needed against the open source upstarts. This is remarkable, because in most product categories, proprietary software vendors usually have greatly larger revenue today.

Variation #2: "Thou shalt state no preference for proprietary over open source software"

This one is by the proponents of open source software and attempts to make sure that the new open source model gets a fair consideration vs. the established and installed proprietary software business models. Most people who adopt such statements are explicitly declaring that they are open source advocates and are willing to settle in for a fight.

Variation #3: "Thou shalt state no advantages of open source over proprietary software"

This is the most insidious of all the variations and is a nastier form of #1. Not only should you not prefer open source, you shouldn't even talk about the potential advantages. The reason why this is so troublesome is that it protects the status quo: if the known and incumbent software is proprietary and you are not allowed to do a full comparison with open source, the incumbent will usually win, except in the situation where it must be tossed because it is a complete failure. This amounts to an incumbent controlling the media to not allow a challenger to compete on an informed, level playing field.

Variation #4: "Thou shalt state no advantages of proprietary over open source software"

This is here for completeness, though I have never seen it used in practice. Clearly it is an attempt to booster open source software, but because of the installed base of proprietary software, I doubt it would be effective in practice today. If the roles were reversed, if more open source software was used than proprietary, it would be just as nasty as #3.

To be clear, if you do favor open source or proprietary software, using one of these variations might appeal to you. My point in this entry is to examine what you should look out for if your intent is to be evenhanded with both of them.

Rather than "technical neutrality," I agree with my IBM colleague Ros Docktor that what we need is the idea of "technical inclusiveness." This means that rather than trying to play word games to improve the consideration of either open source or proprietary software, consider the full and explicit technical and economic advantages of each based on what you are trying to accomplish. For example, in the Massachusetts case, the focus is on getting the best products that implement the OASIS OpenDocument format. Because it is an open standard, neither open source nor proprietary software has an advantage, especially since we are still in the early days of implementation of the latest standard. Here the issue is to choose the best piece of software that implements the standard, open source or propietary, and let the best contender win. In fact, support of the standard will allow multiple implementations to co-exist, and this can be a good thing if different groups nned to use different software.

Examine all the features of all the software you have considered, just like you used to do when comparing several proprietary products. Pay particular attention to interoperability features: these will only grow in importance from now on. If a software provider (open source or proprietary) tries to lock you into their offerings via non-standard data formats when standard ones exist, find out when the standard will be implemented and get a commitment. In the case of open source software, perhaps even assist with the implementation of the standard.

My first statement "Thou shalt state no preference between open source and proprietary software" shows far more symmetry than any of the other variations. In the same way, you should understand that many of the considerations when acquiring software are independent of whether the software comes from a proprietary or open source provider.

Finally, the world is becoming more of a hybrid mix between proprietary and open source software every day. You typically don't need to make choices between using all of one or the other. Even products like IBM's WebSphere Application Server contain a mix of proprietary and open source components, though you should not be surprised that much of the latter is there to support the implementation of standards.

Keep an open mind; watch out for nuanced, asymmetrical statements from anyone; and be more than neutral, be inclusive.

[Read More]

With the increasing use of open standards and open source, often brought about because of the transition to service oriented architecture (SOA), it's important to stop and think strategically about your IT systems from the "open" perspective. To me, this means trying to temporarily put aside some of the other factors among the many that concern IT, such as legacy systems, vendor relationships, hardware and software refreshes, and just think about what you need in terms of increased interoperability and better factored components. To that end, I often ask customers these types of questions:

1. If you could discard everything and start from scratch, what kind of system would you build to meet your current and long term needs?
2. In that ideal system, what would be the role of open standards?
3. In that ideal system, what would be the role of open source?
4. In that ideal system, what would be the role of your current vendors and service providers? Would you use other vendors or service providers?
5. What are you doing to transform your current infrastructure to your ideal system in five to ten years?

To turn around the first sentence above, if you are the midst of transforming your enterprise to an SOA, you should also be changing over to a greater use of technologies built on open standards and, if it is right for you, open source. Web services is here today: you need not wait for the next generation of operating systems or applications to get started today. Anybody who tells you that SOA or web services will only really start with that new operating system or application is speaking with their best interests at heart, not yours. Be willing to mandate support for open standards in the systems that you are buying or having built for you. Use the Harvard Roadmap for Open ICT Ecosystems to measure your current "open maturity" and also use it to understand where your system is heading. If your system will not be more open a year from now, I would assert that you are heading in the wrong direction.[Read More]

Sometimes I think I play guitar like a robot, but this little device was pointed out to me by my brother Richard: it's a Robotic Guitar Tuner that takes that last little bit of effort and thinking out of tuning your guitar. In some former entry I pointed out that if electronic guitar tuners had been around 30 years ago I would now have been playing guitar for thirty years (though I'm guessing I would only be marginally better than I am now). With this device, who knows where I would be today?

Where I actually am is in the Admirals Club at LAX waiting for my flight to Sydney, so I have time to think such deep thoughts as I've expressed here ...

[Read More]

The "Roadmap for Open ICT Ecosystems", sponsored by IBM and Oracle, was released this morning. It is a very good read. From the Introduction:

Technology transformative power has always been a source of great expectations and challenges. Today, globalization, fueled by information and communication technologies (ICT), is rapidly changing every society. Our drive towards globalization creates a new set of unique demands on government, business, and our everyday lives. Increasingly, decision makers in all fields are looking to technology to provide solutions and drive desired changes by commingling local, national and global resources in innovative ways.

The fusion of technology and globalization has also produced a new way to adapt, innovate and grow in our changing world. A potent combination of connectivity, collaboration, access and transparency or openness is emerging. Governments and enterprises around the world are embracing it. This openness is helping governments, companies and individuals respond to the increasing requirements of our on-demand, high-speed world. As openness impacts an ICT ecosystem, it becomes a catalyst for unleashing newfound comparative advantage, invention, social development and market opportunities.

The Berkman Center for Internet & Society at Harvard University, with the support of IBM Corporation and Oracle, has facilitated the creation of this ROADMAP FOR OPEN ICT ECOSYSTEMS. Our hope is to provide policymakers, managers and other stakeholders from industry and civil society a user-friendly tool for understanding what open ICT ecosystems are, why they are embraced and how to evolve them. As a result, we hope to change how people see and manage ICT ecosystems and innovation.

Also see the related New York Times article "Plan by 13 Nations Urges Open Technology Standards". From the introduction:

In a report to be presented at the World Bank today, a group that includes senior government officials from 13 countries will urge nations to adopt open-information technology standards as a vital step to accelerate economic growth, efficiency and innovation.

The 33-page report is a road map for creating national policies on open technology standards, and comes at a time when several countries - and some state governments - are pursuing plans to reduce their dependence on proprietary software makers, notably Microsoft, by using more free, open-source software.

I've seen various statements around the web about how "big business doesn't want standards" and so forth. Remember, this was sponsored by IBM and Oracle.[Read More]

This is news from last month, but better late than never. Here is the current composition of the OASIS Board of Directors following the most recent election: Chris Kurt of Microsoft, John Jackson of General Motors, Claus von Riegen of SAP, Eduardo Gutentag of Sun Microsystems, Michael Winters of IBM, Edward Cobb of BEA Systems, Mike DeNicola of Fujitsu, Patrick Gannon of OASIS, Robert Glushko of the University of California at Berkeley, Frederick Hirsch of Nokia, and Jeff Mischkinsky of Oracle.[Read More]