Contents


IBM's approach to open technology

Find out how we're investing in the communities and projects that matter most to the enterprise

Comments

For several years, IBM has been recognized by many within the open source community as a leader in open source. However, that leadership is less well known and appreciated outside of the open source communities in which IBM engages.

From our work with Linux, Apache, and Eclipse to our current work across all layers of the cloud stack, application development, blockchain, artificial intelligence, quantum computing, and machine learning, IBM has demonstrated a sustained commitment to driving innovation in open source, delivering a broad portfolio of offerings based on open source, and helping to build sustainable, thriving communities and ecosystems around those open source projects about which we care the most.

At IBM, we believe that our leadership in open source is a differentiating value for our clients—so much so that we even came up with a catch phrase that we had trademarked to express that value: IBM is Open by Design.

The case for open governance in open source is clear: Projects that are managed under open governance have been found to be more successful, less risky, and have a longer life than proprietary open source projects.

In this article, you’ll learn how IBM has played, and continues to play, a leading role in the evolution of open source—an approach to software development that has nearly completely displaced proprietary software—and how we leverage that leadership to deliver world-class offerings and solutions that benefit our customers.

A long history

IBM has a long history of working in open source. In fact, in many ways, IBM is partly responsible for the open source movement's success. IBM's strong support for Linux—with patent pledges and investment of significant technical resources in the face of some of the legal uncertainty surrounding its use in the late 1990s—brought about a change in posture toward open source for many enterprises. Under the right circumstances, open source such as Linux could be a compelling and trusted adjunct to proprietary software for a growing number of enterprises.

The successful deep collaboration with other industry leaders on Linux led IBM to again work with others to establish both the Apache Software Foundation and the Eclipse Foundation. We went even further with Eclipse by seeding the new foundation with a significant initial code contribution and dedicated technical resources, and in both cases providing legal leadership in authoring the licenses for those communities. On a broad scale, IBM's participation in these and well over a thousand other projects and communities set the tone for open source adoption among enterprises.

While it might come as a surprise to some, one of our key platforms, WebSphere Application Server, was more than 70% open source, comprising more than 700 open source components. Of course, WebSphere was not alone. Many of IBM’s technology offerings and solutions leverage open source—especially those comprising our most strategic initiatives including Cloud, Big Data and Analytics, Blockchain, IoT, Machine Learning, and AI.

Today, it seems as if just about every enterprise is charting a course toward a preference for open source over proprietary offerings. Generally, the first step is to go it alone and try to integrate your own stack. Gradually, once the scope of the challenge is better appreciated through experience, the pattern often shifts to partnering with others who have deep skills and experience, with a thorough understanding of the relevant communities. More on this later.

The case for open governance

One thing IBM has learned through all of this is that those communities that strive for inclusiveness and open governance tend to attract the largest ecosystems and most expansive markets. However, not all open source is created equal, and not all communities thrive.

There is a broad range of open source, and much of it is not truly open. Many open source projects are run by a single individual (or vendor) and are quite closed in their governance, severely limiting contributions from others. Other projects are more welcoming to outside contributions but are still closed when it comes to setting technical strategy and direction.

The key point is that once a project reaches a certain level of success, it usually reaches a tipping point. Without open governance, users realize there is a greater risk of vendor lock-in, or even project abandonment. Users and contributors alike want a voice in the decisions, and if they feel that their voice is not being heard, projects have been known to fail or fork. This typically has a detrimental effect on the ecosystem, which increases risk to users should the community around it collapse.

We have worked hard over the years to establish a solid and respected reputation in open source circles, and especially in those communities where we invest strategically.

The reality is that open technology projects managed under open governance—the sort of open governance found in organizations such as Apache, Eclipse, Mozilla, and Linux—are demonstrably more successful (by an order of magnitude), have a longer life, and are less risky than those projects that are controlled by a single vendor, or are more restrictive in their governance.

IBM often contributes to and consumes from open source projects that are not under open governance. When our clients or our offering teams feel that a project is important enough, we often work with the individual or vendor that controls the project to help them see the value of open governance, and to see the potential for even greater success. If we can effectively bring the project to open governance, we increase our investment considerably to help ensure that project's success, and work to grow the community and ecosystem.

For the good of the ecosystem

IBM knows that a rising tide floats all boats. It isn't enough that IBM succeeds—we need to ensure that many can succeed to ensure a vibrant ecosystem. This reduces the risk that comes with embracing open source for ourselves and, more importantly, our users. Other vendors see this as well, as participants in the renaissance of open source understand that competing on plumbing or commodity is not where the opportunity lies.

We believe that clients who leverage open source should establish a process for managing its use in their enterprise such that they consider not only technical merit and license, but also the evaluation of the community and ecosystem.

We also believe that clients who embark on a journey to integrate open source should have a partner or partners who can help them engage the community. These partners should be committed to the same enterprise interests, and leverage the accrued goodwill and technical eminence in the community to advance a technical agenda commensurate with their interests.

IBM's commitment and contribution to open source is unrivaled in the industry. We have worked hard over the years to establish a solid and respected reputation in open source circles, and especially in those communities where we invest strategically. IBMers serve on a number of open source foundation boards, including Linux, Eclipse, Apache, CNCF, Node.js, Hyperledger, and many others, and we have thousands of IBMers contributing to open source. We value and work toward open governance because we feel that this is the best way to ensure the long-term success and viability of open source projects. Thousands of IBM developers are working every day in open source projects that matter making thousands of contributions to hundreds of open source projects each month.

Not all open source is created equal

Many open source projects on GitHub and elsewhere could be described as "furniture left by the curbside for anyone to take." The code is published with an open source license (or not!) but isn't being actively maintained. Even in those cases where the code is actively maintained, it is likely that a single individual or company handles it. This is all well and good until that individual (or company) decides to pursue something more interesting, or meets an unfortunate circumstance.

There are countless examples of the consequences of investing in open source projects with a single owner. A while back, Facebook declared that it was going to discontinue Parse (a popular mobile development platform), leaving thousands of developers stranded. There are many more cases where a developer released some cool capability as open source, and then, for whatever reason, abandoned or neglected it.

Now, some open source purists might say, "But you still have the software," which is true. But here's the rub: Are you really going to maintain a mobile app development platform or NoSQL database by yourself?

IBM evaluates open source projects by looking closely at five aspects of the project:

  • Responsible licensing—Obviously, we look to understand the open source license that is associated with the technology.
  • Accessible commit process—We seek to ensure that there is a clearly defined process for making contributions that welcomes outside contributors.
  • Diverse ecosystem—We confirm that there are multiple vendors and ISVs that are delivering offerings based on the technology.
  • Participative community—We require that there be a process for contributors to grow their technical eminence in the community.
  • Open governance—We evaluate the governance model to determine whether it is truly open.

Of course, we also look at the technology and assess whether there's an architectural fit, but the technology can usually be fixed and improved over time. The key is whether we believe that there are enough positives to warrant the investment to help bring the project to true open governance that will benefit all.

IBM's journey begins with Apache, Eclipse, and Linux

IBM's journey in open governance for open source started with our leadership in the formation of the Apache Software Foundation in 1999, which was created to provide open governance for the development of the Apache HTTP server. IBM was one of the founding sponsors of the foundation, helped shape the license and governance, and contributed to numerous projects. In addition, IBMers have been serving in leadership roles within the organization and on the ASF board since its launch.

Eighteen years later, there are nearly 200 projects on ASF, many of which have nothing to do with the original HTTP server project. This includes web technologies, XML, web services, document processing, mobile, cloud, big data and analytics, serverless, and messaging. Clearly, we have helped to create a safe place to collaborate and innovate in the open.

Eclipse

In 2001, IBM worked with others to create the Eclipse Foundation with an initial grant of the Eclipse Java IDE framework. We had similar goals for the Eclipse Foundation as we did with Apache: to create a safe place to collaborate and innovate under open governance. Seventeen years later, there are almost 300 projects at Eclipse with equivalent diversity of domain. Again, the open governance provides a venue that attracts open source developers and provides a crucible in which to collaborate and innovate in the open.

Linux

In 2007, IBM collaborated with other key industry leaders to establish the Linux Foundation as a founding Platinum sponsor. Of course, our investment goes well beyond this sponsorship. We have been, and continue to be, a leader in the Linux community by investing hundreds of engineering resources over the years in the Linux kernel and in many of the collaborative projects under the Linux Foundation.

Docker

IBM has been one of the top contributors to Docker over the last few years. Three of our developers have earned the respect of their peers at Docker, Inc., and were named maintainers. When we initially engaged with Docker, it was not yet under open governance. All control over the technical direction of Docker and extension components (such as Swarm, Compose, and the Registry) resided with Docker, Inc. Over the course of time, as the use and popularity of Docker grew, and as the number of non-Docker contributors grew, the pressure to bring Docker under open governance also grew.

This community pressure initially led Docker to establish the Open Container Initiative in June of 2015. As part of this project, Docker contributed libcontainer and the Docker image and transfer format specification. IBM has been one of the top contributors to the OCI since its formation. We have also been working collaboratively across various projects such as Docker, Cloud Foundry, and OpenStack to ensure that runC, the successor to libcontainer, is integrated into each of these. This will ensure that we get portable containers across these platforms. In July of 2017, OCI announced it was releasing v1.0 of its container standards.

Of course, that is only the first aspect of Docker to be brought under open governance. In December 2016, Docker contributed containerd to the Cloud Native Compute Foundation (CNCF). Containerd is a core container runtime component that can manage the complete container lifecycle for containers on its host system.

The latest step is the decoupling of Docker open source and Docker products by the establishment of the Moby project, which will soon adopt an open governance model and be independently managed from Docker Inc.’s product offerings. IBM’s strong leadership and support of Docker has helped chart this course, and the leadership we had earned with Docker transferred to Moby.

Cloud Native

In July 2015, following close on the heels of the OCI launch, IBM, Google, Docker, Weaveworks, Redhat, and many others launched the Cloud Native Compute Foundation with the intention of providing an open governance model for Google’s Kubernetes project. This is a key component of IBM’s technical strategy for cloud and other technologies relating to cloud native applications.

Since that time, the organization has thrived, growing to a total of 14 projects including Kubernetes, rkt, fluentd, containerd (see below), and gRPC. IBM is growing its investment and contributions to CNCF technologies because of their relevance to our cloud strategy, most notably Kubernetes.

Istio

Our partnership with Google on Docker, Kubernetes, and CNCF has also born additional fruit. IBM and Google joined forces with Lyft to start collaborating on a merger of IBM’s Amalgam8, Lyft’s Envoy, and Google’s Service Control to establish the Istio project, a first-class abstraction for routing and policy management for cloud native microservices. Our goal is to eventually move Istio to CNCF to ensure open governance for this important and increasingly popular project.

Hyperledger

In 2015, IBM recognized the incredible potential of blockchain technology, the same technology that underlies Bitcoin. Our research in this space led us to conclude that none of the existing blockchain technology platforms was really suitable for the enterprise. So, IBM set forth to build a new blockchain platform with enterprise requirements in mind—one that could be used in highly regulated environments.

As with the Apache httpd server and the Linux kernel, we recognized that a technology this important should not be controlled by any single vendor. It would never be successful if it was just available from IBM. So we worked with the Linux Foundation to help establish Hyperledger, the fastest-growing project at the LF, ever. IBM contributed 44 thousand lines of code and established the first of the Hyperledger projects, Hyperledger Fabric, under open governance. Since then, there have been nine additional projects incubated at Hyperledger, but the first project to be graduated to “Active” status and the first to reach a 1.0.0 release was Hyperledger Fabric. There were 159 engineers representing 28 companies that contributed to the first release of Fabric—a testament to the value of developing open source under open governance.

OpenWhisk

When Amazon introduced AWS Lambda in 2014, it signaled a potentially game-changing direction for cloud—function as a service (FaaS), or serverless computing. Many companies started to explore this space including Google, Microsoft, and many others. IBM was no exception. In early 2015, IBM Research began work on development of an implementation of a robust serverless capability for the IBM Cloud.

As with our blockchain efforts that led to the creation of Hyperledger Fabric, we recognized that in order for our serverless efforts to be seen as a viable alternative to the proprietary AWS Lambda offering, our implementation needed to be open sourced under open governance so that we could grow a vibrant community and ecosystem around that open source project. In February 2016, we open sourced our serverless platform implementation and named it OpenWhisk. As interest in OpenWhisk grew, we then worked with partners like Adobe and Red Hat to establish Apache OpenWhisk as an incubating project in November 2016.

And much, much more

These are just a few of the examples of IBM's leadership and investment in open source. In just this past year, IBM also helped drive the formation of the OpenStack Foundation, Cloud Foundry Foundation, Node.js Foundation, the Open API Initiatve, and ODPI. We were also behind the creation of a number of Apache projects aside from Apache OpenWhisk, most recently Apache Toree and Apache SystemML. This past year, we initiated more than 50 new open source projects and published countless tutorials and sample applications as open source.

We are also encouraging more and more IBM-developed projects to be published as open source from by the IBM development and research labs. In the past three years of the program, we have published hundreds of new open source projects through this program, and many of those projects have garnered sufficient community interest to graduate to open governance projects at Apache, Eclipse, and the Linux Foundation.

IBM has invested significant resources in adapting key open technologies to our Open Power and Z platforms, including MariaDB, Go (which is the language used by many of the emerging open source projects targeted at the system level), Docker, runC, Linux, OpenStack, Cloud Foundry, Kubernetes, Swarm, and many more. In fact, much of IBM's open source influence emanates from the Systems division.

Focus on the enterprise

We invest heavily in areas such as security, scalability, robustness, live upgrade, globalization, documentation, continuous integration, and delivery in the projects that we consider strategic. Of course, we also invest in the functional aspects of projects that bring IBM innovation into the open. We contribute extensively in other important aspects, including marketing, evangelism, and various board-level committee activities. Finally, we often lead the efforts to define interoperability and portability. We do this because interoperability and portability are critical to the success of any open technology endeavor. After all, that is what the consumer is expecting.

In all of these cases (and more), IBM has invested close to $1 billion and dedicated hundreds of open source development, marketing, and evangelism resources over the past five years. We have initiated many of these projects and worked tirelessly to help the organizations and the projects they hosted to define and achieve success. We do this because the value that IBM derives from these projects and organizations extends beyond the open source software itself. Much of the benefit comes from the vibrant communities and thriving ecosystems that develop around these centers of gravity for open technology. Our offerings succeed in proportion to the success of the projects in which we invest.

We focus on driving interoperability, portability, and many of the "-ilities" that matter most to the enterprise. We also focus on contributing IBM innovation upstream, and prefer that to keeping it in closed source when it pertains to these strategic projects such as OpenStack. Yes, there is IBM value-add in our offerings, but we don't invest in interoperability and portability so that we can then throw those attributes out the window when we deliver our offerings based on those open technologies. Our strategy is based on leveraging a solid foundation of open technology and ensuring that the interfaces (APIs and SPIs) defined by these technologies are fully exposed and not hidden or inaccessible behind some proprietary veneer of value-add.

We strive hard not to fork the community code by creating an "IBM Hyperledger Fabric" or "IBM Kubernetes." The Hyperledger Fabric in IBM Blockchain Platform is the same Hyperledger fabric that is released by the Hyperledger organization. The Kubernetes that we integrate into IBM Cloud is the same code that is released by the Cloud Native Computing Foundation. The Docker included in IBM Container Service is the same Docker that that community releases. The Cloud Foundry is the same code that is released by the Cloud Foundry Foundation. The IBM value-add is that we integrate all these capabilities developed in open source to realize the IBM Cloud.

We invest in the community code for these strategic technologies and ensure that fixes and new features are up-streamed rather than adding extra complexity and effort on IBM's part to maintain an independent, differentiating version. Where we desire to add extensibility that can leverage IBM (or others') differentiating capabilities, we work within the community to create the necessary API or SPI. We also invest in making sure that those extension points are not abused to create a potential for lock-in.

Conclusion

As you can see, IBM takes open source seriously. When we engage in a project, we bring a focus on those aspects that matter most to the enterprise: interoperability, portability, security, scalability, and accessibility. We do this by investing in the community and helping to shape programs that can deliver those characteristics that matter to our clients. We value open governance because it ensures the long-term success and viability of those projects that form the foundation of our enterprise offerings and solutions.


Downloadable resources


Related topics


Comments

Sign in or register to add and subscribe to comments.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Cloud computing
ArticleID=1031307
ArticleTitle=IBM's approach to open technology
publish-date=01172018