By changing how business and society run, cloud computing is opening up huge avenues of innovation. Developers are now combining systems of record with systems of engagement, and a new style of cloud-based application is emerging: systems of interaction. For these applications to be sustainable, cloud computing needs to be built on open source and open standards. This article lays the groundwork.

Share:

Angel Diaz, Vice President, Software Standards and Cloud Labs, IBM

Angel Diaz author photo

Dr. Angel Luis Diaz is Vice President for IBM’s Software Standards and Cloud Labs. Since 2009, Dr. Diaz has led his team in transforming how work is done by software, driving new value through interoperability and portability, based on standards and open source. In 2013, IBM’s Cloud Labs joined Dr. Diaz’s organization. Under his direction, the labs continue to bring direct value to customers by optimizing their cloud environments and by providing expert solutions that address their most complex challenges. When not innovating in customer environments, the Cloud Labs are working to extend IBM’s capabilities across the breadth of its cloud offerings, using technologies that bring interoperability and efficiency to the cloud.

Prior to this role, Dr. Diaz was Vice President of WebSphere Business Process Management and Connectivity, where he expanded IBM’s middleware footprint by cultivating consistent growth of the business segment. In previous leadership roles, Dr. Diaz was the Director for WebSphere Business Process development and the Web Services Product Manager, where he was responsible for a range of IBM Software product offerings.

Before joining the IBM Software Group, Dr. Diaz was a member of IBM's Research staff and Senior Manager, where he led a range of projects that included technologies such as XML and web services. In 1998 Dr. Diaz was co-chair and co-author of the first XML standard, the World Wide Web Consortium’s (W3C) Mathematical Markup Language (MathML). Dr. Diaz subsequently served on a variety of other W3C activities. In 2002, Dr. Diaz collaborated with the Organization for the Advancement of Structured Information Standards (OASIS) to initiate the first two web services standards: the Web Services For Remote Portals (WSRP) and OASIS Web Services For Interactive Applications (WSIA). As a result, Dr. Diaz was nominated to the OASIS Technical Advisory Board, which set the technical direction for future OASIS work.

Dr. Diaz received his Ph.D. in Computer Science (distributed computing, programming languages, and computer algebra) from Rensselaer Polytechnic Institute. You can contact Angel at aldiaz at us.ibm.com.



Chris Ferris, CTO Cloud Interoperability, Software Group Standards Strategy, IBM

author photo - chris ferrisChris Ferris is an IBM Distinguished Engineer and CTO Cloud Interoperability in the Software Group Standards Strategy organization. He has been involved in the architecture, design, and engineering of distributed systems for most of his 30+ year career in IT and has been actively engaged in open standards development since 1999. Chris is currently leading IBM's technical engagement in the OpenStack community, and providing technical leadership on a number of open initiatives relating to Cloud.



23 July 2013 (First published 27 April 2013)

Also available in Chinese Russian Japanese Portuguese Spanish

Introduction

Cloud computing is changing the way we think about technology, and it’s no passing fad. Consumers are using the cloud to store music. Startups are turning to cloud to get up and running without huge investments. Big businesses and governments are relying on clouds to make more data more accessible. Cloud computing is changing how business and society run, and it's opening up huge avenues of innovation. We are looking at how developers are now combining systems of record with systems of engagement, and we see a new style of cloud-based application emerging. These are systems of interaction.

For these applications to be sustainable, cloud computing needs to be built on open source and standards. Wide adoption of open source software and open standards should be everyone's goal. It means customers won’t have to fear vendor lock-in, and organizations can participate in a growing market that welcomes a wide variety of cloud technology and service providers.

We've learned through our experience that open source and standards allow developers to share information more quickly and easily, and at lower costs. This leads to greater innovation. We are at an inflection point. We're focusing the industry on important standards for interoperability, and their open source reference implementations will:

  • Ensure that end users have a strong voice in establishing and adopting cloud computing paradigms
  • Reduce barriers of entry into cloud computing, such as development skills and freedom of choice
  • Increase the long-term viability of today’s cloud investments
  • Prevent unnecessary architectural complexity and fragmentation

IBM's open cloud architecture

At IBM Pulse, in March 2013, IBM announced its open cloud architecture and our plans to base all of our cloud offerings on OpenStack, starting with IBM Smart Cloud Orchestrator. This announcement was the culmination of over a year's worth of investment in product integration with OpenStack, as well as significant contributions to the OpenStack community to help improve the ability of OpenStack to meet enterprise and cloud service provider requirements. Looking at adoption results, we see that our efforts, as well as those of the vibrant OpenStack community as a whole, have been successful.

Evolution of OpenStack

To get a sense of just how far OpenStack has come, and how it is being leveraged in the enterprise, visit the OpenStack SlideShare channel.

Today, with over 189 organizations and over 9,100 individuals participating, OpenStack is the largest active open source, cloud project community in the world. This massive global collaboration of developers and cloud computing technologists is working to produce a ubiquitous Infrastructure as a Service (IaaS) open source cloud computing platform for public and private clouds.

IBM is proud to be a highly active member of the OpenStack community and proud of the many community achievements reflected in the latest release, Grizzly. OpenStack is well poised to deliver massive portability and interoperability for IaaS applications.


Moving up the stack

We haven’t been exclusively focused on OpenStack, however. Our open cloud architecture extends all the way up the stack, from IaaS, through PaaS and SaaS. In addition to all of our efforts relating to OpenStack, IBM has been hard at work exploring a wide variety of technologies relating to our open cloud architecture, particularly as it extends into the PaaS and SaaS layers, weaving together systems of engagement, systems of record, and the Internet of things, enabling what we call systems of interaction. We collectively call these explorations, Bluemix. I’ll have more to say on Bluemix, in a bit. But first, I’d like to share some more insight into our open cloud architecture.

Today, there is significant pressure on lines of business to create new business value based on mobile, social, big data, and analytics capabilities. These new applications are fueled by the emergence of new, highly iterative development models that use service composition, open architectures, open source components, and polyglot programming models to rapidly deliver solutions. Two additional trends are key enablers for this new style of applications, notably DevOps and Cloud-based PaaS offerings, such as Heroku, Cloud Foundry, and OpenShift. DevOps enables clients to deliver business outcomes in an agile, iterative, and incremental fashion: applications are delivered in days or weeks, rather than the typical months or years. Finally, to achieve such development efficiencies, PaaS technologies are evolving that enable developers to quickly assemble substantial middleware components hosted in the Cloud extremely quickly (in seconds).

Multi-tenant, elastic (“cloud native”) software services are the fuel of this style of development. We have observed the evolution and maturation of the API economy over the past decade, exemplified by over 10,000 services available to the cloud-based developer from sites such as programmableweb. Given IBM’s depth and breadth of software-based services, specifically in the area of mobile, social, and analytics, it is imperative that we provide implementations accessible to developers at the click of a button.

While there are certainly lots of proprietary offerings that attempt to cover various aspects of this new style of development and operations, we have observed that the open offerings deliver real value to the customer, and are more likely to be embraced by developers.


What do we mean by "open"?

So, what do we mean by "open"? Open can take on a number of meanings.

One meaning of "open" is open source software (OSS), such as OpenStack, or one of the many projects hosted by the Linux, Apache, or Eclipse foundations, or simply hosted on Github. Of course, not all OSS is created equal. It is important that the OSS component have an appropriately open license, such as Apache, Eclipse, or MIT, etc. and also that it have an open governance model that welcomes contributions and innovation from anyone, anywhere, without prejudice. Further, it means that the OSS component can be easily extended, and easily consumed and integrated, by means of relatively stable, well-documented APIs, or that it offers a pluggable architecture that allows for a wide variety of plugins or drivers.

Another meaning of "open" is the implementation of open interoperability standards, such as those emerging from the W3C, IETF, OASIS, or OMG, to name a few. Of course, another form of standard is the de facto standard, which is typically established by virtue of the market embracing a particular solution, regardless of whether it's proprietary or open. Ideally, to be open, we should avoid proprietary de facto standards under the control of (and which tend to benefit) a single party.

Increasingly, a symbiotic relationship is emerging between open source and open standards. For instance, proponents of emerging open standards often turn to open source to implement the standards as a means of providing a reference, as well as providing a vehicle to drive adoption. As another example, communities developing an open source project formally document their project’s APIs and/or protocols when they feel that things are stable.


IBM's open cloud architecture

So, back to IBM’s open cloud architecture. What do we mean by this phrase? At a high level, it means that our cloud architecture is based solidly on a composition of open source components and where possible, implements open standard formats, APIs, and protocols that deliver both interoperability and value to the customer. This open architecture has a number of benefits, both to IBM, as well as to our customers. For IBM, we benefit from the unparalleled innovation we find in open source communities such as OpenStack. Additionally, we benefit from the vibrant ecosystem of ISV partners that grow from these open source and open standards initiatives. Our customers benefit from the ability to leverage the open source and open standards APIs and to extend them—either through third-party capability (leveraging the vibrant open source ecosystems we have chosen), or through innovation and extension of their own.

By using the same open cloud architecture and technologies across our cloud portfolio, addressing private, public, and hybrid cloud solutions, we also benefit from interoperability and portability, not only of workloads but of capability, between and across these various offerings, which benefits our clients.

In fact, customers who choose open standards-based cloud computing are on the right course to take advantage of new opportunities. Specifically, they will be able to harness their interconnected data with high-value business analytics across traditional and mobile devices.


IBM's commitment to open cloud architecture

To help make this happen, IBM is applying its experience in supporting and validating open standards. From Linux, Eclipse, and Apache to cloud computing, IBM continues to work closely with the open source community. To make an open cloud architecture a reality, IBM worked to:

  • Create a 400-member Cloud Standards Customer Council that grew from about 50 members at launch
  • Drive related cloud standards, such as Open Service for Lifecycle Collaboration, Linked Data in the W3C, and TOSCA in OASIS, to enhance cloud application portability
  • Dedicate more than 500 developers on open cloud projects
  • Partner closely with the OpenStack Foundation, along with its over 9,100 members from 87 countries and 189 corporate sponsors

Let's get technical

Figure 1 highlights much of open source and open standards-based technologies that comprise our open cloud architecture, today. Of course, because it is open, it will evolve and be extended over time, by IBM, by the open communities that drive the open source and open standards initiatives, by customers, and by our partners.

Figure 1. The emerging open cloud architecture
Image showing the open tools and technologies that support the emerging open cloud architecture
Figure 2. Evolution of IaaS to a more dynamic, analytics-based Software Defined Environment
Diagram showing IaaS as a more dynamic, analytics-based Software Defined Environment

OpenStack

As we announced at IBM Pulse, the foundation of our open cloud architecture is based on OpenStack, which supports a wide variety of virtualization platforms (KVM, XEN, VMware, Hyper-V, PowerVM, ZVM, etc.) as well as bare metal, and which itself runs on most Linux distributions. We chose OpenStack not only for its architecture (which we found compelling), but even more importantly for its development community, open governance, and vibrant and growing ecosystem of providers, plugins, and add-ons.

Software Defined Network

A fundamental component of OpenStack is the OpenStack Networking service. This service provides an abstraction layer API for managing the network. Software Defined Network (SDN) technologies, such as OpenFlow controllers, can be enabled as plugins to the OpenStack Networking service.

In April, the Linux Foundation announced the formation of the Open Daylight Project, the largest open source SDN project to date, with broad industry sponsorship, including IBM as a Platinum sponsor. The Open Daylight Project has a mission to create an open SDN platform, based on open standards, including OpenFlow. As the Open Daylight Project matures, we can expect that the OpenFlow controller developed by the members of the Open Daylight Project will be enabled as a plugin to the OpenStack Networking service.

Deployment automation

For deployment automation, our offerings will typically support Chef cookbooks and recipes. However, we also support Puppet and Juju so that clients can leverage the automation tools they prefer, or already use. This is another key characteristic of our open cloud architecture: it enables customer choice. Chef, Puppet, and Juju are each open source initiatives of their own, and each has a growing ecosystem of recipes, manifests, and charms, which are often, themselves, open source. The openness of these tools, and the vibrancy of their respective ecosystems, benefit both IBM and our clients.

Linked Data and OSLC

Finally, bridging the foundation software-defined environment layer with the compositional layer above it (often referred to as PaaS), we use an emerging open standard, which is based on the W3C’s Linked Data, called Open Services for Lifecycle Collaboration (OSLC).

OSLC is an open community that is working to make software lifecycle tools interoperable by enabling them to share data with one another. This effort covers a wide range of development, operations, and management disciplines and an equally diverse set of tools. The goal is to enable tools to be combined in a way that connects and streamlines workflows to reduce the effort, frustration, and cost throughout the cradle-to-grave lifecycle of an application.

OSLC was inspired by the way data is shared on the web and is based on W3C’s Linked Data. Linked Data is essentially a methodology for publishing structured data on the Internet so that it can be interlinked with similar data, improving its accessibility and usefulness. The idea for Linked Data was first articulated by Tim Berners-Lee and rests on four basic principles:

  1. Use URIs as names for things.
  2. Use HTTP URIs so that people can look up those names.
  3. When someone looks up a URI, provide useful information, using the standards (RDF*, SPARQL).
  4. Include links to other URIs, so that they can discover more things.
Figure 3. Developer-centric platform, marketplace, and services in a Cloud Operating Environment
Diagram showing Developer-centric platform, marketplace, and services in a Cloud Operating Environment

PaaS fabric

Continuing up the stack, the industry is exploring a number of open alternatives for the compositional PaaS fabric. IBM recently announced that it was joining the Cloud Foundry community, and will be contributing towards the development of the open source project. We have been integrating Cloud Foundry into our emerging cloud development environment, Project Bluemix, an implementation of our open cloud architecture. Cloud Foundry provides a platform that allows developers to deploy, manage and scale cloud-based applications quickly, and easily. It supports a wide variety of runtime environments (including Java, JavaScript, Python, Ruby, etc.), popular frameworks (such as Node.js, Tomcat, and Ruby on Rails, etc.), and a rich variety of services (NoSQL databases, queuing, messaging, caching, analytics, social media, big data, etc.). It also provides an open architecture for the incorporation of additional runtime environments, frameworks, and services through the use of the Heroku-developed buildpack technology. Finally, Cloud Foundry also provides an open approach to integrating with the underlying infrastructure. This allows Cloud Foundry to be run on Amazon, VMware, and most importantly to IBM's open cloud architecture, OpenStack.

An important consideration in IBM's joining Cloud Foundry was Pivotal's willingness to work with IBM and other members of the community towards a more open governance model for the open source initiative. An open Cloud Foundry platform enables clients to deploy and manage cloud applications in a more agile, more scalable manner, independent of the underlying infrastructure. An open Cloud Foundry ecosystem will foster even greater innovation and give clients greater choice by enabling them to choose from a broad range of programming languages, frameworks and cloud-based services with which to build their cloud-based applications.

In terms of contributions to the community, our initial collaborations have been around integrating IBM WebSphere Application Server Liberty Core and IBM Java as extensions to the default Java buildpack provided in Cloud Foundry. IBM intends to invest further in making the build pack technology portable across cloud platforms. Additionally, we will contribute in areas such as security, performance and tuning, management and monitoring, and messaging, as well as focus on other projects of mutual interest to IBM and the community.

Portable workload definition and orchestration

For portable workload template definition, our open cloud architecture incorporates the OASIS TOSCA standard. TOSCA is a collaborative open standard being developed by IBM, SAP, HP, Rackspace, and many others. What is really exciting is that many of these same companies are also collaborating on implementing support for OASIS TOSCA template descriptions in the OpenStack Heat project. Initially, we have enabled our IBM pattern deployment capability to import and export the OASIS TOSCA template format. Over time, we will likely see the IBM capability transition to OpenStack Heat.

Internet of things

Addressing the Internet of things, the MQ Telemetry Transport is a lightweight publish/subscribe protocol with predictable bi-directional message delivery. It's ideal for applications where a small code footprint is required, or when network bandwidth is limited. An Internet of things—sensors, network-connected appliances and devices, smart buildings, and networked-automobiles—could all benefit from the efficiency of this protocol.

The MQTT community is currently working to standardize the specification under OASIS. Additionally, there are multiple open source implementations of the MQTT protocol, including the Eclipse Paho project and the Mosquitto project.

Figure 4. Accessible business services available to third-party innovators via API Economy
Diagram showing Accessible business services available to third-party innovators via API Economy

Business applications and the SaaS layer

At the SaaS layer, our open cloud architecture addresses security and federated identity (TLS/SSL, OpenID, OASIS SAML, and the IETF’s OAuth2.0), payments (Google Wallet and Apple’s Passport), and user interface (W3C’s HTML5, the IETF’s JSON, Apache Cordova, and Eclipse).

HTML5 is the common language for building rich Internet applications across platforms, while still supporting native mobile functions without plugins. It is managed by the W3C and represents the evolution of HTML4 / XHTML. HTML5 also incorporates standards such as SVG and CSS.

HTML5 greatly simplifies application development as it brings native support for multimedia across a wide range of desktop and mobile platforms. Additionally, it brings native support for advanced technologies such as offline storage and applications, device access, geolocation, and semantic data.

Apache Cordova is the open source project that makes "write once and run anywhere" hybrid applications using platform services feasible. Cordova focuses on providing device-specific code that acts as a "shim" to enable hybrid application access to native mobile functions across a wide range of devices. The community is focused on continual maintenance and enhancements to the code that enable support for additional devices. Since the project's creation in 2010, IBM has been a significant contributor, adding new capabilities and improving focus on release management. Together, HTML5 and Apache Cordova are critical to the continued development of rich mobile applications.

Social

Activity Streams is an emerging standard that provides developers with a common vocabulary to describe business events, enabling apps that socialize data and knowledge with people in real time. Implementation with both existing and future IT architecture helps to simplify informational and line of business visibility across users, and supports an organization’s ability to share, comment, and collaborate.

OpenSocial is a standards-based component model for cloud-based social apps. This model provides a consistent method to integrate existing applications, with the ability to add third-party applications in a single view. This simplifies the user experience and enables users to work in the manner that best suits them at that moment. Information can be integrated across email, content systems, and rich media—all within a given context and across platforms. The project delivers cross-industry alignment through the Apache Shindig project, an open reference implementation of OpenSocial, and through support of leading standards like ActivityStreams and OAuth2.0.

Activity Streams and OpenSocial both have the potential to help to liberate data from traditional information silos, through real-time information sharing.

Security

OAuth is an open protocol to allow secure authorization in a simple and standard method from cloud, web, mobile, and desktop applications. Specifically, it enables authorization and access to REST API endpoints defined within an application or infrastructure. OAuth has become a key security technology for the integration of REST APIs into the enterprise, as it can provide access to services both inside or outside the firewall.


Technology previews

Attendees of IBM Innovate 2013 may have seen or heard about previews of this emerging capability reflected in our open cloud architecture, that we described above.

Bluemix

Another technology preview is Bluemix, which is an emerging public cloud environment, modeled after our successful jStart program. This public cloud environment offers many of the same services that are available through Project Icap’s sandbox, running on an OpenStack cloud. Presently, we are running on OpenStack Folsom, but we intend to migrate to the recent release of OpenStack Grizzly, very soon.

In this environment, we are hosting one of the emerging open source PaaS platforms where you can develop cloud-centric applications, leveraging a set of hosted services including: PostgreSQL, MySQL, Redis, blob storage, elastic caching (ECaaS), sample location services, an SMS service, simple social service, RabbitMQ, and MongoDB. We intend to deliver additional services online over time, from IBM, open source sources and from IBM partners, in a continuous delivery mode of operations. We will be especially interested in your feedback as to what services you would like to see.

The Bluemix team will be working closely with customers exploring these two preview offerings to gather feedback, to understand what software services are important to them, as well as how to manage the various aspects of developing, delivering, managing, and hosting these new systems of interaction.


Conclusion

We’re always interested in your feedback, so feel free let us know what you think of our open cloud architecture.

Resources

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Cloud computing on developerWorks


  • Bluemix Developers Community

    Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.

  • developerWorks Labs

    Experiment with new directions in software development.

  • DevOps Services

    Software development in the cloud. Register today to create a project.

  • Try SoftLayer Cloud

    Deploy public cloud instances in as few as 5 minutes. Try the SoftLayer public cloud instance for one month.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Cloud computing
ArticleID=900889
ArticleTitle=IBM's open cloud architecture
publish-date=07232013