Continuous integration is a software development process where developers integrate the new code they've written more frequently throughout the development cycle, adding it to the code base at least once a day. Automated testing is done against each iteration of the build to identify integration issues earlier, when they are easier to fix , which also helps avoid problems at the final merge for the release. Overall, continuous integration helps streamline the build process, resulting in higher-quality software and more predictable delivery schedules.

By definition, DevOps outlines a software development process and an organizational culture shift that speeds the delivery of higher quality software by automating and integrating the efforts of development and IT operations teams – two groups that traditionally practiced separately from each other, or in silos.

In practice, the best DevOps processes and cultures extend beyond development and operations to incorporate inputs from all application stakeholders - including platform and infrastructure engineering, security, compliance, governance, risk management, line-of-business, end-users and customers - into the software development lifecycle. 

In the DevOps framework, continuous integration sits at the beginning of the software development process, where you check in your code at least once a day to keep your local copies from drifting too far away from the main branch of the code build. This helps you avoid disastrous merge conflicts that could "break" the build and take the team hours or days to resolve.

Continuous integration serves as a prerequisite for the testing, deployment and release stages of continuous delivery. The entire development team will know within minutes of check-in whether you've created bad code, as the continuous integration service automatically builds and tests your code changes for any errors.

Continuous delivery and continuous deployment follow continuous integration in the DevOps cycle. 

Continuous delivery (CD) picks up where continuous integration ends, automating the delivery of applications to selected infrastructure environments. CD focuses on delivering any validated changes to the code base—updates, bug fixes, even new features—to users as quickly and safely as possible. It ensures the automation of pushing code changes to different environments, such as development, testing and production.

In continuous deployment, the code changes to an application are released automatically into the production environment. This automation is driven by a series of predefined tests. Once new updates pass those tests, the system pushes the updates directly to the software's users.

Commonly cited benefit of continuous integration include:

• Early and improved error detection and metrics that let you address errors early—sometimes within minutes of check-in
  
  
• Continuous and demonstrated progress for improved feedback
  
  
• Improved team collaboration; everyone on the team can change the code, integrate the system and quickly determine conflicts with other parts of the software
  
  
• Improved system integration, which reduces surprises at the end of the software development lifecycle
  
  
• Fewer parallel changes for merging and testing
  
  
• Reduced number of errors during system testing
  
  
• Constantly updated systems to test against

Agile is a software development practice that improves how software development teams organize themselves, adapt to changes in requirements and release software. Because continuous integration (link resides outside IBM) and agile development share many of the same features (e.g., testing automation), it can be useful to talk about continuous integration and agile at the same time. Agile organizes development into smaller groups of work or sprints. When applied in DevOps, these combined practices help to ensure software quality and project flexibility.

Continuous integration requires you to integrate work frequently, often many times per day. You verify integration by an automated build that detects integration errors as early as possible. The build should include run tests as part of verification. Extending the rapid testing to run time tests in an automated testing environment leads naturally towards continuous delivery.

Agile (link resides outside IBM) is also iterative and adapts to change so it can scale and evolve solutions over time. In the context of continuous integration, agile software development is about delivering software iterations based on how you prioritize the value of the features as you continuously integrate.

Popular open source continuous integration tools include:

• Jenkins: A widely used open source continuous integration tool, Jenkins allows developers to automatically build, integrate and test code as soon as they commit it to the source repository, making it easier for developers to catch bugs early and deploy software faster. The Docker plug-in is available on Jenkins.
  
  
• Buildbot: Buildbot can automate all aspects of the software development cycle. As a job scheduling system, it queues and executes jobs, and reports results.
  
  
• Go: What makes Go stand out from the crowd is the concept of pipelines, which makes the modeling of complex build workflows easy.
  
  
• Travis CI: One of the oldest and most-trusted hosted solutions, it is also available in an on-premises version for the enterprise.
  
  
• GitLab CI: An integral part of the open source Rails project, GitLab CI is a free hosted service that provides detailed git repository management with features like access control, issue tracking, code reviews and more.

Conducting continuous integration with open source tools offers many benefits, including the following:

• Hundreds of plug-ins that can support your project
  
  
• Wide support for open source languages, such as Python, Java and JavaScript
  
  
• No cost, giving students, startups and developers working on the side a powerful tool that's easy on the budget
  
  
• Customizability, allowing developers to take the foundation of a CI tool and build on top of it to fit the needs
  
  
• The ability to modify and redistribute the tools

Open source continuous integration tools to consider for your software development workflow include Jenkins, Go, Buildbot and Travis CI, which you can read about in the next section.

A continuous integration server is a software tool that centralizes all your continuous integration operations and provides a reliable, stable platform for you to build your projects. You can configure and adjust CI servers to build various projects for different platforms. A continuous integration server easily models and visualizes complex workflows (enabling continuous delivery) and provides an intuitive interface for building continuous delivery pipelines. A continuous integration server offers the ability to do the following:

• Run automatic builds, tests and releases in a single place
  
  
• Deploy any version, anytime
  
  
• Keep configuration orderly
  
  
• Support plug-ins to enhance functionality
  
  
• Monitor your project’s repository
  
  
• Pull changes and perform tasks you defined for successful commit
  
  
• Send feedback to the relevant project members with details of the build

The following hypothetical use case illustrates how two software developers can use continuous integration to improve their DevOps process.

The two developers must communicate with each other on which features work and how. This small team needs regular updates and must be able to integrate and test their code as a whole. Scheduling code check-in and testing take up a lot of development time. An automatic system for continuous integration is needed.

Negotiating when these combinations and tests would happen would eat up a lot of developer time. They would have to agree on the following:

1. When to begin testing code integration
2. How to test that the integration has been successful
3. How to communicate the results to the team

Continuous integration platforms have default answers to these questions and most allow configuration and setup.

Typically, CI platforms like Jenkins begin integration tests on check-in. When new code is checked in, the CI system will run a set of tests, which can include unit tests and regression tests, and then determine if the code was successfully integrated.

Or, if you're using a compiled language, the default test will be if the code compiles successfully. If not, then the new code broke the build. For languages like Python or JavaScript, you must create your own integration test.

Either way, most CI systems log the integration attempts, success rate and other metrics.

The importance of testing

Continuous testing begins when you produce a continuous integration build and a package (also known as an installable entity or packaged entity). It stops when that packaged entity goes into production. Every step from end to end involves a testing suite.

At a minimum, when you only have one test stage, 30 percent of continuous integration involves testing. In reality, continuous integration activities are comprised of 50 percent to 70 percent testing. It used to be that you had to complete testing manually. Now you can use automated tests—the key to successful continuous integration.

As part of automating testing for continuous integration, test-driven development iteratively builds code and tests one use case at a time to ensure test coverage, improve code quality and set the groundwork for continuous delivery. Automated testing tells you whether new code failed one or more of the tests developed across all functional areas of the application. A best practice requires developers to run all or a subset of tests in their local environments, which ensures that developers only commit source code to version control after the new code changes pass their tests. Experience shows that effective regression testing can help avoid unwelcome surprises later.

The continuous integration pipeline

A continuous integration pipeline automates stages of a project's pipeline—such as builds, tests and deployments—in a repeatable way, with minimal human intervention. An automated continuous integration pipeline is essential to streamline the development, testing and deployment of your applications by enabling controls, checkpoints and speed.

Continuous integration best practices

The continuous integration process is a critical component of DevOps, which helps you unify your development and operations teams into a shared repository for coding, testing, deploying and supporting software. The following are some CI best practices that can help you succeed:

• Maintain a single source code repository: Use source control management to track and control all the files for building a product. This consolidated code base makes distribution and visibility easier.
  
  
• Automate the build: This involves compiling, linking and other processes that produce the build artifacts. Self-testing should also be automated.
  
  
• Use daily mainline commits: Force developers to commit their changes to the main development stream at least once every day. Each developer must verify that their working copy is consistent with the main development stream.
  
  
• Test in a clone of the production environment: Make the test environment as similar as possible to your final production environment.
  
  
• Automate deployment: Implement multiple environments (development, integration, production) to run builds and tests.