We recently presented a proposal for a version of the Agile manifesto for systems engineering.  This was presented at INCOSE UK ASEC conference on 12 November 2012. I have included below the slides which are on Slideshare, and the content of the slides along with some notes. 

This paper will be reworked considerable over the coming weeks to bring it to a publishable form suitable for a journal, and more worthy of the title 'paper'.  In the meantime, the original work is here and any comments or feedback would be welcome.

We did not consider any potential additions to the original content, but only looked at the suitability of what was originally present.

A proposed Agile systems engineering manifesto

Manifesto for Agile Software Development

We are uncovering better ways of developing

software by doing it and helping others do it.

Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on

the right, we value the items on the left more.

© 2001, the above authors this declaration may be freely copied in any form, but only in its entirety through this notice.

Principles behind the Agile Manifesto

We follow these principles:

• Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

• Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

• Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

• Business people and developers must work together daily throughout the project.

• Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

• The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

• Working software is the primary measure of progress.

• Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

• Continuous attention to technical excellence and good design enhances agility.

• Simplicity--the art of maximizing the amount of work not done--is essential.

• The best architectures, requirements, and designs emerge from self-organizing teams.

• At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Great, so what is wrong with that?

… for Systems Engineering

Came from lean thinking and a general desire to reduce uncertainty.  Was written to support the needs of s/w engineering, so there will be differences: some just terminology, some may be more fundamental about the applicability of the concepts.

Why rethink the manifesto and not just tweak a methodology?

"Agile is a mindset: defined by values, guided by principles, manifested through many different practices."
―Ahmed Sidky. co-author of Becoming Agile in an Imperfect World.

Top down or bottom up?  Agile has to be lived, not just codified.  All levels have to understand the mindset and the objectives, which are embodied in the manifesto.  The manifesto is being repurposed away from s/w to systems so we cannot be certain that the principles are still fit for purpose.

When you come to do agile SE, it is appropriate to have a methodology, but that is not something that is currently in a mature state of development and is not something we are trying to cover here.  We do recognise that it would be useful to have an understanding of the implementation, but that has based on something solid which is what we are trying to do here.

Measuring the change

When you change something it is important to measure the change so you know the magnitude and you can give it due consideration.  The approach we are taking here is to measure the change on a relatively informal scale, but will inform us where we sit on the scale of purely syntactic to deeper semantic changes.

Manifesto

Individuals and interactions over processes and tools
Working software Demonstrable capability over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

First the core value statements of the manifesto.  While we value the things on the right, we value the things on the left more.

The original version is shown here, and then a proposed systems engineering version.  Notably there is only one change, replacing valuing workable software with valuing demonstrable capability.   By that we mean possibly implementation of some feature of part of the product or system being delivered. Equally it could mean a model of the product or system being delivered, or some other valuable deliverable such as compliance evidence.

These are value statements not an intent to fail to do the things on the right.  We still expect to have processes and tools, but we value individuals and interactions over them.  We expect to have documentation but value the demonstrable capability over this.  Contract negotiation will be necessary, but customer collaboration on top of that adds value.  Finally while a plan is necessary, responding to change is valuable.

The misunderstanding of this sort of statement has led to skepticism about Agile in the past.

1. Our highest priority is to satisfy the customer through early and continuous regular delivery of valuable software demonstrable capability.

Rewording is required here to remove the emphasis on software.  Supplementing the deliveries with models in an appropriate form would also add achievable benefit.

Early and continuous delivery is included in the Agile principles to reduce risk.  By delivering early, the customer has an opportunity to see how the project is being implemented and this is a part of the customer involvement from the manifesto.

For embedded software, it is often necessary to carry out a great deal of work on low level software such as drivers and data structures before visible progress can be made.  Forcing visible progress early can be counter-productive, as the underlying architecture decisions may be compromised in the process of making visible progress.  For physical systems, particularly large physical systems, it can be difficult to effect continuous delivery.  However, continuous delivery of a model environment should be possible and is valuable for demonstrating progress and features to the customer, and in de-risking future stages of the project by validating incrementally.

When we talk of demonstrable capability we include elements that are not a part of the final product such as compliance evidence, or a progressive step in the architecture.

Discrete delivery is more obviously discrete with hardware than it is with software.  Software deliveries can approximate continuous in the same way that a movie at 24fps approximates motion.

2. Welcome changing managed change to requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

We rebel against changing requirements because they can be very expensive in a large systems project.  Early dialog is needed to fix requirements at an appropriate stage dependent on their impact.  A late change is late relative to the needed implementation date of the requirement, rather than an absolute date based on final delivery for all requirements.

It is only by understanding what we mean by managed that this principle can apply at all.  The time when you want to accept changes to requirements is when the alternative is even more expensive because you are moving away from the customer’s needs.

Collaboration with the customer to identify areas of expected change, or higher requirements volatility can allow the architecture to be designed to support flexibility where needed.  Some changes can be accommodated later than others, and an open, collaborative approach shows the customer which areas of design need to be fixed at every point in the development.  This is not an open invitation to significantly change the key requirements the day before delivery, but rather a principle of allowing changes right up until they cause significant problems.

The big decisions must be made early, the smaller details can be allowed to continue to change for longer.  To reduce the program development cycle of a large program, some of the detail may have to be started before the architecture is fully committed, resulting in the risk that seemingly insignificant decisions can prove problematic in the long run.  This gives a number of starting points for the firming up of requirements, rather like a liquid crystallizing from a number of seed sites.

This principle is about reducing risk as far as possible in the final product’s ability to meet the customer needs, whilst maximising the ability to make necessary changes.

3. Deliver working software actual or modeled functionality frequently, from a couple of weeks to a couple of months upwards, with a preference to the shorter timescale.

Delivery of incremental functionality is a relevant goal for systems.  The emphasis on software needs to be changed.

Perhaps the constraint of a couple of weeks to a couple of months is no longer necessary, although traditional systems approaches of much longer timescales should be shortened where possible.  Frequently may be more variable than in a pure software environment.  As quickly as possible is the aim here, but the constraints of mechanical and electrical/electronic component pushes us toward a longer timescale.

At the detail level, incremental delivery of embedded software is achievable in the timescales stated, and incremental delivery of target hardware is also a realistic proposition at the upper end of that range.  Incremental delivery of functionality to subsystems or full scale systems is harder to achieve within the timescale.  Delivery of working models, however, is a practical possibility.  The use of models, with their relatively low cost of change, lends itself to an agile approach; the model can be worked through with the customer and other stakeholders and functionality can be added to the model iteratively during the early design phase of the project.

Any non-final delivery of product is arguably a model, in that it is an imperfect representation of the final solution.  Some models will be recognisably similar to the final solution, and some will be simulations.  All will have imperfections, many will be useful to aid discussion with the customer and understanding of the progress and options.

‘All models are wrong; some models are useful’ – George Box

4. Business people and developers  the project team must work together daily throughout the project.

This is relevant, the word developers would be better replaced with systems engineers and designers.  Collaboration is helpful.  Some projects are so large that full team communication and collaboration is not practical on this sort of daily cadence.  However, some regular contact between the business and the project should still be achievable.  The project team is broader than just engineers and this principle is a critical piece of alignment to ensure that the project achieves its goals.

The word ‘daily’ is deliberately left in because that level of interaction is vital.  A vertical thread of communication through the business and project is necessary to address problems as they arise rather than after the event.

5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

6. Use the The most efficient and effective method of conveying information to and within a development team, from face-to-face to social business tools and collaborative design tools. is face-to-face conversation.

While face-to-face is effective, with our globally distributed teams it is often impractical. Project teams can be unmanageably large in systems projects.  There are a number of possible communication mechanisms, some more effective than others.  A robust collaboration environment is essential and the new words reflect the advancement in tooling since the original principles were published. Use of a collaborative tool environment for project control reduces the need for face to face meetings.  An informal approach to recording in the tools keeps the time commitment to a minimum.  A project wide instant messaging system allows team members to easily communicate for ad-hoc conversations, instant web meetings enable more detailed technical discussions and collaborative, web based, engineering tools ensure that sufficient information can be easily exchanged about the project progress and the content.

The time and expense taken to bring a globally distributed team together in a face-to-face environment is large, and while face to face conversation is still the most effective means of communication, there are good alternatives available.  When the Agile manifesto was written home broadband internet access, and the always connected computer, was a new phenomenon.  While many offices had good connection speeds and email available from the individual’s desk, the software tools to support collaboration over these connections were immature.

What the development team needs is a high bandwidth, and close to real time method of communication.  Face-to-face wins here every time, if the travel time is discounted, but some very effective social business and collaboration tools are now available.  These tools allow real time reporting of progress, reducing the need for status reports and increasing the accuracy of information available.  It is possible to collaborate on a document, watching another person type when they are 6000 miles away.  Combining these real time high bandwidth tools with a telephone conversation can be almost as productive as a face-to-face meeting.

7. Demonstrable capability Working software is the primary measure of progress.

Working systems, or working subsystems, are a better measure of progress for this domain. Working models may be all that is available for a significant portion of the project.  We talked about demonstrable capability earlier and how it can take many forms.

The key point is that we are aspiring to use the most directly available measure of progress. We accept that there may need to be documentation, reviews, gateways statuses and the like – but we value demonstrable capability as the primary measure.

8. Agile processes promote sustainable development. The sponsors, project team developers, and users should be able to maintain a constant pace indefinitely.

Sustainable development is an important principle, pushing a team until they break is not productive in the long term.  The change here from ‘developers’ to ‘project team’ is to include the full team that will be involved.  A large systems project has a number of different roles included and all of these need to be working sustainably.

Sustainability is derived from an approach which reduces the need for fire drills and heroics by attempting to de-risk projects as early as possible.  Working until midnight for the last panic is certainly not sustainable.

9. Continuous attention to technical excellence and good design enhances agility.

There is very little to argue about here.  Technical excellence and good design are worthwhile regardless of the desire for agility.  Focusing attention of these will enhance agility. This again comes from reduced risk, and building in quality.

10. Simplicity--the art of maximizing the amount of work not done--is essential.

This is not an excuse to not do unpalatable project tasks.  If a safety case is needed then failing to produce it is not agile.  However, if there is an efficient route to an end goal that avoid unnecessary steps then this should be pursued.

Stopping work to compile a status report is not productive, but progress needs to be assessed.  Simplicity is having a tool that engineers can use to track their own progress in the form of a task list, and taking that data into a live report showing the status now, rather than the status as reported up to a week ago.

Selecting tools and practices to automate overhead work is a part of this, recognising genuinely unnecessary steps or work products is another part.

11. The best architectures, requirements, and designs emerge from self-organizing teams.

This seems intuitive for small teams, but putting together a team consisting of thousands of individuals and expecting that to self organize is not realistic.  The reason this is left unchanged, is that even on a project with thousands of people, each individual is most likely working in a sub team of less than ten. Within those smaller subteams, self organization is just as effective as on a very small project.  For instance work for the team can be assigned to a project backlog, and individuals organize among themselves who will take which tasks.  This is not a call for no management, or no structure, but a reminder that most of our interactions are with a small number of individuals who each have differing strengths which can be used.

12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Reflecting on the past and changing behaviour in the light of that information is also a theme in high maturity organizations.  Continuous improvement as called for in CMMI Level 5 is doing the same thing.  This principle is not restricted to Agile, and makes sound business sense regardless of other approaches taken.

Change summary

Here we summarize our change-meter results for the manifesto and each of the principles. The key takeaway from all this is that we have changed relatively little.

The most significant change is the principle #6 – the most efficient means of communication which we have changed from face to face to making use of the collaboration technologies where appropriate. (This is incidentally a change that could equally apply to a scaled software environment).

We have also changed the first principle - the definition of what is delivered and when.

With so little change, was that really worthwhile?

YES!

This is a set of principles to work to

This underpins the culture of the project team

With strong foundations, it is possible to build bigger

“Is it better to spend your time understanding principles or studying practices? We observe that the best results come from combining the two approaches. Copying practices without understanding the underlying principles has a long history of mediocre results.”

—Mary & Tom Poppendieck, Implementing Lean Software Development, Chapter 2 – Principles

What next?

Agree on the principles

Methodologies – what can we learn from the software experience?

INCOSE International WG exists

UK Working group?

This presentation has proposed some amendments to the principles – clearly there is a need to agree on the Agile systems engineering manifesto as a basis for further work.

There is also a need to consider the methodologies for doing agile systems engineering. Clearly existing methodologies were designed for software development and it is unlikely that they will be suitable for agile SE. However they may be able to inform agile systems engineering approaches.

There is already an INCOSE agile systems engineering international working group – that has been running since January this year and is looking at a number of projects:

Fundamentals of agile systems engineering

Decision making guidance for applying agile systems engineering

Systems engineering for software intensive projects using agile methods

Rick Dove is the chair of this group and I would encourage you to get involved with the work-streams

We would also be interested to hear ideas from anyone for additional projects and work-streams – and possibly a UK-based workgroup that  could complement and broaden the  coverage of the international group