This webcast was pretty popular and had a large number of questions that I wasn’t able to address on line, so I’ll address some of them here.
Can these design processes be applied to systems engineering as well?
For the most part, absolutely – especially for the architectural patterns. However, keep in mind that the outcome of system engineering is specification while the outcome of software development is implementation. In addition, the Concurrency and Resource view of architecture is specific to the software domain.
Can you speak about the implications for tool qualification in a DO-178 level A environment?
The DO-178 standard identifies two types of tools (formally three types in DO-330, but it still works out to be effectively two): tools that could insert a defect through their use (development tools) and tools that could fail to identify a defect (verification tools). In general, you need not qualify development tools because verification will identify the defects so they may be fixed – provided, of course, that verification is adequate to discuss such defects. However, verification tools are usually relied upon to identify defects to they usually must be qualified. So modeling tools, editors, and compilers are not usually qualified but testing and test coverage tools usually are.
How large a development team does this scale to and what are some of the scaling issues?
The approach and practices discussed in the talk can be applied to small teams and large teams. Large teams always carry along with them greater concerns for project management and governance. The biggest issue in terms of the design practices per se has to do with the coordination of the work items among the various subteams. Hierarchical planning addresses that concern but does require more diligence and effort. Distributed teams also add some barriers that can be addressed through a combination of process and tooling. For example, Design Manager supports web-based reviews of models to support the review objectives of a number of safety standards. Practices around those issues will be addressed in an upcoming webcast on project management practices.
Can you comment on deploying this approach across a distributed team?
The concerns specific to distributed systems include:
1. Lack of project control
2. Lack of visibility of project and product status
3. Detailed coordination is more difficult because of delays, bandwidth limitations, and difficulty in
4. Difficulty in communication
5. Cultural and organizational differences
6. Configuration management and work product sharing
Cloud-based CM solutions are an approach to share work products and related information. Distributed project management/governance tools such as Rational Team Concert can address visibility of metrics and project status of distributed teams. These concerns will be addressed in the aforementioned upcoming webcast.
Hello Bruce- How what you discuss maps to ISO 26262. Have you ever check/tried Harmony mapping into this?
There are mappings to IEC 61508 and ISO 26262 and the Harmony process content under development. Right now, we have detailed mappings to DO-178B, DO-178C, and IEC 62304.
You have provided and example of SW Design Pattern, more related to SW architecture than to SYS. Are there any good Systems Architecture patterns used in embedded world?
There certainly are patterns defined within the Harmony process. They are similar or identical to some of the software design practices, but apply to designs that have not been allocated to specific engineering disciplines. The Deployment view (done with block diagrams (SysML) or UML (software) are used to show the allocation of responsibilities and requirements to different disciplines and the specification of the cross-discipline interfaces, such as the interface between the software and electronics.
You mentioned using TDD and other processes which include testing into a cross-functional team, how do you deal with keeping the independence of the test team as some standards like Cenelec state in order to verify and validate your safety critical system.
Independence of the development of the system verification test cases is required – that means that the designers/developers cannot be the same people that develop the test cases (it doesn’t affect who executes the test cases though). Independence is not required for integration, although it is useful and it is not recommended for white box developer testing. TDD specifically refers to the white box developer tests so independence is not a concern. It remains a requirement for verification however.
You said we were mainly talking about collaboration between electronics and software. but are there more specific issues when developers interface with nuclear, avionics, mechanical, etc. as different disciplines?
The process I outlined specifies systems level requirements and creates systems architectures. The subsequent hand-off to downstream engineering takes place primarily at the subsystem level, including the allocation of subsystem requirements to different engineering disciplines and the specification of the cross-disciplinary interfaces. It is expected that disciplines such a mechanical, chemical and nuclear will accept their requirements and interfaces specifications and then proceed with the discipline-specific design in appropriate tools. The software development will continue in UML but it is difficult to imagine how mechanical or chemical design would be done in UML or SysML.
Is system level testing applied in Harmony? At what level of development probably not during the nanocycles? My understanding is that during nanocycle there is rather unit testing in place. can you confirm?
There are system verification activities done in the Harmony process but not during the nanocycles. The microcycle (sprint) has a verification activity at the end of each iteration that verifies the produced (but possibly incomplete) system meets all the requirements developed during that and previous iterations. The nanocycle TDD testing is fundamentally white box developer testing.
Bruce, please do not throw away napkins model. We often use them when we communicate and it works as long as we can share ideas during meetings etc. You well know how difficult it is to create precise and fully ready model, often not worth time spend on it. Can you please comment that?
I’m not saying that you should never develop napkin models, just that they shouldn’t be the primary form of model and they shouldn’t be treated as normative. Napkins models are useful for framing a discussion. They are not useful, in my experience, for specifying a normative design. If a set of napkins models are used for discussion to discuss trade offs, for example, the winning napkin should probably be developed into a normative high-fidelity model. IMHO.
Is Rational considering AUTOSAR in their Rhapsody Road Map?
Rhapsody already supports AUTOSAR as a profile in Rhapsody and is continuing to evolve it based on customer experiences.
What Time Frame do you think is necessary put together Design Pattern/ implement system
I think of the microcycle (sprint) as a 4-6 week period of time during which requirements are elaborated, functional software is developed, designs optimize the implementation, and verification is performed. The functional software and design optimizations are usually in the 2-3 week timeframe but this can vary +/1 a week and still stay within the overall microcycle timeframe.
What if Design Flaw shows up, can you trace it back?
Design flaws manifest themselves initially as failed test outcomes, then change requests, then as work items, and ultimately as updated design, code and tests. Good tooling – such as Change or Clear Quest can manage the traceability among the various forms defects take, the work products affected and the workers making the changes.
In defensive design couldn't that also result in extra money which may be a deterrent and as a result not performed?
That is absolutely true and this gives rise to the software folk wisdom “we have time to fix it but we don’t have time to do it right.” I personally believe that the “rush to code” and the lack of defensive design is short sighted. As Grady Booch says “If we developed houses the same way we develop software, the first woodpecker would have destroyed civilization.”
I would like to know why this approach cannot be applied to Waterfall models of development
It can be, but the result is no longer waterfall.
Should the aspects of Defensive Design be considered during analysis (and then designed and implemented) or after should this be put off after implementation of the "working model" - please explain tradeoffs
Defensive design is, first and foremost, an optimization of robustness and so it typically added into the design phase, which follows the creation of the initial functionally-correct high-fidelity model. Code, of course, is continuously being generated and updated during both the functional modeling and the design optimization. The problems with deferring the defensive design approach include 1. If time is short, it will be removed and 2. It is much harder to go back and “make it right” later than it is to “make it right” the first time. This is why, for example, TDD puts developer testing as a parallel activity to modeling and coding. The idea is to avoid defects rather than introduce defects now and find and remove them later.
Can you elaborate on what is meant by 'Continuous Safety Assessment'?
With the software development activities, design and technology decisions are made. These can have impacts on safety. For example, deciding to add a thirty party math library or using a commercial CORBA ORB have impacts on safety that must be addressed. Therefore in parallel with the software development activities, such decisions are examined by the Safety Czar to look at how they impact safety. This may result in new concerns and work items that must be addressed. In the Harmony process, “Update the Hazard Analysis” task runs in parallel to the entire microcycle so that technical decisions can be reviewed and safety concerns can be addressed.
Bruce, for someone who knows UML, how do you suggest to pick up SysML most effectively?
If you know UML, then the biggest changes when you move to SysML are just adopting the name changes (for example, “class” becomes “block”). There are some other important changes, such as the addition of parametric diagrams (useful for trade studies) and continuous behavioral modeling in activity diagrams. There are a few books on SysML but none that I’m really happy with. I’d probably look online for presentations that address my concerns. Also, you can defer learning about SysML features that might be new to you until you need them.
Note the project I'm using employs Rational Software Architecture Real-Time Edition (RSARTE); to what extent are some of these approaches applicable to that as well?
All the design practices can be performed within RSARTE. The Harmony process and practices are tool agnostic. I spend most of my time with Rational Rhapsody so I use that tool for examples and work, but other tools can be applied as well. Certain of the flows are made more difficult with some of the less capable tools. For example, in the High-Fidelity Modeling workflow there is a Translation task which produces code from the model. With Rhapsody (or RSARTE), that is mostly a matter of pushing the Generate/Make/Run button. In a less capable tool, you might pop up vi or edlin and bang out the code by hand. The workflow remains the same though.
The system seems extremely thorough but not very agile - how is this more agile then previous methods?
is really all about avoiding defects with continuous testing, being responsive to changing needs and circumstances, and dynamically project planning and governance. This talk focused specifically on the design practices, which included high-fidelity modeling, TDD, continuous integration, continuous safety assessment, and continuous traceability. These practices are what makes the approach agile. These certainly aren’t all the practices in place – if it was then I wouldn’t need the other four parts to the series! In addition, developing safety critical systems levies another set of needs of the development team and these can be address in an agile way as well.
What if you have portion of your system that is safety-critical (vital) and non-vital, would you recommend splitting to use your techniques (Rhapsody) and standard not critical processes (maybe like Enterprise Architect)
If part of the system is non-critical, then you must have architectural partitioning to ensure that any defects in the less-robust-and-verified parts of the system cannot affect the safety of the safety critical parts. I would tend to use the same tool for all to facilitate architectural allocation and refactoring personally.
What are your recommendations for Information Security-specific design patterns?
Check out Security Patterns: Integrating Security and Systems Engineering http://www.amazon.com/Security-Patterns-Integrating-Engineering-Software/dp/0470858842/ref=sr_1_1?ie=UTF8&qid=1362597852&sr=8-1&keywords=security+patterns.