Topic
  • 4 replies
  • Latest Post - ‏2012-12-07T14:33:43Z by ChrisRommel-VDC
JonChard
JonChard
6 Posts

Pinned topic Virtual Round Table - Agile systems engineering - testing breakout [Event closed]

‏2012-11-29T14:41:23Z |
**Even though this event is over, you are welcome to browse this discussion**
 
This thread is discussing issues relating to testing in agile product and systems development environments.

Some key points raised so far include:

  • Integration testing before unit testing is a key Agile practice, but may be perceived as difficult for embedded systems. Modeling can help here.
  • Agile use-case-driven testing can be much more lean than waterfall testing processes.
  • Meaningful measures of earned value and effective EVM are important.
  • Test-driven development can be used to demonstrate ‘earned value’.
  • If the ‘test harness is the spec’ then value is demonstrated directly.
Updated on 2013-01-07T22:40:39Z at 2013-01-07T22:40:39Z by sjpeich
  • LeeBThomas
    LeeBThomas
    8 Posts

    Re: Virtual Round Table - Agile systems engineering - testing breakout

    ‏2012-11-29T15:32:46Z  
    Another point, Jon, was that Agile testing of all kinds requires greater investment in simulators and prototypes, and far earlier.
     
    In my experience in the Systems domain, the word "testing" implies not an activity, but a department.  Even unit tests of code may be performed by someone other than the writer of that code.  That is distinctly different from the default Agile approach, particularly TDD.  But I could see that changing rapidly, if the testers are getting all the glory from being the only ones to demonstrate earned value.
  • bsandle
    bsandle
    1 Post

    Re: Virtual Round Table - Agile systems engineering - testing breakout

    ‏2012-12-03T20:31:56Z  
     What about "automating" testing?   I hear customers ask if we have automating testing when we test new code.    I hear from our developers that much is mandatory.   Is it really hard to do automated testing?
  • LeeBThomas
    LeeBThomas
    8 Posts

    Re: Virtual Round Table - Agile systems engineering - testing breakout

    ‏2012-12-05T17:15:59Z  
    • bsandle
    • ‏2012-12-03T20:31:56Z
     What about "automating" testing?   I hear customers ask if we have automating testing when we test new code.    I hear from our developers that much is mandatory.   Is it really hard to do automated testing?
    There are a variety of ways to automate testing, Brad, depending on what portion of the system is being tested.  Often what is mandated is code review; if there is a mandated standard, much of the review can be automated.  Unit testing of source code modules can be automated with TestRT (which also automates code review).
     
    Functional testing is a bit harder to automate in the Systems space, since the hardware interfaces may not all be exposed such that they can be controlled automatically.  We used to joke that we wanted a robotic arm that would press the "Reset" button on our ASIC tester so automated testing could proceed from a repeatable hardware state.
     
    Similarly, performance testing can be partially automated, depending on how much of the hardware can be controlled by software.
     
    As to how hard it is, well, it's programming - it requires that level of sophistication.  In some cases the testers must be better programmers than the developers.  Even with excellent tooling, it isn't the case that any warm body will do.  (That's true in the IT space, but especially true for Systems.)
  • ChrisRommel-VDC
    ChrisRommel-VDC
    1 Post

    Re: Virtual Round Table - Agile systems engineering - testing breakout

    ‏2012-12-07T14:33:43Z  
     Hi All,
     
    The adoption of Agile in the systems space makes automated testing both more important and more difficult. Per the discussion above, qa/test is/has often been viewed as a discrete point of process.
     
    The speed of iteration with Agile, however, makes the stepwise testing processes of yesteryear more and more burdensome. Not only does testing need to occur sooner and more often, but higher levels of testing needs to be enabled within the developer ranks as well. Our research shows a growing interest in static analysis tools as solution to address this needs within systems projects aligning to itereative design methodologies.