The concept of continuous certification: get rid of the Big Freeze
By Mitch on Friday, 31 August 2012, 16:47 - Processes - Permalink
Safety critical software always face the big freeze before certification.
This happens because watefall model is the prefered software development cycle for safety critical software. Thus you can't change anything if you're in qualification phase for certification.
To be more flexible, some smart people created the concept of continuous certification. The purpose of continuous certification is to apply the principles of agile methods to safety critical software development.
Using agile methods allows breaking the big freeze. Hence everything can change at each iteration. Agile methods are limited by constraints of safety critical software development. As I wrote in Class C software and agile methods, user requirements shouldn't change in iterations. But small changes in functions, in ergonomics and in architecture should be allowed.
What happens if I push this button?
To give you an image, changing something in a safety critical software may be like leaving Leslie Nielsen pushing a button in Airplane! cockpit.
If you want to be 100% sure of what you do when you change something:
- Either you're a guru who knows everything about his software and can assess the consequences of the changes,
- Or you have a tool that gives you all the artifacts touched by the change.
The second option is the principle of continuous certification. Each time you change something, you know exactly what is going to be touched and what has to be retested/requalified.
Tools for continuous certification
Software development documentation uses extensively traceability. In most of my templates, like the Software Tests Report, there are traceability tables.
The tools that manage continuous certification extends the principle of traceability to (almost) every bit of software documentation and traceability. This kind of tool has:
- databases to store the software structure and the traceability between artifacts,
- rules to verify that structure is consistent,
- algorithms to walk through the structure and find structural "bugs",
- formal language to let humans fill the database.
The data structure is pretty much like a tree, with user requirements in the trunk and tests cases in the leaves. For example, when something has to be changed:
- the change is coded in formal language,
- algorithms walk through the tree to find what is touched by the change,
- other algorithms may raise issues if the change makes the structure inconsistent,
If the results of algorithms are OK, then the change is coded. Some more classical tools like static analysis can be run to ensure that the code is bug-free.
To make a comparison, this is like unit tests, but far more complex:
- when you change someting in the code, units tests show you immediately if it works,
- when you change something in the requirements, continuous certification tool shows you immediately what is touched.
Do these tools exist?
An open-source projet has the ambitious goal to deliver such a tool: Open-DO project.
The Q-Machine is the tool that does what I explained above. It is not fully integrated to well-known IDE like Eclipse or GNU tools (should be soon I think). But this is a big change in the software development. Open-DO is still under construction. Thus you don't have all necessary tools for static analysis, code coverage and the like. But the team is working hard on it and knows its benefits for the software medical device industry.
Conclusion
I wrote once that combining safety critical software with agile methods is like herding cats. I hope tools sets like Open-DO will make me definitely change my mind.
A big thanks to Loïc of RealTime-at-Work who gave me the link to Open-DO.
