Comon sense is: Yes, there are always bugs in software.
Some people get even so far to say: Writing bugless code is not the goal.
Some people are “yelling”: Anyone who tells you otherwise is clueless.
OK. Before I put in my two cents, just in case someone might think that I am insane, here is probably the reason: I am programming computers since approximately 1980. I programmend in lots of imperative computer languages starting with 6502 hex code, 68000 assembler, C, Modula 2, Pascal, C++, PHP, Java, JavaScript. I studied Computer Engineering with focus on hardware and software (Technische Informatik) and have been awarded the degree of Dipl.-Ing. (FH). I was also used to deal with HTML, XML, XSLT, DOM, CSS. I programmed systems for electronics, production mashinery, business processes, telematics systems, and space systems.
I encountered lots of bugs – especially in the 90’th. I remember a moment when I asked myself whether it would be possible to create a programming language that cannot have bugs by design. Lately I started to programm in Haskell, and figured out that Haskell is very close to what I was looking for.
So here is my proposal: The statement: “Bugs are inevitable.” is wrong – and the statement: “Bug free software is possible” is wrong, as well, and…
…the answer is: You don’t know whether there is a bug – until you encounter one.
This answer seems to be as stupid as it is true. If you encounter a bug then you know that the software is not free of bugs. This is true for both:
- untested software
- completely verfied & validated, hence heavily tested, analysed, cross checked and bugfixed software
The difference is that the software of the second case – just might be free of bugs – at least it has a chance to.
Questions for sceptics: How do you know that there is a bug in a part of a software that is completely verfied & validated, hence heavily tested, analysed, cross checked and bugfixed software – when you have never encountered a bug? Which verification & validation activity did you skip – that you expect a bug?
Known excuses to have bugs in software
The excuses to have bugs in software are as follows:
- Complexity of software is to high
- It is not running in a bug free environment, anyway
- You may not have the source code
- Parallel execution
- It is not possible to test every possible combination
- Hardware has also bugs and hickups
However, the points sound reasonable but are not completely valid. We have the following counter arguments:
- Complexity can be divided into small pieces of low complexity.
- When a part of a software can be bug free then a larger software can be bug free as well. This refers also to libraries, operating systems and so on.
- If you do not have the source code then we can write our own code to replace libraries and operating systems.
- If the system is well designed, completely verified and validated then parallel execution will not neccessarily lead to bugs.
- Equivalence class partitioning, or proof assistants like Isabelle can be applied.
- Hardware hickups and bugs are not software bugs.
Hence, low propability of bugs in software is just a matter of effort. I know, the effort can be very huge in large systems.
Questions for sceptics: Have you ever seen a function in source code where it is unpossible to reduce the complexity – if there is two ore more statements or sub-functions? Have you ever seen a function of one or two statements or sub-functions in source code which cannot be proven bug free? Do you know any operating system or library that could not follow the same priciple?
Exercises towards bug free software
If you want to have bug free software then this is the minimum that you have to do:
- divide the complexity or a large function into small units of low complexity
- low enough for a complete description (including valid parameter ranges)
- low enough for a complete test (considering all equivalence classes)
- conduct unit test for all units
- use of a programming language with a strong type system (e.g. like Haskell)
- document all functions completely including all restrictions, if any
- conduct function tests for all functions, according to it’s documentation
- analyse whether all functions are integrated into other functions, considering their documented restrictions
- validation of documented functions
- validation of documentation vs. tests
Additionally, it is wise to do the following:
- write a verification & validation plan and conduct accordingly, also considering
- fixing all warnings
- application of an lint-tool (e.g. HLint)
- aviodance of programming construct thats might cause errors
- applying checklist to make sure that the verification & validation plan has correctly been applied
- automation of all tests
- application of a proof assistant
Can bugless software be a goal?
It can be a goal to have bugless software as long as you consider that you may not reach the goal. Since it is not the best approach to set objectives that cannot be accomplished we need to do better.
Proposed objective: To conduct a planned and documented verification and validation plan that shows that it unlikely that there is still a bug – assuming the plan is (going to be) conducted.
When should it be unlikely that there is still a bug ?
In my opinion there is the following reasons to use the abovementioned excercises:
- the software design is stable and a lot of code depends on that software
- bugs would lead to
- security breach,
- data loss (e.g. cryptoalgorithms),
- loss of money (e.g. money transactions),
- loss of mission (e.g. satellite attitude control),
- or even loss of life (e.g. brake system controlled via CAN-Bus)
Questions for sceptics: Would you accept that a password database has a security breach? Would you accept the your reposity is geting corrupt?
Why is this discussed here?
Software 4sd is aiming to create software that creates software – also for mission critical systems, or finance. Hence, a lot of code depends on a low bug propability of our main products.
How does software 4sd deal with software quality?
Software 4sd established
- a testing convention, which also consideres validation of tests vs. description
- and a coding convention
The main products have a verification & valdation plan, which defines the rules and procedures for
- verification,
- validation,
- pre-release and
- final release.
Verification
Verification mostly takes place in form of unit and function tests. Additionally, there may be integration tests, revew of design and analysis. The verification is based on documented functions.
Example: The module Code.hs has been completely been tested in 3667 unit test cases, plus 133 property tests cases where each has been tested with 100 random variables – which summs up to 16,967 test cases.
Validation
The validation checks whether the tests are according to documentation (e.g. specification, and unit descriptions).
Example: The module Code.hs has been completely validated by validation comments according to the testing conventions – which means that the followings has its own criteria to be validated:
- Functions and laws
- Classes
- Modules
- Libraries
The following checks are included to check whether the tests are according to documentation:
- functions (units), and laws vs. function description, and law descriptions
- function, and law description vs. unit tests
- unit tests results vs. unit tests
- coverage reports vs. functions
Pre-release
The pre-release takes place to
- collect the final product and byproducts
- executables
- resources
- software configuration file
- version identification
- version number
- check sums
- test results
- software problem reports
- validation results
- version identification
- test the final product and byproducts, e.g. by testing
- download procedures for all platforms
- installation procedures for all platforms
- opening the documentation
- function of the product
- availability of licensing information
- availability of disclaimers
Final release
The final release takes place by bringing the product and byproducts online, if and only if all final tests are successful or minor and documented.