IfSQ

Research into the Maintenance of Software

The following research findings are relevant to the IfSQ Standards:

• Why should you inspect software?
  • Code inspection is up to 20 times more efficient than testing
  • Code reading detected about 80% more faults per hour than testing
  • Debugging and associated rework takes about 50% of the time spent in a typical software development cycle
  • Microsoft's applications division has found that it takes 3 hours to find and fix a defect using code inspection compared to 12 hours by using testing
  • The combination of design and code inspections usually removes at least 70% of the defects in a product
  • Detection of design defects costs 6 times more using testing than by using inspections
  • Code inspection at NASA's Software Engineering Laboratory found 20 to 60 percent more errors than testing did
  • A study of 13 reviews at AT&T found that the importance of the review meeting itself was overrated; 90% of the defects were found in preparation for the review meeting and only about 10% were found during the review itself
  • IBM found that each hour of inspection prevented about 100 hours of related work
  • As much as 90% of development effort on a typical software system comes after its initial release, with two-thirds being typical
  • Designers and programmers learn to improve their work through participating in inspections and inspections improve productivity by 20% or more
  • About 85% of errors can be fixed in a few hours if caught in the development stage
  • Construction errors detected during system testing cost 10 times more to fix than in the construction phase and 10-25 times more to fix when detected post-release
  • Correcting defects earlier rather than later in a project can cut overall development costs by factors of two or more
  • ICI found that maintaining a portfolio of about 400 programs was only about 10% of the cost of maintaining a similar set of programs that had not been inspected
  • A study of large programs found that every hour spent on inspections avoided an average of 33 hours of maintenance
  • Increased quality assurance is associated with a decreased error rate but does not increase overall development cost
• When should you inspect software?
  • The average project experiences about a 25 percent change in requirements during development
  • 70 to 85 percent of rework on a typical project is caused by changes in requirements during development
  • Raytheon reduced its cost of rework from about 40% of total project cost to 20% through an initiative that focused on inspections
  • Software defect removal is the most expensive and time-consuming form of work for software
  • NASA's Software Engineering Laboratory found that code inspection detected about 3.3 defects per hour of effort, compared to 1.8 errors per hour of testing effort
  • The average cost of finding an error using code inspections is 3.5 staff hours compared to 15-25 hours to find each error through testing
• What should you look for?
  • A study showed that when routines averaged 100 to 150 lines each, code was more stable and required less changes
  • Routines with 143 lines or less are 2.4 times less expensive to fix than larger routines
  • A study at IBM found that the most error-prone routines were those larger than 500 lines of code
  • Few people can understand more than 3 levels of nested "if" statements
  • Many researchers recommend avoiding going to more than four levels
  • The DRY principle: Don't Repeat Yourself
  • Control-flow complexity has been correlated with low reliability and frequent errors
  • The competent programmer is fully aware of the strictly limited size of his own skull
  • Copy and Paste is a design error
  • 39% of all errors are caused by internal interface errors / errors in communication between routines
  • 50% to 80% of plain "if" statements should have had an "else" clause
  • A study of 450 routines found that 50 percent of highly cohesive programs were fault free, compared to only 18 percent of routines with low cohesion
  • Given a sample of 450 routines, the routines with the highest coupling-to-cohesion ratios had 7 times more errors than those with the lowest ratios
  • Routines with low cohesion can cost 20 times more to fix than fixing highly cohesive routines
  • The effort required to debug a program is minimized when variables had names averaging 10 to 16 characters long
  • 46 percent of routines with no unused variables had no errors compared to only 17 to 29 percent for those with one or more unreferenced variables
  • Large programs that use information hiding are a factor 4 easier to modify than programs which don't