Tag Archives: quality

Who needs Formal Measurement?

Posted on by
Reply

We all know the expression “You can’t manage what you can’t measure“, but do we really understand it?

After execution, feedback is an essential part of all processes.  Just think about how difficult it would be to drive from home to work wearing a blindfold.  Without your sense of sight to give you feedback on the traffic signals and the locations of other cars you would crash your car.  Yet we develop software systems without instituting formal measurement programs all the time and wonder why we succeed so rarely?  (for success rates see Understanding your chances of having a successful software project)

You can’t manage what you can’t measure

No measurement means no feedback, which means your chances of success are minimized. Success is possible without formal measurement but it is much easier with formal measurement.

Formal measurement raises productivity by 20.0% and quality by 30.0%

A best practice is one that increases your chance of succeeding, it does not guarantee it. It has been established that formal measurement is a best practice, so why do so few people do it?

Measurement has a cost and organizations are petrified of incurring costs without incurring benefits. After all what if you institute a measurement program and things don’t improve?  In some sense managers are correct that measurement programs cost money to develop and unless measurement is executed correctly it will not yield any results.  But is there a downside to avoiding measurement?

Inadequate progress tracking reduces productivity by 16.0% and quality by 22.5%

Failure to estimate requirements changes reduces productivity by 14.6% and quality by 19.6%

Inadequate measurement of quality reduces productivity by 13.5% and quality by 18.5%

So there are costs to not having measurement.  Measurement is not optional, measurement is a hygiene process, that is, essential to any process but especially to software development where the main product is intangible.

A hygiene process is one which can prevent very bad things from happening. Hygiene processes are rarely fun and take time, i.e. taking a shower, brushing your teeth, etc.  But history has show that it is much more cost effective to execute a hygiene process than take a chance of something very bad from happening, i.e. disease or your teeth falling out.

There are hygiene practices that we use every day in software development without even thinking about it:

  • Version control
  • Defect tracking

Version control is not fun, tracking defects is not fun; but the alternative is terrible.  Only the most broken organizations think that they can develop software systems without these tools.  These tools are not fun to use and virtually everyone complains about them, but the alternative is complete chaos.

Formal measurement is a best practice and a hygiene practice

The same way that developers understand that version control and defect tracking is necessary, an organization needs to learn that  measurement is necessary.
Is Formality Necessary?

The reality is that informal measurement is not comprehensive enough to give consistent results. If measurement is informal then when crunch time comes then people will stop measuring things when you need the data the most.

When you don’t have enough formality then processes take longer and by extension cost more.  When you have too much formality then you have process for processes sake and things will also take a long time.  Any organization that implements too much formality is wasting their time, but so is any organization that does not implement enough.

When you suggest any formal process people immediately imagine the most extreme form of that process; which would be ridiculous if it is implemented that way. We have all been in organizations that implement processes that make no sense, but without measurement how do you get rid of these processes that make no sense? For every formal process that makes sense, there is a spectrum of implementations. The goal is to find the  minimum formality that reduces time and costs. When you find the minimum amount of formal measurement you will accelerate your development by giving yourself the feedback that you need to drive your development.

What to Measure

It seems obvious, but incorrect measurement and/or poor execution leads to useless results.  For example, trying to measure productivity by measuring the hours that the developers sit at their machines is as useful as measuring productivity by the number of cups of coffee that the developers drink.Another useless measure is lines of code (LOC), in fact, Capers Jones believes that anyone using LOC as a measurement should be tried for professional malpractice!Measuring the the three things mentioned above will improve productivity and quality because there will not be a negative effect on your organization:

  • Measuring progress tracking (productivity +16.0%, quality +22.5%)
  • Estimating requirements changes  (productivity +14.6%, quality +19.6%)
  • Measurement of quality (productivity +13.5%, quality +18.5%)

Other things to measure are:

  • Activity based productivity measures
    (productivity +18.0%, quality by 6.7%)
  • Automated sizing tools (function points)
    (productivity +16.5%, quality by 23.7%)
  • Measuring requirement changes  (productivity +15.7%, quality by 21.9%)

So to answer the question: who needs formal measurement?

We all need formal measurement

 

References

N.B. All productivity and quality percentages were derived over 15,000+ actual projects

Articles in the “Loser” series

Want to see sacred cows get tipped? Check out:

Make no mistake, I am the biggest “Loser” of them all.  I believe that I have made every mistake in the book at least once 🙂

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Not planning is for Losers

Posted on by
Reply

Only the ignorant don’t plan their code pathways before they write them.  Unless you are implementing classes of only getter and setter routines code needs to be planned. We talk about The Path Least Traveled The total number of pathways through a software system grow so quickly that it is very hard to imagine their total number. If a function X() with 9 pathways calls function Y() which has 11 pathways then the composition function X() ° Y() will have up to 9 x 11 = 99 possible pathways. If function Y() calls function Z() with 7 pathways, then X()  ° Y()  ° Z()  will have up to 693 = 9 x 11 x 7 pathways. The numbers add up quickly, for example a call depth of 8 functions each with 8 pathways means 10.7 million different paths; the number of possible pathways in a system is exponential with the total depth of the call tree. Programs, even simple ones, have hundreds if not thousands (or millions) of pathways through them.

Negative vs Positive Assurance

Quality assurance can only come from the developers, not the testing department.  Testing is about negative assurance, which is only a statement that “I don’t see anything wrong”; it doesn’t mean that everything is correct, just that they can’t find a problem.  Positive assurance which is guaranteeing that the code will execute down the correct pathways and only the developer can do that. Quality assurance comes from adopting solid practices to ensure that code pathways are layed down correctly the first time

Any Line of Code can be Defective

If there are 10 pathways through a function then there there must be branching statements based on variable values to be able to direct program flow down each of the pathways. Each pathway may compute variable values that may be used in calculations or decisions downstream. Each downstream function can potentially have its behavior modified by any upstream calculation. When code is not planned then errors may cause execution to compute a wrong value.  If you are unlucky that wrong value is used to make a decision which may send the program down the wrong pathway.  If you are really unlucky you can go very far down the wrong pathways before you even identify the problem.  If you are really, really, really unlucky not only do you go down the wrong pathway but the data gets corrupted and it takes you a long time to recognize the problem in the data. It takes less time to plan code and write it correctly than it takes to debug complex pathways.

Common Code Mistakes

Defects are generally caused either because of one of the following conditions:

  1. incorrect implementation of an algorithm
  2. missing pathways
  3. choosing the wrong pathway based on the variables

1) Incorrect implementation of an algorithm will compute a wrong value based on the inputs.  The damage is localized if the value is computed in a decision statement, however, if the value is computed in a variable then damage can happen everywhere that value is used.  Example, bad decision at node 1 causes execution to flow down path 3 instead of 2. 2) Missing pathways have to deal with conditions.  If you have 5 business conditions and only 4 pathways then one of your business conditions will go down the wrong pathway and cause problems until you detect the problem.  Example, there was really 5 pathways at node 1, however, you only coded 4. 3a) The last problem is that the base values are correct but you select the wrong pathway.  This can lead to future values being computed incorrectly.  Example: at node 10 you correctly calculate that you should take pathway 11 but end up going down 12 instead. 3b) You might also select the wrong pathway because insufficient information existed at the time that you needed to make a decision.  Example:  insufficient information at node 1 causes execution to flow down path 3 instead of 4. The last two issuses (2 or 3) can either be a failure of development or of requirements.  In both cases somebody failed to plan…

What if it is too late to Plan

Whenever you are writing a new section of code you should take advantage of the ability to plan the code before you write it.  If you are dealing with code that has already been written then you should take advantage of inspections to locate and remove defects.  Don’t wait for defects to develop, proactively inspect all code, especially in buggy modules and fix all of the code pathways. Code inspections can raise productivity by 20.8% and quality by 30.8%

Code Solutions

The Personal Software Process (PSP) has a specific focus that every code section should be planned by the developer before it is implemented.  That means that you sit down and plan your code pathways using paper or a white board before using the keyboard.  Ideally you should spend the first part of your day planning with your colleagues on how best to write your code pathways.  The time that you spend planning will pay you dividends in time saved. PSP can raise productivity by 21.2% and quality by 31.2% If you insist on writing code at the keyboard then you can use pair programming to reduce errors.  By having a second pair of eyes looking at your code algorithmic mistakes are less likely and incorrect decisions for conditions are looked at by two people.  The problem is that pair programming is not cost effective overall. Pair Programming can raise productivity by 2.7% and quality by 4.5% Studies confirm that code sections of high cyclomatic complexity have more defects than other code sections.   At a minimum, any code section that will have a high cyclomatic complexity should be planned by two or more people.  If this is not possible, then reviewing sections of high cyclomatic complexity can reduce downstream defects. Automated cyclomatic complexity analysis can raise productivity by 14.5% and quality by 19.5% Design Solutions All large software projects benefit from planning pathways at the macroscopic level.  The design or architectural planning is essential to making sure that the lower level code pathways will work well. Formal architecture for large applications can raise productivity by 15.7% and quality by 21.8% Requirements Solutions Most pathways are not invented in development.  If there is insufficient information to choose a proper pathway or there are insufficient pathways indicated then this is a failure of requirements.  Here are several techniques to make sure that the requirements are not the problem. Joint application design (JAD) brings the end-users of the system together with the system architects to build the requirements.  By having end-users present you are unlikely to forget a pathway and by having the architects present you can put technical constraints on the end-users wish list for things that can’t be built. The resulting requirements should have all pathways properly identified along with their conditions. Joint application design can raise productivity by 15.5% and quality by 21.4% Requirements inspections are the best way to make sure that all necessary conditions are covered and that all decisions that the code will need to make are identified before development.  Not inspecting requirements is the surest way to discovering that there is a missing pathway or calculation after testing. Requirement inspections can raise productivity by 18.2% and quality by 27.0% Making sure that all pathways have been identified by requirements planning is something that all organizations should do.  Formal requirements planning will help to identify all the code pathways and necessary conditions, however, formal requirements planning only works when the business analysts/product managers are skilled (which is rare 🙁 ). Formal requirements analysis can raise productivity by 16.3% and quality by 23.2%

Other articles in the “Loser” series

Want to see more sacred cows get tipped? Check out
Moo?

Make no mistake, I am the biggest “Loser” of them all.  I believe that I have made every mistake in the book at least once 🙂

References

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Defects are for Losers

Posted on by
Reply

A developer is responsible for using any and all techniques to make sure that he produces defect free code.  The average developer does not take advantage of all of the following opportunities to prevent and eliminate defects:

  1. Before the code is written
  2. As the code is written
  3. Writing mechanisms for early detection
  4. Before the code is executed
  5. After the code is tested

The technique that is used most often is #5 above and will not be covered here.  It involves the following:

  1. Code is delivered to the test department
  2. The test department identifies defects and notifies development
  3. Developer’s fire up the debugger and try to chase down the defect

Like the ‘rinse and repeat‘ process on a shampoo bottle, this process is repeated until the code is cleaned or until you run out of time and are forced to deliver.

The almost ubiquitous use of #5 leads to CIOs and VPs of Engineering assuming that the metric of one tester to two developers is a good thing.  Before assuming that #5 is ‘the way to go‘ consider the other techniques and statistical evidence of their effectiveness.

Before the Code is Written



A developer has the most options available to him before the code is written.  The developer has an opportunity to plan his code, however, there are many developers who just ‘start coding’ on the assumption that they can fix it later.

How much of an effect can planning have?  Two methodologies that focus directly on planning at the personal and team level are the Personal Software Process (PSP) and the Team Software Process (TSP) invented by Watts Humphrey.

PSP can raise productivity by 21.2% and quality by 31.2%

TSP can raise productivity by 20.9% and quality by 30.9%

Not only does the PSP focus on code planning, it also makes developers aware of how many defects they actually create.  Here are two graphs that show the same group of developers and their defect injection rates before and after PSP training.

Before PSP training After PSP training

The other planning techniques are:

  • Decision tables
  • Proper use of exceptions

Both are covered in the article Debuggers are for Losers and will not be covered here.

As the Code is Written

Many developers today use advanced IDEs to avoid common syntax errors from occurring   If you can not use such an IDE or the IDE does not provide that service then some of the techniques in the PSP can be used to track your injection of syntax errors and reduce those errors.

Pair Programming

One technique that can be used while code is being written is Pair Programming.  Pair programming is heavily used in eXtreme Programing (XP).  Pair programming not only allows code to be reviewed by a peer right away but also makes sure that there are two people who understand the code pathways through any section of code.

Pair programming is not cost effective overall (see Capers Jones).  For example, it makes little sense to pair program code that is mainly boiler plate, i.e. getter and setter classes. What does make sense is that during code planning it will become clear which routines are more involved and which ones are not.  If the cyclomatic complexity of a routine is high (>15) then it makes sense for pair programming to be used.

If used for all development, Pair Programming can raise productivity by 2.7% and quality by 4.5%

Test Driven Development




Test driven development (TDD) is advocated by Kent Beck and stated in 2003 that TDD encourages simple designs and inspires confidence.  TDD fits into the category of automated unit testing.

Automated unit testing  can raise productivity by 16.5% and quality by 23.7%

Writing Mechanisms for Early Detection

Defects are caused by programs either computing wrong values, going down the wrong pathway, or both.  The nature of defects is that they tend to cascade and get bigger the further in time and space between the source of the defect and the noticeable effects of the defect.

Design By Contract

One way to build checkpoints into code is to use Design By Contract (DbC), a technique that was pioneered by the Eiffel programming language   It would be tedious and overkill to use DbC in every routine in a program, however, there are key points in every software program that get used very frequently.

Just like the roads that we use have highways, secondary roads, and tertiary roads — DbC can be used on those highways and secondary roads to catch incorrect conditions and stop defects from being detected far away from the source of the problem.

Clearly very few of us program in Eiffel.  If you have access to Aspect Oriented Programming (AOP) then you can implement DbC via AOP. Today there are AOP implementations as a language extension or as a library for many current languages (Java, .NET, C++, PHP, Perl, Python, Ruby, etc).

Before the Code is Executed

Static Analysis

Most programming languages out there lend themselves to static analysis.  There are cost effective static analysis for virtually every language.

Automated static analysis can raise productivity by 20.9% and quality by 30.9%

Inspections


Of all the techniques mentioned above, the most potent pre-debugger technique is inspections. inspections are not sexy and they are very low tech, but the result of organizations that do software inspections borders on miraculous.The power of software inspections can be seen in these two articles:

Code inspections can raise productivity by 20.8% and quality by 30.8%

Design inspections can raise productivity by 16.9% and quality by 24.7%

From the Software Inspections book on p.22.

In one large IBM project, one half million lines of networked operating system, there were 11 development stages (document types: logic, test, user documentation) being Inspected.  The normal expectation at IBM, at that time, was that they would be happy only to experience about 800 defects in trial site operation.  They did in fact experience only 8 field trial defects.

Evidence suggests that every 1 hour of code inspection will reduce testing time by 4 hours

Conclusion

Overworked developers rarely have time to do research, even though it is clear that there is a wealth of information available on how to prevent and eliminate defects. The bottom line is that if your are only using technique #5 from the initial list, then you are not using every technique available to you to go after defects.My opinion only, but:

A professional software developer uses every technique at his disposal to prevent and eliminate defects

Other articles in the “Loser” series

Want to see more sacred cows get tipped? Check out:

Make no mistake, I am the biggest “Loser” of them all.  I believe that I have made every mistake in the book at least once 🙂

References

Gilb, Tom and Graham, Dorothy. Software Inspections

Jones, Capers. SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.

Radice, Ronald A. High Quality Low Cost Software Inspections.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

NO Experience Necessary!!!

Posted on by
Reply

Did you know that we have never found a relationship between a developer’s years of experience and code quality or productivity?

The original study that found huge variations in individual programming productivity was conducted in the late 1960s by Sackman, Erikson, and Grant (1968).

This study has been repeated at least 8 times over 30 years and the results have not changed! (see below)

Sackman et al studied professional programmers with an average of 7 years’ experience and found that:

  • the ratio of initial coding time was about 20 to 1
  • the ratio of debugging times over 25 to 1
  • program execution speed about 10 to 1
  • program size 5 to 1

They found no relationship between a programmer’s number of years of experience and code quality or productivity.  That is there was NO correlation between experience and productivity (i.e. ability to produce code) and there was NO correlation between experience and quality (i.e. minimizing defects) .

Think about that for a minute…

That is the worst programmers and the best programmers made distinct groups and each group had people of low and high experience levels.  Whether training helps developers or not is not indicated by these findings, only that years of experience do not matter.

Without considering legality, this means that it is simpler to get rid of expensive poor performers with many years of experience and hire good performers with few years of experience!

Results Have Been Confirmed for 30 Years!

There were flaws in the study, however, even after accounting for the flaws, their data still shows more than an order of magnitude difference between the best programmers and the worst, and that difference was not related to experience.  In years since the original study, the general finding that “There are order-of-magnitude differences among programmers” has been confirmed by many other studies of professional programmers (full references at the end of the article):

  • Curtis 1981
  • Mills 1983
  • DeMarco and Lister 1985
  • Curtis et al. 1986
  • Card 1987
  • Boehm and Papaccio 1988
  • Valett and McGarry 1989
  • Boehm et al 2000

Technology is More Sophisticated, Developers are not

You might  think that we know much more about software development today than we knew in 1968, after all today:

  • we have better computer languages
  • we have more sophisticated technology
  • we have better research on effective software patterns
  • we have formal software degrees available in university

It turns out that all these things are true, but we still have order of magnitude differences among programmers and the difference is not related to years of experience.  That means that there is some other x-factor that drives productive developers;  that x-factor is probably the ability to plan and make good decisions.

The bad news is that if you are not a productive developer writing quality code  then you will probably not get better simply because of years of experience.

Developers face making decisions on how to structure their code every day.  There is always a choice when it comes to:

  • laying out code pathways
  • packaging functions into classes
  • packaging classes into packages/modules

Because developers face coding decisions, many of which are complex, the best developers will plan their work and make good decisions.  Bad developers just ‘jump in’; they assume that they can always rewrite code or make up for bad decisions later. Bad developers are not even aware that their decision processes are poor and that they can become much better by planning their work.

Solution might be PSP and TSP

Watts Humphrey tried to get developers to understand the value of estimating, planning development, and making decisions in the Personal Software Process (PSP) for individuals and the Team Software Process (TSP) for teams, but only a handful of organizations have embraced it.  Capers Jones has done analysis of over 18,000 projects and discovered that1:

PSP can raise productivity by 21.2% and quality by 31.2%
TSP can raise productivity by 20.9% and quality by 30.9%

All of these findings should have a profound effect on the way that we build our teams. Rather than having large teams of mediocre developers, it makes much more sense to have smaller teams of highly productive developers that know how to plan and make good decisions.  The PSP and TSP do suggest that the best way to rehabilitate a poor developer is to teach them how to make better decisions.

Be aware, there is a difference between knowledge of technologies which is gained over time and the ability to be productive and write quality code.

Conclusion

We inherently know this, we just don’t do it.  If the senior management of organizations only knew about these papers, we could make sure that the productive people get paid what they are worth and the non-productive people could seek employment in some other field.  This would not only reduce the cost of building software but also increase the quality of the software that is produced.

Unfortunately, we are doomed to religious battles where people debate methodologies, languages, and technologies in the foreseeable future.  The way that most organizations develop code makes voodoo look like a science!

Eventually we’ll put the ‘science’ back in Computer Science, I just don’t know if it will be in my lifetime…

Check out Stop It! No… Really stop it. to learn about 5 worst practices that need to be stopped right now to improve productivity and quality.

Bibliography

Boehm, Barry W., and Philip N. Papaccio. 1988. “Understanding and Controlling Software Costs.” IEEE Transactions on Software Engineering SE-14, no. 10 (October): 1462-77.

Boehm, Barry, et al, 2000. Software Cost Estimation with Cocomo II, Boston, Mass.: Addison Wesley, 2000.

Card, David N. 1987. “A Software Technology Evaluation Program.” Information and Software Technology 29, no. 6 (July/August): 291-300.

Curtis, Bill. 1981. “Substantiating Programmer Variability.” Proceedings of the IEEE 69, no. 7: 846.

Curtis, Bill, et al. 1986. “Software Psychology: The Need for an Interdisciplinary Program.” Proceedings of the IEEE 74, no. 8: 1092-1106.

DeMarco, Tom, and Timothy Lister. 1985. “Programmer Performance and the Effects of the Workplace.” Proceedings of the 8th International Conference on Software Engineering. Washington, D.C.: IEEE Computer Society Press, 268-72.

1Jones, Capers. SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.

Mills, Harlan D. 1983. Software Productivity. Boston, Mass.: Little, Brown.

Valett, J., and F. E. McGarry. 1989. “A Summary of Software Measurement Experiences in the Software Engineering Laboratory.” Journal of Systems and Software 9, no. 2 (February): 137-48.

VN:F [1.9.22_1171]
Rating: 4.3/5 (6 votes cast)
VN:F [1.9.22_1171]
Rating: +1 (from 1 vote)

Stop It! No… really stop it.

Posted on by
Reply

Stop, this means you!There are 5 worst practices that if stopped immediately will  improve your productivity by a minimum of 12% and improve quality by a minimum of 15%.  These practices are so common that people assume that they are normal — they are not, they are silent killers wherever they are present.

We often hear the term best practices enough to know that we all have different definitions for it.  Even when we agree on best practices we then disagree on how to implement and measure them. A best practice is one that increases the chance your project will succeed.

How often do we talk about worst practices?  More importantly, what about those worst practices in your organization that you don’t do anything about?

CatConeFail

When it comes to a worst practice, just stop it.

If your company is practicing even one worst practice in the list below it will kill all your productivity and quality. It will leave you with suboptimal and defective software solutions and canceled projects.

To make matters worse, some of the worst practices will cause other worst practices to come into play.   Capers Jones had statistics on over 18,000 projects and has hard evidence on the worst practices1.  The worst practices and their effect on productivity and quality are as follows:

Worst Practice Productivity Quality
Friction/antagonism among team members -12.0% -15.0%
Friction/antagonism among management -13.5% -18.5%
Inadequate communications with stakeholders -13.5% -18.5%
Layoffs/loss of key personnel -15.7% -21.7%
Excessive schedule pressure -16.0% -22.5%

Excessive Schedule Pressure

Excessive schedule pressure is present whenever any of the following are practiced:

Excessive schedule pressure causes the following to happen:

This alone can create a Death March project and virtually guarantee project failure.

Effect of excessive schedule pressure is that productivity will be down 16% and quality will be down 22%

Not only is excessive schedule pressure one of the worst practices it tends to drive the other worst practices:

  • Friction amongst managers
  • Friction amongst team members
  • Increases the chance that key people leave the organization

If your organization has a habit of imposing excessive schedule pressure — leave!

Friction Between People

Championship TrophySoftware development is a team activity in which we transform our intangible thoughts into tangible working code.  Team spirit and collaboration is not an option if you want to succeed.  The only sports teams that win championships are those that are cohesive and play well together.

You don’t have to like everyone on your team and you don’t have to agree with all their decisions.  However, you must understand that the team is more important than any single individual and learn to work through your differences.

Teams only work well when they are hard on the problem, not each other

Fighting ManagersFriction among managers because of different perspectives on resource allocation, objectives, and requirements.  It is much more important for managers to come to a consensus than to fight for the sake of fighting. Not being able to come to a consensus will cave in projects and make ALL the managers look bad. Managers win together and lose together.

Effect of management friction is that productivity will be down 13.5% and quality will be down 18.5%

Team FrictionFriction among team members because of different perspectives on requirements, design, and priority.  It is also much more important for the team to come to a consensus than to fight for the sake of fighting.  Again, everyone wins together and loses together — you can not win and have everyone else lose.

Effect of team friction is that productivity will be down 12% and quality will be down 15%

Any form of friction between managers or the team is deadly.

Inadequate Stakeholder Communication

Inadequate stakeholder communication comes in several forms:

  • Not getting enough information on business objectives
  • Not developing software in a transparent manner

If you have insufficient information on the business objectives of a project then you are unlikely to capture the correct requirements.  If you are not transparent in how you are developing the project then you can expect excessive schedule pressure from senior management.

Effect of inadequate stakeholder communication is that productivity will be down 13.5% and quality will be down 18.5%

Loss of Key Personnel

To add insult to injury, any of the other four worst practices above will lead to either:

  • Key personnel leaving your organization
  • Key personnel being layed off

I Quit!!Badly managed organizations and projects will cause the most competent people to leave the organization, simply because they can more easily get a job in another organization.

When organizations experience financial distress from late projects they will often cut key personnel because they are expensive.  The reality is that laying off key personnel will sandbag your ability to get back on track.  The correct thing to do is to find your least effective personnel and let them go.

Effect of layoffs/loss of key personnel is that productivity will be down 15.7% and quality will be down 21.7%

The loss of key personnel has a dramatic effect on team productivity and morale and a direct effect on product quality.

Conclusion

Any of the worst practices mentioned above will cause a project to be late and deliver defective code. Even worse, the worst practices tend to feed each other and cause a negative spiral. If you are in an organization that habitually practices any of these worst practices then your only real option is to quit.

The most deadly situation is when there is the following cascading of worst practices:

  • Excessive schedule pressure (leads to)
  • Management and team friction (leads to)
  • Loss of key personnel

If you are in senior management then none of these practices can be allowed if you want to avoid canceled projects or highly defective products.

1Jones, Capers. SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Efficiency is for Losers

Posted on by
Reply

Focusing on efficiency and ignoring effectiveness is the root cause of most software project failures.

Effectiveness is producing the intended or expected result. Efficiency is the ability to accomplish a job with a minimum expenditure of time and effort.

Effective software projects deliver code that the end users need; efficient projects deliver that code with a minimum number of resources and time.

Sometimes, we become so obsessed with things we can measure, i.e. project end date, kLOC, that we somehow forget what we were building in the first place.  When you’re up to your hips in alligators, it’s hard to remember you were there to drain the swamp.

Efficiency only matters if you are being effective.

After 50 years, the top three end-user complaints about software are:

  1. It took too long
  2. It cost too much
  3. It doesn’t do what we need
Salaries are the biggest cost of most software projects, hence if it takes too long then it will cost too much, so we can reduce the complaints to:

  1. It took too long
  2. It doesn’t do what we need

The first issue is a complaint about our efficiency and the second is a complaint about our effectiveness. Let’s make sure that we have common  definitions of these two issues before continuing to look at the interplay between efficiency and effectiveness.

Are We There Yet?

Are you late if you miss the project end date? 

That depends on your point of view; consider a well specified project (i.e. good requirements) with a good work breakdown structure that is estimated
by competent architects to take a competent team of 10 developers at least 15 months to build. Let’s consider 5 scenarios where this is true except as stated below:

Under which circumstances is a project late?

A. Senior management gives the team 6 months to build the software.
B. Senior management assigns a team of 5 competent developers instead of 10.
C. Senior management assigns a team of 10 untrained developers
D. You have the correct team, but, each developer needs to spend 20-35% of their time maintaining code on another legacy system
E. The project is staffed as expected

Here are the above scenarios in a table:

#
Team
Resource
Commitment
Months Given
Result
A
10 competent developers
100%
6
Unrealistic estimate
B
5
competent developers
100%
15
Under staffed
C
10 untrained developers
100%
15
Untrained staff
D
10 competent developers
65-80%
15
Team under committed
E
10 competent developers
100%
15
Late


Only the last project (E) is late because the estimation of the end date was consistent with the project resources available.

Other well known variations which are not late when the end date is missed:

  • Project end date is a SWAG or management declared
  • Project has poor requirements
  • You tell the end-user 10 months when the estimate is 15 months.

If any of the conditions of project E are missing then you have a problem in estimation.  You may still be late, but not based on the project end date computed with bad assumptions

Of course, being late may be acceptable if you deliver a subset of the expected system.

It Doesn’t Work



“It doesn’t do what we need” is a failure to deliver what the end user needs. How so we figure out what the end user needs?

The requirements for a system come from a variety of sources:

  1. End-users
  2. Sales and marketing (includes competitors)
  3. Product management
  4. Engineering

These initial requirements will rarely be consistent with each other. In fact, each of these constituents will have a different impression of the requirements. You would expect the raw requirements to be contradictory in places. The beliefs are like the 4 circles to the left, and the intersection of their beliefs would be the black area.

The different sources of requirements do not agree because:

  • Everyone has a different point of view
  • Everyone has a different set of beliefs about what is being built
  • Everyone has a different capability of articulating their needs
  • Product managers have varying abilities to synthesize consistent requirements
It is the job of product management to synthesize the different viewpoints into a single set of consistent requirements. If engineering starts before
requirements are consistent then you will end up with many fire-fighting meetings and lose time.

Many projects start before the requirements are consistent enough. We hope the initial requirements are a subset of what is required.
In practice, we have missed requirements and included requirements that are not needed (see bottom of post, data from Capers Jones)

The yellow circle represents what we have captured, the black circle represents the real requirements.

We rarely have consistent requirements when we start a project, that is why there are different forms of the following cartoon lying around on the Internet.

If you don’t do all the following:

  • Interview all stakeholders for requirements
  • Get end-users to articulate their real needs by product management
  • Synthesize consistent requirements

Then you will fail to build the correct software.  So if you skip any of this work then you are guaranteed to get the response, “It doesn’t do what we need”.

Effectiveness vs. Efficiency

So, let’s repeat our user complaints:
  1. It took too long
  2. It doesn’t do what we need

It’s possible to deliver the correct software late.

It’s impossible to deliver on-time if the software doesn’t work

Focusing on effectiveness is more important than efficiency if a software project is to be delivered successfully.


Ineffectiveness Comes from Poor Requirements

Most organizations don’t test the validity or completeness of their requirements before starting a software project.
The requirements get translated into a project plan and then the project manager will attempt to execute the project plan. The project plan becomes the bible and everyone marches to it. As long as tasks are completed on time everyone assumes that you are effective, i.e. doing the right thing.

That is until virtually all the tasks are jammed at 95% complete and the project is nowhere near completion.

At some point someone will notice something and say, “I don’t think this feature should work this way”. This will provoke discussions between developers, QA, and product management on correct program behavior. This will spark a series of fire-fighting meetings to resolve the inconsistency, issue a defect, and fix the problem. All of the extra meetings will start causing tasks on the project plan to slip.

We discussed the root causes of fire-fighting in a  previous blog entry.

When fire-fighting starts productivity will grind to a halt. Developers will lose productivity because they will end up being pulled into the endless meetings. At this point the schedule starts slipping and we become focused on the project plan and deadline. Scope gets reduced to help make the project deadline; unfortunately, we tend to throw effectiveness out the window at this point.

With any luck the project and product manager can find a way to reduce scope enough to declare victory after missing the original deadline.

The interesting thing here is that the project failed before it started. The real cause of the failure would be the inconsistent requirements.But, in the chaos of fire-fighting and endless meetings, no one will remember that the requirements were the root cause of
the problem.

What is the cost of poor requirements? Fortunately, WWMCCS has an answer.  As a military organization they must tracks everything in a detailed fashion and perform root cause analysis for each defect (diagram).

This drawing shows what we know to be true.

The longer a requirement problem takes to discover, the harder and more expensive it is to fix!  A requirement that would take 1 hour to fix will take 900 hours to fix if it slips to system testing.

Conclusion

It is much more important to focus on effectiveness during a project than efficiency. When it becomes clear that you will not make the project end date, you need to stay focused on building the correct software.
Are you tired of the cycle of:
  • Collecting inconsistent requirements?
  • Building a project plan based on the inconsistent requirements?
  • Estimating projects and having senior management disbelieve it?
  • Focusing on the project end date and not on end user needs?
  • Fire-fighting over inconsistent requirements?
  • Losing developer productivity from endless meetings?
  • Not only miss the end date but also not deliver what the end-users need?

The fact that organizations go through this cycle over and over while expecting successful projects is insanity – real world Dilbert cartoons.

How many times are you going to rinse and repeat this process until you try something different? If you want to break this cycle, then you need to start collecting consistent requirements.

Think about the impact to your career of the following scenarios:

  1. You miss the deadline but build a subset of what the end-user needs
  2. You miss the deadline and don’t have what the end-user needs
You can at least declare some kind of victory in scenario 1 and your resume will not take a big hit. It’s pretty hard to make up for scenario 2 no matter how you slice it.
Alternatively, you can save yourself wasted time by making sure the requirements are consistent before you start development. Inconsistent requirements will lead to fire-fighting later in the project.
As a developer, when you are handed the requirements the team should make a point of looking for inconsistent requirements.
The entire team should go through the requirements and look for inconsistencies and force product management to fix them before you start developing.
It may sound like a waste of time but it will push the problem of poor requirements back into product management and save you from being in endless meetings. Cultivating patience on holding out for good requirements will lower your blood pressure and help you to sleep at night.
Of course, once you get good requirements then you should hold out for proper project estimates 🙂
Want to see other sacred cows get tipped?Check out:
Moo?

Courtesy of Capers Jones via LinkedIn on 6/22

Customers themselves are often not sure of their requirements.

For a large system of about 10,000 function points, here is what might be seen for the requirements.

This is from a paper on requirements problems – send an email to capers.jones3@gmail.com if you want a copy.

Requirements specification pages = 2,500
Requirements words = 1,125,000
Requirements diagrams = 300
Specific user requirements = 7,407
Missing requirements = 1,050
Incorrect requirements = 875
Superfluous requirements = 375
Toxic harmful requirements = 18

Initial requirements completeness = < 60%

Total requirements creep = 2,687 function points

Deferred requirements to meet schedule = 1,522

Complete and accurate requirements are possible < 1000 function points. Above that errors and missing requirements are endemic.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)