Who should set defect priority?

Posted on November 19, 2013 by Dalip Mahal

Surprisingly, defect priority should not be set by QA. QA are generally the owners of the defect tracking system and control it, but this is one attribute that they should not control. The defect tracker is a shared resource between QA, engineering, engineering management, and product mangement and is a coordinating mechanism for all these parties.

Commonly people mix up priority and severity, for example, there may be a severe defect that causes the software either not to install or to cease functioning.

It is common for new releases to have various installation problems when it initially gets to QA. This blocks QA so they mark the defects with a high severity and high priority. This issue has a high severity and needs to be addressed right away, but remember bug tracking systems are append only — once this defect gets into the system, it will never get out. This kind of issue should be escalated to the engineers and engineering management because it makes little sense to clog the defect tracking system with it.

Now there may be intermittent issues that cause the software to fail and you may assume that this defect would be high priority, but if this defect occurs very rarely and would cost too much to fix then this defect may be a low priority. Once again the priority of an intermittent severe issue can not be determined by QA.

Similarly, there may be many cosmetic or minor defects where fixing them might make a huge difference in the user experience and reduce support calls. Even though these defects are minor, they may be easy to fix and save you serious money. Once again, this can not be decided by QA.

Therefore, QA should reserve the right to set the initial severity, and may have an internal field for QA priority, but the priority of a defect should be determined by a product manager (PM) who has a more complete understanding of the overall context of the product. The priority of a defect is a business issue, not an engineering or QA issue.

Ideally, the product manger will set the priority of a defect during a defect triage session where representatives from engineering, engineering management, and QA are present. As you go through each new defect each person can present their logic for what the priority should be. The PM should then set the priority defect and assign the release it should be fixed in (i.e. this release, minor release, major release, won’t fix). Ideally there are extra fields in the defect record for both QA and development to put their priority beliefs.

It is important that the status of a defect not include any of the following or your defect tracker will go to hell (see Bug Tracker Hell and How to Get Out):

Priority
Severity
Fix version

It is very hard to generate meaningful reports when these attributes creep into the defect status. You know that this has happened if you have a multi-page training manual for entering defects into the system. Some of the statuses that make sense initially but turn the defect system into a nightmare are:

FAD (functions as designed)
WontFix
NextVersion
CantReproduce

Of course, if you don’t have regular bug triage sessions with all the parties mentioned above then you are probably sand-bagged in fire-fighting.

Enhancements don’t belong in the bug tracker

Posted on October 26, 2013 by Dalip Mahal

As development progresses we inevitably run into functionality gaps that are either deemed as enhancements.

These issues often get captured by QA in the bug tracker and assigned to a developer.

Enhancements should not be managed from the bug tracker

The life cycle of a defect and the life-cycle of a enhancement are two entirely different things. A defect is a difference between a stated requirement and the code. If there is no documentation there is no code defect (see It’s not a bug, it’s…) — in fact, most enhancements will eventually be coded by some developer; they just should not be managed from the bug tracker.

Defect Life-cycle

The defect life-cycle is well known:

Defect is identified as a departure from the requirements
Defect is assigned to a developer
Defect is corrected
Correct is verified
- If not corrected re-open and re-assign to developer
The defect is closed

This is the incorrect way to manage enhancements. When a functionality gap is determined by QA and it is not covered by the requirements then we have an issue. It is rarely the case that the issue can be resolved by the developer.

Enhancement Life-cycle

If enhancements are assigned to a developer then they are likely to try to resolve the issue. The problem is that “enhancements” determined at the QA level may be phantom problems caused by either:

Insufficient requirements
Correct requirements but incorrect test plans

Enhancements may or may not become code changes. Even when enhancements turn into code change requests they will generally not be implemented as the developer or QA think they should be implemented.

Enhancements are really requirement defects. Enhancements should be logged as such in the bug tracker and assigned to the person in charge of requirements (business analyst or product manager). Those individuals should be responsible to track down how these issues should be handled.

If the requirements are correct and the test plans are defective then it should be logged as a test defect. This is tricky because QA often controls the bug tracker and will not log errors that they have made.

At a minimum, the implementation of requirements and test defects can do several positive things for you:

It removes the responsibility to find a solution from development.
It makes it clear how many defects are in the requirements or test plans.
It reduces stress; no developer wants to be blamed for an issue that is not his.
Many enhancements call for updated project plans and pushing back the deadline.

Put Responsibility Where it Belongs

The creation of requirements and test defects in the bug tracker goes a long way to cleaning up the bug tracker. In fact, requirements and test defects represent about 25% of defects in most systems (see Bug Tracker Hell and How to Get Out!). The percentages break down as follows:

Requirements defects: 9.58%
Testing defects: 15.42%

The creation of requirement and test defects in the bug-tracker alleviate pressure on the engineering department and redirect it to either the product manager or QA. Eventually enough data will accumulate in the bug-tracker to get management’s attention.

At a minimum, these categories should help reduce the amount of fire-fighting in late projects (Root cause of ‘Fire-Fighting’ in Software Projects)

Rating: 0.0/5 (0 votes cast)

Rating: 0 (from 0 votes)

It’s not a bug, it’s…

Posted on October 22, 2013 by Dalip Mahal

When does a bug become a bug?
Who decides that it is a bug?

How many legs does a lamb have if I say the tail is a leg? The answer is 4, just because I say the tail is a leg does not make it a leg!

Bugs should be obvious, but we say It’s not a bug, it’s a feature because often it isn’t obvious. Watson Humphrey felt that we should use the term defect and not bug because most people don’t take bugs seriously, so let’s use the term defect instead.

So when does a defect become a defect?

When quality assurance tells you that you have a defect?
When product management says that it is a defect?
When the customer says that it is a defect?

The answer is: none of the above.

Now it might turn out that there is a problem and that code needs to change, but a defect only exists if:

code behaves differently than the requirements specification

This is important because most systems are under specified (if they are specified at all) and so when code misbehaves it is only a defect if the code behavior differs from the specification. We call defects undocumented features because we know that the problem is that the requirements were never written.

Incomplete and Inconsistent Requirements

Many organizations do not create sufficiently complete requirements before starting development, either because they don’t know how to capture requirements properly or because they don’t have resources capable of capturing complete requirements. Incomplete (and inconsistent) requirements and unrealistic deadlines often force developers into making decisions about how to implement features. The end result is that developers are regularly told that they have defects in their code.

While this process is common, it is destructive. When requirements are under specified and inconsistent developers end up needing to perform serious rework. The rework will can require dramatic changes that will impact the architecture of the code.

The time required to find a work around (if it is possible) is rarely included in the project plan. Complicating matters is that the organizations that are reluctant to spend time creating requirements also tend to underestimate their projects. This puts tremendous pressure on the engineering department to deliver; this promotes the 5 worst practices in software development (see Stop It! No… really stop it.)

Only 54% of Issues are Resolved by Engineers

The attitude that all defects must be resolved by the engineering department is severely misguided. Analysis by Capers Jones of over 18,000+ projects shows that only about 54% of all defects can be resolved by the engineers! (only the 3 highlighted rows below)

Defect Role Category	Frequency	Role
Requirements defect	9.58%	BA/Product Management
Architecture or design defect	14.58%	Architect
Code defect	16.67%	Developer
Testing defect	15.42%	Quality Assurance
Documentation defect	6.25%	Technical Writer
Database defect	22.92%	Data base administrator
Website defect	14.58%	Operations/Webmaster

This means that precious time will be wasted assigning issues to developers that they can not resolve. The time necessary to redirect the issue to the correct person is a major contributing factor to fire-fighting

Getting Control of the Defect Process

For most organizations fixing the defect process involves understanding and categorizing defects correctly. Organizations that are not tracking the different sources of defects probably have a bug tracker that has gone to hell. Here is how you can fix that problem, see Bug Tracker Hell and How To Get Out!

At a minimum you need to implement the requirements defect, once you identify issues that are caused by poor requirements it will shine the white hot light of shame onto the resources that are capturing your requirements. Once you realize how many requirements defects exist in your system you can begin to inform senior management about the requirements problem.

Reducing Fire-Fighting

The best way to reduce fire fighting is to start writing better requirements (or writing requirements 🙂 ). To do so you need to figure out which of the following are broken:

Not enough time is allocated to the requirements phase
Unskilled people are capturing your requirements

In all likelihood both of these issues need to be fixed in your organization. When requirements are incomplete and inconsistent you will have endless fire-fighting meetings involving everyone (see Root cause of ‘Fire-Fighting’ in Software Projects)

Stand your ground if someone tells you that you have coded a defect when there is no documentation for the requirement.

Rating: 0.0/5 (0 votes cast)

Rating: 0 (from 0 votes)

Stupid is as stupid does

Posted on October 11, 2013 by Dalip Mahal

Senior management does not set out to have a failed project, however, when the failure rate is 7 out of 10 projects you wonder what the problem is.

As Ma Gump stated, “stupid is as stupid does“, that is, smart people sometimes do stupid things. The collective IQ of a project is reduced by the number of managers involved in the project.

Now from a human perspective, something very interesting is going on here. For example, suppose you were give the following choices:

An 80% chance of making $100
A guaranteed $40

If people behaved according to expected value then everyone would choose the first choice. However, because humans are risk averse, almost everyone will choose the second alternative with guaranteed money.

Now out of 10 software projects:

3 will succeed
4 will be challenged
3 will outright fail

To put that in perspective, if you were watching people cross the street at an intersection:

3 cross the street successfully
4 get maimed
3 get killed

How interested would you be in crossing that street?

You can Google “software project failure rates” to see that this has been demonstrated by multiple reliable institutions over every industry. Challenged projects are generally projects which go over budget and under deliver a software solution that can be declared a moral victory by management.

So if human beings are risk averse and the odds of project success are so low then:

Why does senior management ignore risks on software projects?

Damn the torpedoes, full speed ahead The only possible conclusion is that senior management can’t conceive of a their projects failing. They must believe that every software project that they initiate will be successful, that other people fail but that they are in the 3 out of 10 that succeed.

This inability to understand the base rate of failure in software development is systemic. There are so many software projects that are started by senior management where the technical team knows that the chance of success is 0% from the start.

Senior management is human and is risk averse, you just need to find a way to remind them of this. One way to get senior management to think twice about projects is to make sure that there is a meeting before launching the project where management is asked the following question:

Assume that this project will fail, why would it have failed? What will the consequences be?

This exercise (if done seriously) may have the effect of causing senior management to realize that the project can indeed fail. With luck, the normal risk aversion that every human being is endowed with will kick in and the project may get re-evaluated.

Why Management Declared Deadlines Lead to Disaster
Stop It! No… really stop it
- Article about the 5 most common worst practices in software
Shift Happens
- Why scope shift is inevitable in software projects

Bibliography

Kahneman, Daniel. Thinking, Fast and Slow.

Rating: 0.0/5 (0 votes cast)

Rating: 0 (from 0 votes)

Size Matters

Posted on September 30, 2013 by Dalip Mahal

Some say that software development is challenging because of complexity. This might be true, but this definition does not help us find solutions to reduce complexity. We need a better way to explain complexity to non-technical people.

The reality is that when coding a project size matters. Size is measured in the number of pathways through a code base, not by the number of lines of code. Size is proportional to the number of function points in a project.

There are many IT people that succeed with programs of a certain size and then fail miserably when taking on programs that are more sophisticated. Complexity increases with size because the number of pathways increase exponentially in large programs.

Virtually anyone (even CEOs 🙂 ) can build a hello, world! application; an application that only has a single pathway through it and is as simple as you can get. Some CEOs write the simple hello, world! program and incorrectly convince themselves that development is easy. Hello, world! only has a single pathway through it and virtually anyone can write it.

main() {
printf( “hello, world” );
}

If you have an executive that can’t even complete hello,world then you should take away his computer 🙂

Complexity Defined

As programs get more sophisticated, the number of decisions that have to be made increase and the depth of the call tree increases. Every non-trivial routine will have multiple pathways through it.

If your average call depth is 10 with an average of 4 pathways through each routine then this represents over 1 million pathways. If the average call depth is 15 then it represents 107 million pathways.

Increasing sophisticated programs have greater call depth than ever and distributed applications increase the call depth even because the call depth of a system is additive. This is what we mean by complexity; it is impossible for us to test all of the different pathways in a black box fashion.

Now in reality every combination of pathways is not possible, but you only have to leave holes in a few routines and you will have hundreds, if not thousands, of pathways where calculations and decisions can go wrong.

In addition, incorrect calculations or decisions higher up in the call tree can lead to difficult to find defects that may blow up much further away from the source of the problem.

What are Defects?

Software defects occur for very simple reasons, an incorrect calculation is performed that causes an output value to be incorrect. Sometimes there is no calculation at all because input data is not validated to be consistent and that data is either stored incorrectly or goes on to cause incorrect calculations to be performed.

We only recognize that we have a defect when we see an output value and recognize that it is incorrect. More likely QA sees it and tells us that we are incorrect.

Basically we follow a pathway that is correct through nodes 1, 2, 3, 4, and 5. At point 6 we make a miscalculation calculation, and then we have the incorrect values at points 7 and 8 and discover the problem at node 9.

So once we have a miscalculation, we will either continue to make incorrect calculations or make incorrect decisions and go down the wrong pathways (where we will then make incorrect calculations).

Not all Defects are Equal

It is clear that the more distance there is between a miscalculation and its discover will make defects harder to detect. The longer the call depth the greater the chance that there can be a large distance between the origin and detection, in other words:

Size Matters

Today we build sophisticated systems of many cooperating applications and the call depth is exponential with the size of the system. This is what we mean by complexity in software.

Reducing Complexity

Complexity is reduced for every function where:

You can identify when inconsistent parameters are passed to a function
All calculations inside of a function are done correctly
All decisions through the code are taken correctly

The best way to solve all 3 issues is through formal planning and development.Two methodologies that focus directly on planning at the personal and team level are the Personal Software Process (PSP) and the Team Software Process (TSP) invented by Watts Humphrey.

Identifying inconsistent parameters is easiest when you use Design By Contract (DbC) , a technique that was pioneered by the Eiffel programming language. It is important to use DbC on all functions that are in the core pathways of an application.

Using Test Driven Development is a sure way to make sure that all calculations inside of a function are done correctly, but only if you write tests for every pathway through a function.

Making sure that all calculations are done correctly inside a function and that correct decisions are make through the code is best done through through code inspections (see Inspections are not Optional and Software Professionals do Inspections).

All techniques that can be used to reduce complexity and prove the correctness of your program are covered in Debuggers are for Losers. N.B. Debuggers as the only formalism will only work well for systems with low call depth and low branching.

Conclusion

Therefore, complexity in software development is about making sure that all the code pathways are accounted for. In increasingly sophisticated software systems the number of code pathways increases exponentially with the call depth. Using formal methods is the only way to account for all the pathways in a sophisticated program; otherwise the number of defects will multiply exponentially and cause your project to fail.

Only projects with low complexity (i.e. small call depth) can afford to be informal and only use debuggers to get control of the system pathways. As a system gets larger only the use of formal mechanisms can reduce complexity and develop sophisticated systems. Those formal mechanisms include:

Personal Software Process and Team Software Process
Design by Contract (via Aspect Oriented Programming)
Test Driven Development
Code and Design Inspections

Rating: 0.0/5 (0 votes cast)

Rating: 0 (from 0 votes)

Are we there yet?

Posted on September 18, 2013 by Dalip Mahal

We associate “are we there yet?” with kids asking incessantly if a long trip is almost over. It is generally funny, however, it is less funny on a project that should be complete.

Projects follow distinct phases:

Basic requirements are collected
Project plan and end date are established
Development starts
Projects track to the project plan

Often, tasks start off well until they all level off at 90-95% complete and get stuck. Management was satisfied with progress until progress stall, they see frantic activity picking up and they wonder “are we there yet?.”

Like a family vacation, this trip is sometimes not even close to being finished. You expect that if 9,000 hours have been spent on a project estimated at 10,000 hours that you would be 90% done. Surprisingly this is not the case for 7 projects out of 10 — have you ever worked on a project where a 90% project plan meant the project was 90% done?

Using a project plan to estimate completion is useful if there is a direct correlation between the project goal and the project plan. You often discover that the goal and the plan differ late in a project. Project plans and results differ because

Requirements and tasks are missing
The project is incorrectly estimated
Work is performed on tasks that do not advance the project

Requirements and Tasks are Missing

Clearly missing requirements mean that more work will be necessary to get to the goal, but the time for this work is rarely added to the project deadline.

A relative of missing requirements is missing tasks, this occurs when the work breakdown structure is incomplete and more subtasks are necessary to complete a task than estimated.

In both cases, if there are 2,000 hours of missing requirements and tasks then a project initially forecasted for 10,000 hours should move the deadline to 11,000 hours. Therefore if 9,000 hours are done then you are only 75% complete.

Unfortunately, weak IT leadership, internal politics, and embarrassment over poor estimates will not move the deadline and teams will have pressure put on them by overbearing senior executives to get to the original deadline even though that is not possible.

The Project is Incorrectly Estimated

There is much literature about how accurate estimates are possible and necessary to successful projects. Weak and uniformed IT leadership will cave in to senior management demands for project deadlines without formal estimates. A typical interaction looks as follows:

CEO: We need feature X, how much will it cost and how long will it take to built?

VP Engineering: Well we need to define feature X properly, see how it will be implemented, determine if we have the necessary skill sets, and see what the impact to our other operations will be. It will take time to do do this work.

CEO: We don’t have time for formal estimates. How hard can it be to add feature X? But next Friday, I will need a ballpark estimate for time and cost.

VP Engineering: I’ll see what I can come up with for next week.

Weak IT executives allow themselves to be bullied all the time by other executives that have no idea what is involved in IT projects. The end result is an underspecified project that will be underestimated in time and cost (see Why Executive Declared Deadlines lead to Disaster)

The more inaccurate the requirements the more extra work there is to do to get to the target. That is why short requirements processes lead to strongly shifting requirements and cancelled projects (see Shift Happens). In fact, the degree of requirements shift is equal to the chance of a project being cancelled.

Work is Performed that Does Not Advance the Project

Even if a project is correctly specified, there are several activities that will be performed that will not advance the project:

Some requirements can not be implemented as specified and time will be spent researching and implementing work arounds
Some requirements will be ambiguously specified and be implemented incorrectly and need to be redone
Some requirements will be inconsistent and require time and analysis to establish consistent requirements
Infrastructure might need to be refactored when you discover that it will not support the code created later

Rework Work executed on these activities will not advance your project and should not be counted in the total of completed hours. So if 2,000 hours have been spent on activities that don’t advance the project then if 9,000 hours have been done on a 10,000 hour project then you have really done 7,000 hours of the 10,000 hour project and you are only 70% done.

Not Changing the Deadline Leads to Bad Practices

Whether a project is off course because of missing activities or non-productive activities, not changing the project end date will lead to schedule pressure as the project advances and people slowly start to realize that you are not going to make it.

When it becomes clear that a project can’t make it’s original deadline many organizations will start common but deadly practices. Excessive schedule pressure often leads to the following bad practices (see Stop It! No… Really stop it.):

Friction within the team
Friction amongst the managers
Inadequate communication with stakeholders
Layoffs of key personnel

Solutions

There are only a few cures for the ‘Are we there yet?‘ problem:

IT management with the intestinal fortitude to hold out for the creation of proper work breakdown structures and formal estimates
Proper requirement processes that yield complete, consistent, and concise requirements
Proper change management processes to alter the project deadline when missing tasks and non-productive activities are encountered

Failure to comply with these 3 principles means that you will continue to be subject to chaotic environments where 7 out of 10 projects fail (see Executives: Understanding your Chances of a Successful Project)

Rating: 0.0/5 (0 votes cast)

Rating: 0 (from 0 votes)

Let the Machines Take Over

Posted on June 6, 2013 by Dalip Mahal

Whether you believe that the machines will take over or not will always be up for debate (until they actually take over :-)).

What is not up for debate is that many processes for software creation have been automated and can help improve productivity and quality in every phase of a software project.

Here are some statistics about how useful automation can be and give you some ammunition when you confront management about the need for these tools.

These results have been generated by Capers Jones who has been collecting data for over 15,000 projects for about 40 years. Full results shown in the paper in the references.

Requirements Analysis

There have been tools available for tracing requirements for quite some time. Generally the only companies forced to provide requirements traceability, but this makes sense to do on all large projects:

Automated requirements tracing can raise productivity by 12.89% and quality by 17.8%

There are tools that can help size programs in terms of function points, which then allows you to get relatively accurate estimates of a project`s cost and time before the project is executed.

Automated sizing in function points can raise productivity by 16.5% and quality by 23.7%

Development

There are several tools that are useful during development, but by far the biggest bang for your buck comes in the form of automated static analysis.

Automated static analysis can raise productivity by 20.9% and quality by 30.9%

There are now static analysis tools available for virtually all current languages that are cost effective and can help to locate those hard to find occasional defects. You can find a list here of automated tools for C/C++, Java, JavaScript, Objective-C, Perl, PHP, and Python.

Then getting cyclomatic complexity counts for your code can be done in an automated fashion. Studies have shown that files with a count of 74+ were 98% likely to have defects.

Automated cyclomatic complexity analysis can raise productivity by 14.5% and quality by 19.5%

Performance is a tricky thing, we will talk about it often but rarely do we actually do something about it. Automated performance analysis allows you to determine if there will be production problems before deploying new code.

Automated performance analysis can raise productivity by 12.5% and quality by 19.5%

There are tools for restructuring code automatically, many of these are built directly into today’s IDEs. There are many stand alone tools for automated restructuring, some are shown here.

Automated restructuring can raise productivity by 8.0% and quality by 11.7%

Automated unit testing is the easiest way to detect defects early. If your developers are using Test Driven Development (TDD) and any continuous integration tool (i.e, Hudson or Jenkins) then you can find out with each build if a defect has crept into the code.

Automated unit testing can raise productivity by 16.5% and quality by 23.8%

Testing

Believe it or not I was working with a team that was not using version control or defect tracking last year. So there are clearly still some organizations not using defect tracking.

Automated defect tracking can raise productivity by 17.5% and quality by 25.9%

Testing the percentage of the application pathways that are actually being tested is critical. Most of your defects will lie in the pathways that you can not test.

Automated test coverage analysis can raise productivity by 15.5% and quality by 21.2%

Test cases can be generated automatically in some situations.

Automated test case generation can raise productivity by 15.0% and quality by 20.0%

Deployment

The next two tools are not about technology that you acquire, rather it is about building automated tools to support critical deployment functions. When the pressure is on it is tough to think about creating applications to manage configuration in an automated way. When management tells you that it would be a luxury to build these tools then show them these results.

Some organizations rely on manually updating configurations, but can be error prone if there are many values to update.

Automated configuration control can raise productivity by 16.4% and quality by 23.5%

Also, when automated configuration tools don`t exist then you often don`t have automated deployment tools.

Automated deployment support can raise productivity by 14.6% and quality by 19.6%

Conclusion

One of the most cost effective ways to improve productivity and quality in your software development is to automate any of the issues above. In most cases you can find a vendor that will provide you support; for deployment you can make the case that building automated tools is the way to go.

References

Egyed, Alexander and Grunabacher, Paul, Supporting Software Understanding with Automated Requirements Traceability
Jones, Capers. SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.¹
Marjchrzak, T.A., Automated Test Case Generation Based on Coverage Analysis
Rushby, John. Automated Test Generation and Verified Software.

¹N.B. All productivity and quality percentages were derived over 15,000+ actual projects

Articles in the “Loser” series

Want to see sacred cows get tipped? Check out:

Moo?

Make no mistake, I am the biggest “Loser” of them all. I believe that I have made every mistake in the book at least once 🙂

Rating: 0.0/5 (0 votes cast)

Rating: 0 (from 0 votes)

To be, or not to be… Formal

Posted on June 5, 2013 by Dalip Mahal

What the heck is a formal process? Is it in the category of you know it when you see it?

A formal process is a methodical way of tackling any repetitive process so that it can be handled in a standardized way. The idea of formality is to reduce variation in output and minimize the chance of forgetting something. Formal processes include the ideas of studying, planning, measurement, and process refinement; they cost more and take longer than just just jumping in and getting things done.

Imagine if they tried to make cars in an informal way… would you drive them?

Informal, or not formal processes, are faster and cheaper because you get rid of the overhead of a formal process. It makes sense to be informal when the variation of the process output is not very sensitive to studying, planning, measurement, or refinement.

We are all familiar with trying to use formality when it makes no sense, you simply end up with process for processes sake and waste time and money.

In software, formality has become synonymous with documentation, and frankly we all hate documentation. We don’t like to produce it and we don’t like to read it. I’ve instructed hundreds of engineers and only a few people bother to read the requirements brick. The way education is going right now I’m not even sure that university graduates can read anyways, but I digress… 🙂

For example, documentation for requirements, design, and testing is the end result of a formal process, it is not the formal process itself. It is smart not to produce more documentation than you need, but that does not mean that the formal processes that creates the documentation are optional.

Formal processes in software development cluster around two very simple ideas:

getting the correct requirements
elimination of defects

The problem is that formal processes only work when you do enough of them to make a difference. That is doing too little of a formal process is just as bad as doing too much. For example, UML class diagrams will not only help you to plan code but also educate new developers on the team.

Producing UML diagrams is a formal practice; however, if you insist on producing UML diagrams for everything then you will end up spending a significant amount of time creating the diagrams to get a diminishing benefit. Not having any UML diagrams will lead to code being developed more than once simply because no one has a good overview of the system.

Selective UML diagrams can really accelerated your development

Visualizing Formality

As you increase any formal practice the effect of that practice will increase, for example the more formal you are with UML diagrams the less defects you are likely to have:

However, as you increase formality, the costs of keeping track of all your artifacts goes up:

When you put the two graphs together you get this effect:

This shows that there is a cost to not having enough formality, i.e. no UML diagrams means that more code will get developed and less reuse means more defects. However, producing too many UML diagrams will also have the cost of producing the UML diagrams and keeping them updated. The reality is that for any formal practice there is a sweet spot where the minimal formality gives the lowest cost.

What that means is that productivity will be maximized in the sweet spot.

Remember that most formal practices are hygiene practices, these are things that no one wants to do but are really necessary to be highly productive and produce high quality code (see here for more on hygiene practices)

Statistics Behind Formal / Informal Practices

Formal practices can be applied to many things, but Capers Jones has measured the following formal practices (when properly executed) can have a positive effect on a project (15,000+ projects):

Formal risk management can raise productivity by 17.8% and quality by 26.4%

Formal measurement programs can raise productivity by 20.0% and quality by 30.0%

Formal test plans can raise productivity by 16.6% and quality by 24.1%

Formal requirements analysis can raise productivity by 16.3% and quality by 23.2%

Formal SQA teams can raise productivity by 15.2% and quality by 20.5%

Formal scope management can raise productivity by 13.5% and quality by 18.5%

Formal project office can raise productivity by 11.8% and quality by 16.8%

A special category of formal practices are inspections:
Formal code inspections can raise productivity by 20.8% and quality by 30.9%

Formal requirements inspections can raise productivity by 18.2% and quality by 27.0%
Formal design inspections can raise productivity by 16.9% and quality by 24.7%

Formal inspection of test materials can raise productivity by 15.1% and quality by 20.2%

More information on inspections can be found here:

Not being formal enough has its costs, and it is rarely neutral. Here are some of the costs of not being formal enough:

Informal progress tracking can lower productivity by 0.5% and quality by 1.0%

Informal requirements gathering can lower productivity by 5.7% and quality by 8.4%

Inadequate risk analysis can lower productivity by 12.5% and quality by 17.5%

Inadequate testing can lower productivity by 13.1% and quality by 18.1%

Inadequate measurement of quality can lower productivity by 13.5% and quality by 18.5%

Inadequate defect tracking can lower productivity by 15.3% and quality by 20.9%

Inadequate inspections can lower productivity by 16.0% and quality by 22.1%

Inadequate progress tracking can lower productivity by 16.0% and quality by 22.5%

N.B. progress tracking is on the list twice. Informal progress tracking is fairly neutral, but inadequate progress tracking is deadly.

Conclusion

In development practices it is not a matter of formal vs informal. It is a matter of seeing what kind of variation that you are exposed to and recognizing that formality can make a difference in that area. Then it is a matter of only adding as much formality as you need to get maximum productivity.

Formal practices must be monitored, you must have some way of testing if the formal practice is having an effect on your development. You want to make sure that you are executing all formal practices properly and not just going through the motions of a formal practice with no effect. You must monitor the effects of formality to understand when you are not doing enough and when you are doing too much!

Articles in the “Loser” series

Want to see sacred cows get tipped? Check out:

Moo?

Make no mistake, I am the biggest “Loser” of them all. I believe that I have made every mistake in the book at least once 🙂

Rating: 0.0/5 (0 votes cast)

Rating: 0 (from 0 votes)

Agility is not Informality

Posted on May 31, 2013 by Dalip Mahal

Agility is about following the Manifesto for Agile Software Development, not the sloppy and crazy implementations of Scrum, XP, or other home-grown schemes out there. Sometimes I feel like people are using Agility as an excuse to be informal.Agility is about discipline and being formal for key things.

Two of the key principles of the Agile Manifesto are:

Working software over comprehensive documentation
Responding to change over following a plan

chaos Somehow some developers have interpreted this as meaning that there are no formal processes to be followed. But just because you are putting the priority on working software does not mean that there can be no documentation; just because you are responsive to change doesn’t mean that there is no plan.

Agile development when properly implemented can accomplish great things. When it is just an excuse to be undisciplined and cover-up cowboy development then a so called ‘Agile’ process will work very poorly and the organization will just spin its wheels.

Agile development in general raises productivity by 18.2% and quality by 27.1%

Extreme programming (XP) specifically raises productivity by 12.8% and quality by 17.8%

Agile software development does have formality and it requires quite a bit of discipline to stick with it. A few formal practices for Scrum include:

Stand-up meetings
Work burn down charts
Sprint demos
Simplicity
Improvement through reflection

Stand-up Meetings

Stand-up meetings are an integral part of the Scrum process and is a key discipline of Scrum, when they are done properly. This meeting is a coordinating mechanism of a self-organizing team where each member states:

What have they finished
What are they working on
What external issues do they need help on

The scrum daily meeting raises productivity by 16.4% and quality by 23.5%

Believe it or not, there are some Scrum teams that sit down for the stand-up meeting. What the heck is this? The purpose of standing up is to be slightly uncomfortable and keep in mind that the meeting is supposed to move quickly. The meeting will be short if you do it standing up.

You state what features that you have finished so that team mates waiting on your deliverables know where you are at. Finished means that you have unit tested your code (i.e. TDD) and that black box testing can start if there is a tester in your Scrum team.

Automated unit testing raises productivity by 16.5% and quality by 23.8%

You state what you are working on so that other team mates know what you are doing and can organize their work accordingly. Don’t bother to state how you solved problems, that was an issue for planning with team mates before you started coding.

You state what you need help on that is external to the Scrum team so that the Scrum master can resolve the issue. This is not a design meeting, if there are issues that you need to resolve with team mates then you can resolve this issue right after the meeting.

Burn Down Charts

Work burn down charts is a formal discipline and essential to success (see Who Needs Formal Measurement?). The work burn down chart is the heart of accountability in Scrum. For the chart hours to get to 0 on a regular basis, it relies on the principles of: 1) commitment, 2) execution, and 3) estimates.

The point of a self-organizing team is that it commits to work each sprint to get the work done. If it turns out that they over committed to a sprint then they must do whatever it takes to get the job done. That may mean extra hours or it may mean being extra creative, but once you commit to work then you have got to get it done. That means execution, it means doing what you say that you will do.

You may have a sprint or two where the work burn down chart does not go to 0. If this is happening on a regular basis then you have either an estimation problem or a team commitment problem. If engineers are regularly underestimating their tasks then it means that you really don’t know how long things take. You should consider some of the techniques in the Personal Software Process (PSP) to improve your ability to estimate.

PSP development raises productivity by 21.3% and quality by 31.3%

Another cause of work burn down charts not going to 0 is that the team is not really self-organizing. The team needs to commit to the work to be done, it will never work if the organization assigns the work to be done in the sprint.

If you lose discipline during the sprint and don’t keep the work burn down charts up to date, then you are not tracking progress correctly. Remember, you can’t manage what you don’t measure.

Inadequate progress tracking reduces productivity by 16.0% and quality by 22.5%

Sprint Demos

The sprint demo is not optional, it is a mandatory part of the process. The sprint demo is to demonstrate near-production code and demonstrate a working system. The team needs to put pressure on itself that the code must be complete by the end of the sprint; this forces a self-organizing team think through the amount of work that they can commit to. No demo = no team commitment.

Remember the whole point of Agile development is to put the emphasis on working software. If you are not producing working software every sprint, then what are you doing? Whatever you are doing it is not Agile development.

The Agile Manifesto puts emphasis on working software. If your sprints are not producing working software then you are not doing agile development.

Simplicity

The best designs are the simplest designs because they are easier to understand and maintain. In a previous article No Experience Required! I discuss how years of experience do not make for better programmers. You might expect that the best developers have more years of experience, but 8 studies over 30 years show that this is not the case (you might even want to get rid of poor performers with many years of experience, but see the article 🙂 )

In fact two persistent facts about the differences between the best and worst developers is that:

program execution speed about 10 to 1
program size 5 to 1

Much of this results from simplicity. The best developers plan their code and write simple and straight forward routines. The are rewarded by having smaller programs that execute quickly. The worst developers do not plan their code and shoot from the hip. They write convoluted code that is much larger and has much worse execution time.

Agility is about simplicity. It is spending 80% of your time thinking and 20% of the time writing simple and elegant code. It is not about spending 100% of your time writing complex and convoluted code that even you can’t understand.

Simplicity does not happen, it is planned; it takes tremendous effort to write simple code. The best developers plan their code (see PSP above) and are continually refactoring their code to make designs simpler to understand and maintain.

Automated restructuring raises productivity by 8.0% and quality by 11.7%

Code refactoring raises productivity by 4.3% and quality by 6.7%

Improvement through Reflection

A key discipline in Agile development is to reflect on what you can do better as individuals and as a team. There is always room for improvement, there is often a better way to do things. Professional developers are keen to keep learning and improving their craft, and the best way to do this is by reflecting on the sub-optimal choices that we have made in the past and trying to do better in the future.

Conclusion

Agile development is about following the Agile Manifesto, it is not about informality. True agility is developed by having the discipline to be formal in many things:

Team synchronization
Learning to estimate
Tracking progress through burn down charts
Committing to create production code
Planning simple code
Improving by not making the same mistakes

Agility is about being disciplined and formal in processes that count

References

Jones, Capers. SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.

N.B. All productivity and quality percentages were derived over 15,000+ actual projects

Articles in the “Loser” series

Want to see sacred cows get tipped? Check out:

Moo?

Make no mistake, I am the biggest “Loser” of them all. I believe that I have made every mistake in the book at least once 🙂

Rating: 0.0/5 (0 votes cast)

Rating: 0 (from 0 votes)

Who needs Formal Measurement?

Posted on May 31, 2013 by Dalip Mahal

We all know the expression “You can’t manage what you can’t measure“, but do we really understand it?

After execution, feedback is an essential part of all processes. Just think about how difficult it would be to drive from home to work wearing a blindfold. Without your sense of sight to give you feedback on the traffic signals and the locations of other cars you would crash your car. Yet we develop software systems without instituting formal measurement programs all the time and wonder why we succeed so rarely? (for success rates see Understanding your chances of having a successful software project)

You can’t manage what you can’t measure

No measurement means no feedback, which means your chances of success are minimized. Success is possible without formal measurement but it is much easier with formal measurement.

Formal measurement raises productivity by 20.0% and quality by 30.0%

A best practice is one that increases your chance of succeeding, it does not guarantee it. It has been established that formal measurement is a best practice, so why do so few people do it?

Measurement has a cost and organizations are petrified of incurring costs without incurring benefits. After all what if you institute a measurement program and things don’t improve? In some sense managers are correct that measurement programs cost money to develop and unless measurement is executed correctly it will not yield any results. But is there a downside to avoiding measurement?

Inadequate progress tracking reduces productivity by 16.0% and quality by 22.5%

Failure to estimate requirements changes reduces productivity by 14.6% and quality by 19.6%

Inadequate measurement of quality reduces productivity by 13.5% and quality by 18.5%

So there are costs to not having measurement. Measurement is not optional, measurement is a hygiene process, that is, essential to any process but especially to software development where the main product is intangible.

A hygiene process is one which can prevent very bad things from happening. Hygiene processes are rarely fun and take time, i.e. taking a shower, brushing your teeth, etc. But history has show that it is much more cost effective to execute a hygiene process than take a chance of something very bad from happening, i.e. disease or your teeth falling out.

There are hygiene practices that we use every day in software development without even thinking about it:

Version control
Defect tracking

Version control is not fun, tracking defects is not fun; but the alternative is terrible. Only the most broken organizations think that they can develop software systems without these tools. These tools are not fun to use and virtually everyone complains about them, but the alternative is complete chaos.

Formal measurement is a best practice and a hygiene practice

The same way that developers understand that version control and defect tracking is necessary, an organization needs to learn that measurement is necessary.
Is Formality Necessary?

The reality is that informal measurement is not comprehensive enough to give consistent results. If measurement is informal then when crunch time comes then people will stop measuring things when you need the data the most.

When you don’t have enough formality then processes take longer and by extension cost more. When you have too much formality then you have process for processes sake and things will also take a long time. Any organization that implements too much formality is wasting their time, but so is any organization that does not implement enough.

When you suggest any formal process people immediately imagine the most extreme form of that process; which would be ridiculous if it is implemented that way. We have all been in organizations that implement processes that make no sense, but without measurement how do you get rid of these processes that make no sense? For every formal process that makes sense, there is a spectrum of implementations. The goal is to find the minimum formality that reduces time and costs. When you find the minimum amount of formal measurement you will accelerate your development by giving yourself the feedback that you need to drive your development.

What to Measure

It seems obvious, but incorrect measurement and/or poor execution leads to useless results. For example, trying to measure productivity by measuring the hours that the developers sit at their machines is as useful as measuring productivity by the number of cups of coffee that the developers drink.Another useless measure is lines of code (LOC), in fact, Capers Jones believes that anyone using LOC as a measurement should be tried for professional malpractice!Measuring the the three things mentioned above will improve productivity and quality because there will not be a negative effect on your organization:

Measuring progress tracking (productivity +16.0%, quality +22.5%)
Estimating requirements changes (productivity +14.6%, quality +19.6%)
Measurement of quality (productivity +13.5%, quality +18.5%)

Other things to measure are:

Activity based productivity measures
(productivity +18.0%, quality by 6.7%)
Automated sizing tools (function points)
(productivity +16.5%, quality by 23.7%)
Measuring requirement changes (productivity +15.7%, quality by 21.9%)

So to answer the question: who needs formal measurement?

We all need formal measurement

References

Jones, Capers. SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.

N.B. All productivity and quality percentages were derived over 15,000+ actual projects

Articles in the “Loser” series

Want to see sacred cows get tipped? Check out:

Make no mistake, I am the biggest “Loser” of them all. I believe that I have made every mistake in the book at least once 🙂

Rating: 0.0/5 (0 votes cast)

Rating: 0 (from 0 votes)

Defect Life-cycle

Enhancement Life-cycle

Put Responsibility Where it Belongs

Incomplete and Inconsistent Requirements

Only 54% of Issues are Resolved by Engineers

Getting Control of the Defect Process

Reducing Fire-Fighting

Related Articles

Bibliography

Complexity Defined

What are Defects?

Not all Defects are Equal

Reducing Complexity

Conclusion

Requirements and Tasks are Missing

The Project is Incorrectly Estimated

Work is Performed that Does Not Advance the Project

Not Changing the Deadline Leads to Bad Practices

Solutions

Requirements Analysis

Development

Testing

Deployment

Conclusion

References

Visualizing Formality

Statistics Behind Formal / Informal Practices

Conclusion

Stand-up Meetings

Burn Down Charts

Sprint Demos

Simplicity

Improvement through Reflection

Conclusion

References