User stories or use cases?

Alistair Cockburn says “A user story is to a use case as a gazelle is to a gazebo“.

If you are extremely familiar with both techniques then this makes sense.  If you are not familiar with both then suffice it to say that you can put a gazelle in a gazebo but not vice versa.

A user story is of the form As a <type of user>, I want <some goal> so that <some reason> (see here).  

A use case for withdraw money has many more components to it, and a skeleton of such a use case can be found here.  I develop this skeleton use case in a four part tutorial:

  • Part 1: Use case as a dialog
  • Part 2: What is an actor?
  • Part 3: Adding interface details
  • Part 4: Adding application context

The above user story is only a part of the entire use case.  For an ATM, a user story might be As a user, I want to withdraw money so that I can have physical cash.  This user story is only the summary of the withdraw money use case without the details.

User stories come from the Extreme Programming methodology where the assumption was that there will be a high degree of interaction between the developers and the end customer and that QA will largely be done through test driven development.  It is important to realize that Extreme Programming does not scale.

Once your development team gets large, i.e. you have 3+ agile teams and your code base gets large enough to warrant a formal testing environment, then you will outgrow user stories as your only method of capturing requirements.  You can still use user stories for new modules
developed with an end customer to take advantage of the light weight and rapid nature of user stories, but at some point those user stories should transition to use cases.

Unfortunately, there are quite a few ways to build use cases and not all of them are effective.  Alistair Cockburn’s method (found here) is an excellent way to capture use cases for more sophisticated systems.  There is no doubt that a full use cases is heavier weight than a user story, but consider this:

  • Use cases can more easily be turned into test cases by QA
  • With use cases you more easily prove that you have all the requirements
  • Use cases annotated with screens and reports can be used to collaborate with remote off site customers
  • Well written use cases can easily be broken up into a sprint back log
  • Use cases will work if you are not using Agile development methods

So don’t forgo the speed and dynamic nature of user stories, just recognize that there are limits to user stories and that you will need to transition to use cases when your project team or application grows.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

What do you do if the customer is not right?

We often hear the customer is always right!, but is this really true? Haven’t we all been in situations where the customer is asking for something unreasonable or is simply downright wrong?  Aren’t there times when the customer is dead wrong?

This general strategy reflects the fact that 5:1 it is more expensive to acquire a new customer than to retain an existing one.  So even when the customer is wrong, accommodating their idiosyncrasies is worth losing a battle to win the profit war because acquiring customers is so expensive.

Today we take a more nuanced position.

If the customer is unreasonable and unprofitable then it makes no sense to adopt the motto that the customer is always right!.  Unprofitable accounts are retained if they are strategic and either 1) become profitable, or 2) draw enough profitable accounts into the company to make up for the loss. This strategy is employed by start-ups to get their first customer.

Recognizing bad customers is usually not difficult. Transactional customers are often bad customers; especially those that want the lowest price and act as though every product is a commodity; they try to play vendors off against each other despite quality requirements.

It is often useful to allow transactional customers interested in the lowest price to purchase from competitors on price and let the lack of a quality solution come back to haunt them. Reducing quality to meet customer price objectives will leave you with customer complaints when product and service quality is substandard.

When sales executives chase all opportunities hoping for a sale is when transactional buyers are courted and you get pulled into pricing concessions from demanding customers. The problem is demanding transactional buyers won’t just ask for the best price, they will also ask for product changes.

There is no doubt that customer requirements need to be a driver for product management it is an early indication of changing markets. But, accepting all customization requests is impossible and would cripple your product and brand.

Constant unreasonable requests leave internal resources believing that sales people have extremely low IQs and morals.

Good sales people understand these principles and don’t chase bad customers.  But, there are not enough good sales people to go around, so virtually every company has a less-than-excellent sales person making trouble for product management and engineering.

To make matters worse, sales people are very good at making a case that all customers are strategic and argue that a short term loss will eventually turn into a long term gain. This behavior is normal and expected because sales is driven by commissions, which are often revenue based. Hopefully, your sales process is robust enough to catch these attempts which would saddle you with bad customers.

If you find yourself in a position where you have acquired one or more bad customers (you know who they are..) then your best course of action is to find some way to send them to your competitors. This will increase your profitability and reduce the stress of unreasonable requests flooding into product management and engineering.

The customer is not always right, but with due diligence and account reviews you can determine the customers that should be retained and those that should be let go.

Don’t be afraid to let unprofitable and non-strategic customers go. You will feel less stressed and be better off in the long run.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Ready, Fire, Aim: Why correct requirements are rarely gathered

missed_target-300x204Connecting with your customers and delivering value depends on understanding your customer’s requirements and selling the correct product or solution that solves your customer’s problems.

Commonly, the requirements gathering process is done hastily or not at all in the rush to get the sale. After all, the faster you can make the sales process go, the faster the money is in the bank — correct?

tailoringThe more that product customization or creation is required, the more it is important understand the customer’s actual problem.  The more time required to build and implement a solution means will not only lead to a failed sale but also to a more disappointed customer and a loss of future revenue.

If you build software, which has long lead times, it is even more important to make sure that what you are building will satisfy the customer’s requirements as changing the final software solution will be nearly impossible and very costly.

Why do we continually misunderstand and sell the wrong solutions and building the wrong software?

Human Nature

Time pressures to make a sale put us under pressure and this stress leads to making quick decisions about whether a product or solution can be sold to a customer.  We listen to the customer but interpret everything he says according to the products and solutions that we have.

Often the customer will use words that seem to match exactly the products we have.  But then after the sale once we have to implement we often find that either we deliver a poor solution to the customer or a solution is infeasible and we must refund the customers money.

However, behind every word that the customer is using there is an implied usage, and understanding that implied usage is where we fail to gather requirements.

For example, suppose the customer says I need a car. Suppose that you sell used cars:

  • the customer asked for a car
  • you sell cars

Ergo problem solved!

  • What if the customer needs an SUV but you don’t have any?
  • What if the customer really needs a truck?
  • What if the customer needs a car with many modifications?
  • What if the car the customer needs has never been built?

We hear the word car and we think that we know what the customer means.  The order-taker sales person will spring into action and sell what he thinks the customer needs. Behind the word car is an implied usage and unless you can ferret out the meaning that the customer has in mind, you are unlikely to sell the correct solution.

If you don’t understand how the customer will to use your solution then you don’t understand the problem.

Comedy of Errors

For products that require customization, the sale will get transferred to professional services that will dig deeper into the customer’s requirements.  At this point you discover that the needs of the customer cannot be met.  This leads to sales people putting pressure on professional services and product management to ‘find a solution’, after all, losing the sale is not an option.

Sometimes heroic actions by the product management, professional services, and software development teams lead to a successful implementation, but usually not until there has been severe pain at the customer and midnight oil burned in your company.  You can eventually be successful but that customer will never buy from you again.

Consequences

As WIlliam Ralph Inge said, There are no rewards or punishments — only consequences.

The consequence of selling the wrong solution to a customer is:

  • Whether you lose the sale or not, the customer loses faith in you
  • The sales person is perceived as incompetent in the rest of the organization especially by professional services and software devleopment
  • Your cost of sales and implementation is much higher than expected
  • Your reputation is damaged

Conclusion

Selling the correct solution to the customer requires that you understand the customer’s problem before you sell the solution.

When customization is required, good sales people engage resources that can capture the customer’s requirements accurately and assess that you can deliver a solution to the customer.

Slowing down to understand the customer requirements and how you will solve his problems is the key.  By understanding the customer’s requirements and producing the correct solutions you become a trusted adviser to the customer.

 

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

When BA means B∪<<$#!t Artist

BA typically means Business Analyst, but what makes for a good BA?  When do you have a good BA and when don’t you? Many projects fail at the beginning due to incomplete, inconsistent, and overly verbose analysis produced by BAs.

Are your BAs any good?  The success of many projects depend on the ability of the BA do correctly identify the problem you are trying to solve.  If your BA is not competent then you are doomed before you start.

RubiksCubeBusiness analysis consists of all facets of solving business problems. Business analysis is a role executed by project, product, and engineering managers.  However, there are people that do business analysis as their main role and we simply label them as business analysts or product managers. We shall stick to the term business analyst for simplicity.

Business analysts perform a range of tasks including:

  • Gathering requirements
  • Writing business cases and project charters
  • Performing gap analysis between products and corporate processes

Ever suspect that the people responsible for performing business analysis for you are not up for the challenge?  Here are three simple questions; good business analysts will get all three correct and not take longer than 20 seconds.

  Context Question
#1  MullerLyerLines Which of these two lines is longer?
#2  paris_spring_puzzle What does this sign say?
#3 I have two products whose prices add up to $1.10 and one product is $1 more than the other. How much is the more expensive product?

The answers are:

  1. The lines are the same length
    1. Take a ruler if you are not convinced
  2. Paris in the the spring
    1. Notice that the occurs twice
  3. The more expensive product is $1.05
    1. If the more expensive one had been $1 then the cheaper one would be $0.10 but then the expensive one would only be $0.90 more expensive than the cheaper one.

Jumping+to+conclusionsThese three questions illustrate that the mind can jump to conclusions that are often wrong.  The mind can be tricked because it assesses things quickly and only some people have the instinct to double check their results.

Most BAs are well educated; however, the interesting thing is that studies show that educated people are less able to see their biases and they jump to conclusions more readily than other people.1

A good business analyst realizes that during analysis there will be situations where the mind will jump to conclusions.  Many business analysts are asked to resolve conflicting requirements, recognize missing requirements, and deal with biases coming from many different sources.  Unless you have the reflex of checking facts for consistency and eliminating bias then you won’t make a good business analyst.

So what is the quality of your BAs?

Other articles

Bibliography

West, Richard F. and Meserve, Russel J. Cognitive Sophistication does not Attenuate the Bias Blind Spot. Journal of Personality and Social Psychology.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

No Business Case = Failed Project

A business case comes between a bright idea for a software project and the creation o that project. Project Timeline, Business Case

  • To – idea to have a project is born
  • Tcheck – formal or informal business case
  • Tstart – project is initiated
  • Tend – project finishes successfully or is abandoned

Not all ideas for software projects make sense.  In the yellow zone above, between idea and project being initiated, some due diligence on the project idea should occur.  This is where you do the business case, even if only informally on the back of a napkin.

The business case is where you pause and and estimate  whether the project is worth it, i.e. will this project leave you better off than if you did not do it.

For those who want precise definitions the project should be NPV +ve.  In layman’s terms, the project should leave the organization better off on it’s bottom line or at least improve skill levels so that other projects are better off.

Projects that do not improve skills or the bottom line are a failure.

Out of 10 software projects (see Understanding your chances):

  • 3 are successful
  • 4 are challenged, i.e. over cost, over budget, or deliver much less functionality
  • 3 will fail, i.e. abandoned

This means that the base rate of success for any software project is only 3 out of 10.

Yet executives routinely execute projects assuming that they can not fail even though the project team knows that the project will be a failure from day 1.

Business cases give executives a chance to stop dubious projects before they start. (see Stupid is as Stupid Does)

Understanding how formal the business case needs to be comes down to uncertainty. There are three key uncertainties with every project:

  • Requirements uncertainty
  • Technical uncertainty
  • Skills uncertainty

When there is a moderate amount of uncertainty in any of these three areas then a formal business case with cash flows and risks needs to be prepared.

Requirements Uncertainty

Requirements uncertainty is what leads to scope shift (scope creep).  The probability of a project failing is proportional to the number of unknown requirements when the project starts (see Shift Happens).

Requirements uncertainty is only low for two particular projects: 1) re-engineering a project where the requirements do not change, and 2) the next minor version of a software project.

For all other software projects the requirements uncertainty is moderate and a formal business case should be prepared.

Projects new to you have high requirements uncertainty.

Technical Uncertainty

Technical uncertainty exists when it is not clear that all requirements can be implemented using the selected technologies at the level of performance required for the project.

Technical uncertainty is only low when you have a strong understanding of the requirements and the implementation technology. When there is only a moderate understanding of the requirements or the implementation technology then you will encounter the following problems:

  • Requirements that get clarified late in the project that the implementation technology will not support
  • Requirements that can not be implemented once you improve your understanding of the implementation technology

Therefore technical uncertainty is high when you are doing a project for the first time and requirement uncertainty is high.  Technical uncertainty is high when you are using new technologies, i.e. shifting from Java to .NET or changing GUI technology.

Projects with new technologies have moderate to high uncertainty.

Skills Uncertainty

Skills uncertainty comes from using resources that are unfamiliar with the requirements or the implementation technology.  Skills uncertainty is a knowledge problem.

Skills uncertainty is only low when the resources understand the current requirements and implementation technology.

Resources unfamiliar with the requirements will often implement requirements in a suboptimal way when requirements are not well written.  This will involve rework; the worse the requirements are understood the more rework will be necessary.

Resources unfamiliar with the implementation technology will make mistakes choosing implementation methods due to lack of familiarity with the philosophies of the implementation libraries.  Often after a project is complete, resources will understood that different implementation tactics should have been used.

Formal or Informal Business Cases?

An informal business case is possible only if the requirements, technical, and skills uncertainty is low.  This only happens in a few situations:

  • Replacing a system where the requirements will be the same and the implementation technology is well understood by the team
  • The next minor version of a software system

Every other project requires a formal business case that will quantify what kind of uncertainty and what degree of uncertainty exists in the project.  At a minimum project managers facing moderate to high uncertainty should be motivated to push for a business case (see Stupid is as Stupid Does). Here is a list of projects that tend to be accepted without any kind of real business case that quantifies the uncertainties:

  • Change of implementation technology
    • Moving to object-oriented technology if you don’t use it
    • Moving from .NET to Java or vice versa
  • Software projects by non-software companies
  • Using generalists to implement technical solutions
  • Replacing systems with resources unfamiliar with the requirements
    • Often happens with outsourcing

Projects with moderate to high risks and no business case are doomed to fail.

Related articles

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Who should set defect priority?

ControlFreakSurprisingly, defect priority should not be set by QA.  QA are generally the owners of the defect tracking system and control it, but this is one attribute that they should not control.  The defect tracker is a shared resource between QA, engineering, engineering management, and product mangement and is a coordinating mechanism for all these parties.

Commonly people mix up priority and severity, for example, there may be a severe defect that causes the software either not to install or to cease functioning.

Car won't startIt is common for new releases to have various installation problems when it initially gets to QA. This blocks QA so they mark the defects with a high severity and high priority.  This issue has a high severity and needs to be addressed right away, but remember bug tracking systems are append only — once this defect gets into the system, it will never get out.  This kind of issue should be escalated to the engineers and engineering management because it makes little sense to clog the defect tracking system with it.

Now there may be intermittent issues that cause the software to fail and you may assume that this defect would be high priority, but if this defect occurs very rarely and would cost too much to fix then this defect may be a low priority.  Once again the priority of an intermittent severe issue can not be determined by QA.

Similarly, there may be many cosmetic or minor defects where fixing them might make a huge difference in the user experience and reduce support calls.  Even though these defects are minor, they may be easy to fix and save you serious money.  Once again, this can not be decided by QA.

Therefore, QA should reserve the right to set the initial severity, and may have an internal field for QA priority, but the priority of a defect should be determined by a product manager (PM) who has a more complete understanding of the overall context of the product.  The priority of a defect is a business issue, not an engineering or QA issue.

Ideally, the product manger will set the priority of a defect during a defect triage session where representatives from engineering, engineering management, and QA are present. As you go through each new defect each person can present their logic for what the priority should be.  The PM should then set the priority defect and assign the release it should be fixed in (i.e. this release, minor release, major release, won’t fix).  Ideally there are extra fields in the defect record for both QA and development to put their priority beliefs.

It is important that the status of a defect not include any of the following or your defect tracker will go to hell (see Bug Tracker Hell and How to Get Out):

  • Priority
  • Severity
  • Fix version

Big documentIt is very hard to generate meaningful reports when these attributes creep into the defect status.  You know that this has happened if you have a multi-page training manual for entering defects into the system.  Some of the statuses that make sense initially but turn the defect system into a nightmare are:

  • FAD (functions as designed)
  • WontFix
  • NextVersion
  • CantReproduce

Of course, if you don’t have regular bug triage sessions with all the parties mentioned above then you are probably sand-bagged in fire-fighting.


Other articles:

 

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Enhancements don’t belong in the bug tracker

Not my faultAs development progresses we inevitably run into functionality gaps that are either deemed as enhancements.

These issues often get captured by QA in the bug tracker and assigned to a developer.

Enhancements should not be managed from the bug tracker

The life cycle of a defect and the life-cycle of a enhancement are two entirely different things.  A defect is a difference between a stated requirement and the code. If there is no documentation there is no code defect (see It’s not a bug, it’s…) — in fact, most enhancements will eventually be coded by some developer; they just should not be managed from the bug tracker.

Defect Life-cycleDefectLifecycle

The defect life-cycle is well known:

  • Defect is identified as a departure from the requirements
  • Defect is assigned to a developer
  • Defect is corrected
  • Correct is verified
    • If not corrected re-open and re-assign to developer
  • The defect is closed

This is the incorrect way to manage enhancements.  When a functionality gap is determined by QA and it is not covered by the requirements then we have an issue.  It is rarely the case that the issue can be resolved by the developer.

Enhancement Life-cycle

If enhancements are assigned to a developer then they are likely to try to resolve the issue. The problem is that “enhancements” determined at the QA level may be phantom problems caused by either:

  1. Insufficient requirements
  2. Correct requirements but incorrect test plans

Enhancements may or may not become code changes.  Even when enhancements turn into code change requests they will generally not be implemented as the developer or QA think they should be implemented.

Enhancements are really requirement defects. Enhancements should be logged as such in the bug tracker and assigned to the person in charge of requirements (business analyst or product manager).  Those individuals should be responsible to track down how these issues should be handled.

If the requirements are correct and the test plans are defective then it should be logged as a test defect.  This is tricky because QA often controls the bug tracker and will not log errors that they have made.

At a minimum, the implementation of requirements and test defects can do several positive things for you:

  1. It removes the responsibility to find a solution from development.
  2. It makes it clear how many defects are in the requirements or test plans.
  3. It reduces stress; no developer wants to be blamed for an issue that is not his.
  4. Many enhancements call for updated project plans and pushing back the deadline.

Put Responsibility Where it Belongs

The creation of requirements and test defects in the bug tracker goes a long way to cleaning up the bug tracker.  In fact, requirements and test defects represent about 25% of defects in most systems (see Bug Tracker Hell and How to Get Out!).  The percentages break down as follows:

  • Requirements defects: 9.58%
  • Testing defects: 15.42%

The creation of requirement and test defects in the bug-tracker alleviate pressure on the engineering department and redirect it  to either the product manager or QA. Eventually enough data will accumulate in the bug-tracker to get management’s attention.

At a minimum, these categories should help reduce the amount of fire-fighting in late projects (Root cause of ‘Fire-Fighting’ in Software Projects)

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

It’s not a bug, it’s…

When does a bug become a bug?
Who decides that it is a bug?

Caution, BugHow many legs does a lamb have if I say the tail is a leg?  The answer is 4, just because I say the tail is a leg does not make it a leg!

Bugs should be obvious, but we say It’s not a bug, it’s a feature because often it isn’t obvious.  Watson Humphrey felt that we should use the term defect and not bug because most people don’t take bugs seriously, so let’s use the term defect instead.

So when does a defect become a defect?

  • When quality assurance tells you that you have a defect?
  • When product management says that it is a defect?
  • When the customer says that it is a defect?

The answer is: none of the above.

Now it might turn out that there is a problem and that code needs to change, but a defect only exists if:

code behaves differently than the requirements specification

This is important because most systems are under specified (if they are specified at all) and so when code misbehaves it is only a defect if the code behavior differs from the specification.  We call defects undocumented features because we know that the problem is that the requirements were never written.

Incomplete and Inconsistent Requirements

Many organizations do not create sufficiently complete requirements before starting development, either because they don’t know how to capture requirements properly or because they don’t have resources capable of capturing complete requirements. Incomplete (and inconsistent) requirements and unrealistic deadlines often force developers into making decisions about how to implement features.  The end result is that developers are regularly told that they have defects in their code.

While this process is common, it is destructive.  When requirements are under specified and inconsistent developers end up needing to perform serious rework. The rework will can require dramatic changes that will impact the architecture of the code.

The time required to find a work around (if it is possible) is rarely included in the project plan. Complicating matters is that the organizations that are reluctant to spend time creating requirements also tend to underestimate their projects.  This puts tremendous pressure on the engineering department to deliver; this promotes the 5 worst practices in software development (see Stop It! No… really stop it.)

Only 54% of Issues are Resolved by Engineers

The attitude that all defects must be resolved by the engineering department is severely misguided.  Analysis by Capers Jones of over 18,000+ projects shows that only about 54% of all defects can be resolved by the engineers! (only the 3 highlighted rows below)

Defect Role Category Frequency Role
Requirements defect 9.58% BA/Product Management
Architecture or design defect 14.58% Architect
Code defect 16.67% Developer
Testing defect 15.42% Quality Assurance
Documentation defect 6.25% Technical Writer
Database defect 22.92% Data base administrator
Website defect 14.58% Operations/Webmaster

This means that precious time will be wasted assigning issues to developers that they can not resolve.  The time necessary to redirect the issue to the correct person is a major contributing factor to fire-fighting

Getting Control of the  Defect Process

For most organizations fixing the defect process involves understanding and categorizing defects correctly.  Organizations that are not tracking the different sources of defects probably have a bug tracker that has gone to hell.  Here is how you can fix that problem, see Bug Tracker Hell and How To Get Out!

At a minimum you need to implement the requirements defect, once you identify issues that are caused by poor requirements it will shine the white hot light of shame onto the resources that are capturing your requirements.  Once you realize how many requirements defects exist in your system you can begin to inform senior management about the requirements problem.

Reducing Fire-Fighting

Fire fightingThe best way to reduce fire fighting is to start writing better requirements (or writing requirements 🙂 ).  To do so you need to figure out which of the following are broken:

  1. Not enough time is allocated to the requirements phase
  2. Unskilled people are capturing your requirements

In all likelihood both of these issues need to be fixed in your organization.  When requirements are incomplete and inconsistent you will have endless fire-fighting meetings involving everyone (see Root cause of ‘Fire-Fighting’ in Software Projects)

Stand your ground if someone tells you that you have coded a defect when there is no documentation for the requirement.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Stupid is as stupid does

StupidIsAsStupidDoesSenior management does not set out to have a failed project, however, when the failure rate is 7 out of 10 projects you wonder what the problem is.

As Ma Gump stated, “stupid is as stupid does“, that is, smart people sometimes do stupid things.  The collective IQ of a project is reduced by the number of managers involved in the project.

Now from a human perspective, something very interesting is going on here.  For example, suppose you were give the following choices:

  1. An 80% chance of making $100
  2. A guaranteed $40

If people behaved according to expected value then everyone would choose the first choice.  However, because humans are risk averse, almost everyone will choose the second alternative with guaranteed money.

Now out of 10 software projects:

  • 3 will succeedExecutives, Understanding your chances, PedestriansCrossingStreet
  • 4 will be challenged
  • 3 will outright fail

To put that in perspective, if you were watching people cross the street at an intersection:

  • 3 cross the street successfully
  • 4 get maimed
  • 3 get killed

How interested would you be in crossing that street?

You can Google “software project failure rates” to see that this has been demonstrated by multiple reliable institutions over every industry.  Challenged projects are generally projects which go over budget and under deliver a software solution that can be declared a moral victory by management.

So if human beings are risk averse and the odds of project success are so low then:

Why does senior management ignore risks on software projects?

Damn the torpedoes, full speed aheadThe only possible conclusion is that senior management can’t conceive of a their projects failing. They must believe that every software project that they initiate will be successful, that other people fail but that they are in the 3 out of 10 that succeed.

This inability to understand the base rate of failure in software development is systemic. There are so many software projects that are started by senior management where the technical team knows that the chance of success is 0% from the start.

Senior management is human and is risk averse, you just need to find a way to remind them of this. One way to get senior management to think twice about projects is to make sure that there is a meeting before launching the project where management is asked the following question:

Assume that this project will fail, why would it have failed? What will the consequences be?

This exercise (if done seriously) may have the effect of causing senior management to realize that the project can indeed fail. With luck, the normal risk aversion that every human being is endowed with will kick in and the project may get re-evaluated.

Related Articles

Bibliography

Kahneman, Daniel. Thinking, Fast and Slow.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Inspections are not Optional

Every developer is aware that code inspections are possible, some might have experienced the usefulness of code inspections, however, the fact is that inspections are not optional.

Without inspections your defect removal rate will stall out at 85% of defects removed; with inspections defect removal rates of 97% have been achieved.

Code inspections are only the most talked about type of inspection; the reality is that all artifacts from all phases of development should be inspected prior to being used.  Inspections are necessary because software is intangible and it is not once everything is coded that you want to notice problems.

In the physical world it is easier to spot problems because they can be tangible.  For example, if you have specified marble tiles for your bathroom and you see the contractor bring in a pile of ceramic tiles then you know something is wrong.  You don’t need the contractor to install the ceramic tiles to realize that there is a problem.

In software, we tend to code up an entire set of functionality, demonstrate it, and then find out that we have built the wrong thing!  If you are working in a domain with many requirements then this is inevitable, however, many times we can find problems through inspection before we create the wrong solutions

Let’s look at some physical examples and then discuss their software equivalents.

Requirement Defects

The requirements in software design are equivalent to the blueprints that are given to a contractor.  The requirements specify the system to be built, however, if those requirements are not inspected then you can end up with the following:

Balcony no Door Missing Landing Chimney Covering Window
Stairs Displaced Stairs to Ceiling Door no Balcony/th>

All of the above pictures represent physical engineering failures.  Every one of these disasters could have been identified in the blueprints if a simple inspection had been done. Clearly it must have become clear to the developers that the building features specified by the requirements were incompatible, but they completed the solution anyways.

Balcony no Door

This design flaws can be caused by changing requirements; here there is a balcony feature that has no access to it.  In Balcony no Door it is possible that someone noticed that there was sufficient room for a balcony on the lower floor and put it into one set of plans. The problem was that the developers that install the sliding doors did not have their plans updated.

Here the changed requirement did not lead to an inspection to see if there was an inconsistency introduced by the change.

Door no Balcony

In Door no Balcony something similar probably happened, however, notice that the two issues are not symmetric.  In Balcony no Door represents a feature that is inaccessible because no access was created, i.e. the missing sliding door.  In Door no Balcony we have a feature that is accessible but dangerous if used.

In this case a requirements inspection should have turned up the half implemented feature and either: 1) the door should have been removed from the requirements, or 2) a balcony should have been added.

Missing Landing

The Missing Landing occurs because the requirements show a need for stairs, but does not occur to the architect that there is a missing landing.  Looking at a set of blueprints gives you a two dimensional view of the plan and clearly they drew in the stairs.  To make the stairs usable requires a landing so that changing direction is simple.  This represents a missing requirement that causes another feature to be only partially usable.

This problem should have been caught by the architect when the blueprint was drawn up. However, barring the architect locating the issue a simple checklist and inspection of the plans would have turned up the problem.

Stairs to Ceiling

The staircase goes up to a ceiling and therefore is a useless feature.  Not only is the feature incomplete because it does not give access to the next level but also they developers wasted time and effort in building the staircase.

If this problem had been caught in the requirements stage as an inconsistency then either the staircase could have been removed or an access created to the next floor.  The net effect is that construction starts and the developers find the inconsistency when it is too late.

At a minimum the developers should have noticed that the stairway did not serve any purpose and not build the staircase which was a waste of time and materials.

Stairs Displaced

Here we have a clear case of changed requirements.  The stairs were supposed to be centered under the door, in all likelihood plans changed the location of the door and did not move the dependent feature.

When the blueprint was updated to move the door the designer should have looked to see if there was any other dependent feature that would be impacted by the change.

Architectural Defects

Architectural defects come from not understanding the requirements or the environment that you are working with.  In software, you need to understand the non-functional requirements behind the functional requirements — the ilities of the project (availability, scalability, reliability, customizability, stability, etc).

Architectural features are structural and connective.  In a building the internal structure must be strong enough to support the building, i.e. foundation and load bearing walls.

insufficient foundation Insufficient structure Connectivity problem

Insufficient Foundation

Here the building was built correctly, however, the architect did not check the environment to see if the foundation would be sufficient for the building.  The building is identical to the building behind it, so odds are they just duplicated the plan without checking the ground structure.

The equivalent in software is to design for an environment that can not support the functionality that you have designed.  Even if that functionality is perfect, if the environment doesn’t support it you will not succeed.

Insufficient Structure

Here the environment is sufficient to hold up the building, however, the architect did not design enough structural strength in the building.

The equivalent in software design is to choose architectural components that will not handle the load demanded by the system.  For example, distributed object technologies such as CORBA, EJB, and DCOM provided a way to make objects available remotely, however, the resulting architectures did not scale well under load.

Connectivity Problem

Here a calculation error was made when the two sides of the bridge were started.  When development got to the center they discovered that one side was off and you have an ugly problem joining the two sides.

The equivalent for this problem is when technologies don’t quite line up and require awkward and ugly techniques to join different philosophical approaches.

In software, a classic problem is mapping object-oriented structures into relational databases.  The philosophical mismatch accounts for large amounts of code to translate from one scheme into the other.

Coding Defects

Coding defects are better understood (or at least yelled about :-), so I won’t spend too much time on them.  Classic coding defects include:

  • Uninitialized data
  • Uncaught exceptions
  • Memory leaks
  • Buffer overruns
  • Not freeing up resources
  • Concurrency violations
  • Insufficient pathways, i.e. 5 conditions but only 4 coded pathway

Many of these problems can be caught with code inspections.

Testing Defects

Testing defects occur when the test plan flags a defect that is a phantom problem or a false positive. This often occurs when requirements are poorly documented and/or poorly understood and QA perceives a defect when there is none.

The reverse also happens where requirements are not documented and QA does not perceive a defect, i.e. false negative.

Both false positives and negatives can be caught by inspecting the requirements and comparing them with the test cases.

False positives slow down development. False negatives can slip through to your customers…

Root Cause of Firefighting

When inspections are not done in all phases of software development there will be fire-fighting in the project in the zone of chaos.  Most software organizations only record and test for defects during the Testing phase.  Unfortunately, at this point you will detect defects in all previous phases at this point.

QA has a tendency to assume that all defects are coding defects — however, the analysis of 18,000+ projects does not confirm this.  In The Economics of Software Quality, Capers Jones and Olivier Bonsignour show that defects fall into different categories. Below we give the category, the frequency of the defect, and the business role that will address the defect.

Note, only the bolded rows below are assigned to developers.
Defect Role Category Frequency Role
Requirements defect 9.58% BA/Product Management
Architecture or design defect 14.58% Architect
Code defect 16.67% Developer
Testing defect 15.42% Quality Assurance
Documentation defect 6.25% Technical Writer
Database defect 22.92% DBA
Website defect 14.58% Operations/Webmaster

Notice that fully 25% of the defects (requirements, architecture) occur before coding even starts.  These defects are just like the physical defects shown above and only manifest themselves once the code needs to be written.

It is much less expensive to fix requirements and architecture problems before coding.

Also, only about 54% of defects are actually resolvable by developers, so by assigning all defects to the developers you will waste time 46% of the time when you discover that the developer can not resolve the issue.

Fire-fighting is basically when there need to be dozens of meetings that pull together large numbers of people on the team to sort out inconsistencies.  These inconsistencies will lie dormant because there are no inspections.  Of course, when all the issues come out, there are so many issues from all the phases of development that it is difficult to sort out the problem!

Learn how to augment your Bug Tracker to help you to understand where your defects are coming from in Bug Tracker Hell and How to Get Out!

Solutions

There are two basic solutions to reducing defects:

  1. Inspect all artifacts
  2. Shorten your development cycle

The second solution is the one adopted by organizations that are pursuing Agile software development.  However, shorter development cycles will reduce the amount of fire-fighting but they will only improve code quality to a point.

In The Economics of Software Quality the statistics show that defect removal is not effective in most organizations.  In fact, on large projects the test coverage will drop below 80% and the defect removal efficiency is rarely above 85%.   So even if you are using Agile development you will still not achieving a high level of defect removal and will be limited in the software quality that you can deliver.

Agile development can reduce fire-fighting but does not address defect removal

Inspect All Artifacts

Organizations that have formal inspections of all artifacts have achieved defect removal efficiencies of 97%!  If you are intent on delivering high quality software then inspections are not optional.

Of course, inspections are only possible for phases in which you have actual artifacts. Here are the artifacts that may be associated with each phase of development:

Phase Artifact
Requirements use case, user story, UML Diagrams (Activity, Use Case)
Architecture or design UML diagrams (Class, Interaction, Deployment)
Coding UML diagrams (Class, Interaction, State, Source Code)
Testing Test plans and cases
Documentation Documentation
Database Entity-Relationship diagrams, Stored Procedures

Effective Inspections

Inspections are only effective when the review process involves people who know what they are looking for and are accountable for the result of the inspection.  People must be trained to understand what they are looking for and effective check lists need to be developed for each artifact type that you review, e.g. use case inspections will be different than source code reviews.

Inspections must have teeth otherwise they are a waste of time.  For example, one way to put accountability into the process is to have someone other than the author be accountable for any problems found.  There are numerous resources available if you decide that you wish to do inspections.

Conclusion

The statistics overwhelming suggest that inspections will not only remove defects from a software system but also prevent defects from getting in.  With inspections software defect removal rates can achieve 97% and without inspections you are lucky to get to 85%.

Since IBM investigated the issue in 1973, it is interesting to note that teams trained in performing inspections eventually learn how to prevent injecting defects into the software system.  Reduced injection of defects into a system reduces the amount of time spent in fire-fighting and in QA.

You can only inspect artifacts that you take the time to create.  Many smaller organizations don’t have any artifacts other than their requirements documents and source code. Discover which artifacts need to be created by augmenting your Bug Tracker (see Bug Tracker Hell and How To Get Out!).   Any phase of development where significant defects are coming from should be documented with an appropriate artifact and be subject to inspections.


Good books on how to perform inspections:


All statistics quoted from The Economics of Software Quality by Capers Jones and Olivier Bonsignour:

Capers Jones can be reached by sending me an email: Dalip Mahal


VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)