SR&ED and Eligibility

Most Canadian corporations know that the Canada Revenue Agency (CRA) gives out tax credits for Scientific Research and Experimental Development (SR&ED) work done by technical companies. This tax credit works out to 35% of the qualifying work and can be as high as 68% when overhead is factored in.

The Scientific Research part of the name makes some organizations assume that unless you are curing cancer, building rockets, or building a better mouse-trap that they must be engaged in rocket science to qualify for the tax credit — nothing could be further from the truth.

For organizations involved in software development the key to claiming SR&ED is the Experimental Development part of the title.  Assuming that you have a project which has technical uncertainty then you will qualify for the tax credit.

So what qualifies as technical uncertainty?

Technical uncertainty occurs when you face a business problem that is well specified with skilled resources and it is unclear how to proceed.  Examples of technical uncertainty would be needing to:

  • double the number of transactions that you currently process
  • increase the efficiency of an compression algorithm
  • implement a security model that does not exist

Experimental development occurs when you hit a fork in the proverbial development road and it is unclear which direction to take.  

Sometimes you will know that there are multiple design alternatives and have to create prototypes for the different alternatives to determine the best solution. Sometimes you will choose a design alternative and have to abandon the choice and back-up and take another path.  In both cases there is a clear decision point where code needs to be tested for multiple alternatives.

There is actually an easy way to know if you are facing technical uncertainty and facilitate applying for SR&ED tax credits.  Most developers do not like reinventing the wheel; when faced with a requirement that is technically challenging most developers will Google it to see if there is a solution to the problem.  If your developers are regularly looking for technical solutions odds are that you have SR&ED eligible work.

Saving technical searches is the easiest way to figure out how much SR&ED eligible work you have.

If you search for a technical solution to a challenge and discover:

  1. There is a solution for the problem but it is proprietary
  2. There is an available solution that would cost too much to acquire

Then this work will be SR&ED eligible if it leads to experimental development.  The CRA does not require that you be the first to solve a technical problem, only that you search for public solutions before executing experimental development.

What isn’t technical uncertainty?

There are a few issues which can masquerade as technical uncertainty and the CRA will not pay SR&ED credits for them:

  • Training
  • Poor requirements

If a COBOL programmer starts to develop software in Java then you will end up with quite a bit of experimental development as the programmer learns the new language.  However, the CRA will not pay for you to train developers.   Experimental development only occurs when developers who are familiar with the technologies that you are using (language, O/S, IDE, API) and then engage in experimental development.

To be explicit, the following would not qualify:

  • Language, Java developers needing to do C#
  • O/S, Developers familiar with Windows development developing on Android tablets
  • IDE, Developers familiar with Eclipse needing to use Sun’s NetBeans
  • API, Developers familiar with one SQL database switching to the API of another SQL database

Only competent developers that hit technical uncertainty and face experimental development qualify for SR&ED.

The CRA will not pay for you to figure out what your requirements are.  While you are working out your “business rules” you may look like you are resolving a technical challenge as you attempt multiple alternative paths.  However, creating code to solve a business problem does not qualify for SR&ED.

Changing requirements because of a technical challenge does qualify

How do I know if I did Experimental Development?

The CRA gives you up to 18 months from a fiscal year end to claim your tax credits.  The problem is that if you wait this long none of your developers will remember what they did!

If your year end was March 31, 2011 then as of today (August 27, 2012) you can still claim your tax credits from 2011.  The problem is that your developers will have trouble remembering what they did from March 31, 2010 to March 31, 2011.

Frankly speaking, you would be lucky to have your developers recall what they did last month. When looking back over time, there are two kinds of development that easily qualify for SR&ED:

  1. Work abandoned for a technical reason
  2. Building multiple prototypes to solve a technical problem

If you were trying to accomplish something, let’s say implementing a fine grain security model in a database and were forced to abandon the work for a technical reason then this will qualify for SR&ED.  If you abandoned the work because you no longer had the requirement, i.e. a business reason, then the work would not qualify for SR&ED.

If you ran up against a technical challenge and there were multiple design alternatives that lead to multiple prototypes being tried, then the work qualifies for SR&ED.  Even if the multiple design alternatives involved 3rd party software, as long as there were multiple prototypes and you had to write code then this work should qualify.

Document your technical challenges right away!

How do I Simplify the SR&ED Process?

The easiest way to simplify the SR&ED process is to track experimental development as it happens.  Once your developers solve a problem and use that solution for a few months then they will forget how difficult it was to solve the problem.

There are several techniques to help in the documentation of your SR&ED claim for the next year:

  1. Save your technical searches
  2. Tag your tasks in your project management system
  3. Train project managers to recognize SR&ED tasks
When the developers search for technical solutions and find none have them save the search (PDF, web page, etc).   If you work on several projects simultaneously then create a directory under each project where the developer will save the search.  In your project management system then have the developer document this information.

In your project management system (JIRA, Redmine, etc) have a tag for SR&ED so that tasks can be tagged for SR&ED.  As the developer or project manager discovers SR&ED tasks you can tag these tasks so that computing the hours for SR&ED next year will be easy.

Train your project managers to look for SR&ED tasks.  Inevitably, if a developer has a task that expands for a technical reason then he will have to notify the project manager about the event.  That will be the best time for the project manager to recognize SR&ED tasks and update the project management system.

Summary

All companies should make sure to have SR&ED trained people help you to make your claim.   The number of companies making SR&ED claims has increased strongly in the last few years and the CRA is more strict with regards to which projects qualify.
Do not be afraid to claim your SR&ED tax credits, if you have technical challenges that involve experimental development then they are yours.  Also, keep in mind that the earlier that you document your technical challenges the easier (and cheaper) it will be to make your SR&ED claim.

VN:F [1.9.22_1171]
Rating: 5.0/5 (3 votes cast)
VN:F [1.9.22_1171]
Rating: +1 (from 1 vote)

Efficiency is for Losers

Focusing on efficiency and ignoring effectiveness is the root cause of most software project failures.

Effectiveness is producing the intended or expected result. Efficiency is the ability to accomplish a job with a minimum expenditure of time and effort.

Effective software projects deliver code that the end users need; efficient projects deliver that code with a minimum number of resources and time.

Sometimes, we become so obsessed with things we can measure, i.e. project end date, kLOC, that we somehow forget what we were building in the first place.  When you’re up to your hips in alligators, it’s hard to remember you were there to drain the swamp.

Efficiency only matters if you are being effective.

After 50 years, the top three end-user complaints about software are:

  1. It took too long
  2. It cost too much
  3. It doesn’t do what we need
Salaries are the biggest cost of most software projects, hence if it takes too long then it will cost too much, so we can reduce the complaints to:

  1. It took too long
  2. It doesn’t do what we need

The first issue is a complaint about our efficiency and the second is a complaint about our effectiveness. Let’s make sure that we have common  definitions of these two issues before continuing to look at the interplay between efficiency and effectiveness.

Are We There Yet?

Are you late if you miss the project end date? 

That depends on your point of view; consider a well specified project (i.e. good requirements) with a good work breakdown structure that is estimated
by competent architects to take a competent team of 10 developers at least 15 months to build. Let’s consider 5 scenarios where this is true except as stated below:

Under which circumstances is a project late?

A. Senior management gives the team 6 months to build the software.
B. Senior management assigns a team of 5 competent developers instead of 10.
C. Senior management assigns a team of 10 untrained developers
D. You have the correct team, but, each developer needs to spend 20-35% of their time maintaining code on another legacy system
E. The project is staffed as expected

Here are the above scenarios in a table:

#
Team
Resource
Commitment
Months Given
Result
A
10 competent developers
100%
6
Unrealistic estimate
B
5
competent developers
100%
15
Under staffed
C
10 untrained developers
100%
15
Untrained staff
D
10 competent developers
65-80%
15
Team under committed
E
10 competent developers
100%
15
Late


Only the last project (E) is late because the estimation of the end date was consistent with the project resources available.

Other well known variations which are not late when the end date is missed:

  • Project end date is a SWAG or management declared
  • Project has poor requirements
  • You tell the end-user 10 months when the estimate is 15 months.

If any of the conditions of project E are missing then you have a problem in estimation.  You may still be late, but not based on the project end date computed with bad assumptions

Of course, being late may be acceptable if you deliver a subset of the expected system.

It Doesn’t Work



“It doesn’t do what we need” is a failure to deliver what the end user needs. How so we figure out what the end user needs?

The requirements for a system come from a variety of sources:

  1. End-users
  2. Sales and marketing (includes competitors)
  3. Product management
  4. Engineering

These initial requirements will rarely be consistent with each other. In fact, each of these constituents will have a different impression of the requirements. You would expect the raw requirements to be contradictory in places. The beliefs are like the 4 circles to the left, and the intersection of their beliefs would be the black area.

The different sources of requirements do not agree because:

  • Everyone has a different point of view
  • Everyone has a different set of beliefs about what is being built
  • Everyone has a different capability of articulating their needs
  • Product managers have varying abilities to synthesize consistent requirements
It is the job of product management to synthesize the different viewpoints into a single set of consistent requirements. If engineering starts before
requirements are consistent then you will end up with many fire-fighting meetings and lose time.

Many projects start before the requirements are consistent enough. We hope the initial requirements are a subset of what is required.
In practice, we have missed requirements and included requirements that are not needed (see bottom of post, data from Capers Jones)

The yellow circle represents what we have captured, the black circle represents the real requirements.

We rarely have consistent requirements when we start a project, that is why there are different forms of the following cartoon lying around on the Internet.

If you don’t do all the following:

  • Interview all stakeholders for requirements
  • Get end-users to articulate their real needs by product management
  • Synthesize consistent requirements

Then you will fail to build the correct software.  So if you skip any of this work then you are guaranteed to get the response, “It doesn’t do what we need”.

Effectiveness vs. Efficiency

So, let’s repeat our user complaints:
  1. It took too long
  2. It doesn’t do what we need

It’s possible to deliver the correct software late.

It’s impossible to deliver on-time if the software doesn’t work

Focusing on effectiveness is more important than efficiency if a software project is to be delivered successfully.


Ineffectiveness Comes from Poor Requirements

Most organizations don’t test the validity or completeness of their requirements before starting a software project.
The requirements get translated into a project plan and then the project manager will attempt to execute the project plan. The project plan becomes the bible and everyone marches to it. As long as tasks are completed on time everyone assumes that you are effective, i.e. doing the right thing.

That is until virtually all the tasks are jammed at 95% complete and the project is nowhere near completion.

At some point someone will notice something and say, “I don’t think this feature should work this way”. This will provoke discussions between developers, QA, and product management on correct program behavior. This will spark a series of fire-fighting meetings to resolve the inconsistency, issue a defect, and fix the problem. All of the extra meetings will start causing tasks on the project plan to slip.

We discussed the root causes of fire-fighting in a  previous blog entry.

When fire-fighting starts productivity will grind to a halt. Developers will lose productivity because they will end up being pulled into the endless meetings. At this point the schedule starts slipping and we become focused on the project plan and deadline. Scope gets reduced to help make the project deadline; unfortunately, we tend to throw effectiveness out the window at this point.

With any luck the project and product manager can find a way to reduce scope enough to declare victory after missing the original deadline.

The interesting thing here is that the project failed before it started. The real cause of the failure would be the inconsistent requirements.But, in the chaos of fire-fighting and endless meetings, no one will remember that the requirements were the root cause of
the problem.

What is the cost of poor requirements? Fortunately, WWMCCS has an answer.  As a military organization they must tracks everything in a detailed fashion and perform root cause analysis for each defect (diagram).

This drawing shows what we know to be true.

The longer a requirement problem takes to discover, the harder and more expensive it is to fix!  A requirement that would take 1 hour to fix will take 900 hours to fix if it slips to system testing.

Conclusion

It is much more important to focus on effectiveness during a project than efficiency. When it becomes clear that you will not make the project end date, you need to stay focused on building the correct software.
Are you tired of the cycle of:
  • Collecting inconsistent requirements?
  • Building a project plan based on the inconsistent requirements?
  • Estimating projects and having senior management disbelieve it?
  • Focusing on the project end date and not on end user needs?
  • Fire-fighting over inconsistent requirements?
  • Losing developer productivity from endless meetings?
  • Not only miss the end date but also not deliver what the end-users need?

The fact that organizations go through this cycle over and over while expecting successful projects is insanity – real world Dilbert cartoons.

How many times are you going to rinse and repeat this process until you try something different? If you want to break this cycle, then you need to start collecting consistent requirements.

Think about the impact to your career of the following scenarios:

  1. You miss the deadline but build a subset of what the end-user needs
  2. You miss the deadline and don’t have what the end-user needs
You can at least declare some kind of victory in scenario 1 and your resume will not take a big hit. It’s pretty hard to make up for scenario 2 no matter how you slice it.
Alternatively, you can save yourself wasted time by making sure the requirements are consistent before you start development. Inconsistent requirements will lead to fire-fighting later in the project.
As a developer, when you are handed the requirements the team should make a point of looking for inconsistent requirements.
The entire team should go through the requirements and look for inconsistencies and force product management to fix them before you start developing.
It may sound like a waste of time but it will push the problem of poor requirements back into product management and save you from being in endless meetings. Cultivating patience on holding out for good requirements will lower your blood pressure and help you to sleep at night.
Of course, once you get good requirements then you should hold out for proper project estimates 🙂

Want to see other sacred cows get tipped?Check out:

Moo?

Courtesy of Capers Jones via LinkedIn on 6/22

Customers themselves are often not sure of their requirements.

For a large system of about 10,000 function points, here is what might be seen for the requirements.

This is from a paper on requirements problems – send an email to capers.jones3@gmail.com if you want a copy.

Requirements specification pages = 2,500
Requirements words = 1,125,000
Requirements diagrams = 300
Specific user requirements = 7,407
Missing requirements = 1,050
Incorrect requirements = 875
Superfluous requirements = 375
Toxic harmful requirements = 18

Initial requirements completeness = < 60%

Total requirements creep = 2,687 function points

Deferred requirements to meet schedule = 1,522

Complete and accurate requirements are possible < 1000 function points. Above that errors and missing requirements are endemic.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Comments are for Losers

If software development is like driving a car then comments are road signs along the way.

Comments are purely informational and do NOT affect the
final machine code. Imagine how much time you would waste driving in a city where road signs looked like this one.

A good comment is one that reduces the development life cycle for the next developer that drives down the road.

A bad comment is one that increases the development life cycle for any developer unfortunate enough to have to drive down that road. Sometimes that next unfortunate driver will be you several years later!

Comments do not Necessarily Increase Development Speed

I was in university in 1985 (yup, I’m an old guy 🙂 ) and one of my professors presented a paper (which I have been unable to locate 🙁 ) of a study done in the 1970s. The study took a software program, introduced defects into it, and then asked several teams to find as many defects as they could. The interesting part of the study was that 50% of the teams had the comments completely
removed from the source code. The result was that the teams without comments not only found more defects but also found them in less time.

So unfortunately, comments can serve as weapons of mass distraction

Bad comments
A bad comment is one that wastes your time and does not help you to drive your development faster.

Let’s go through the categories of really bad comments:

  • Too many comments
  • Excessive history comments
  • Emotional and humorous comments

Too many comments are a clear case of where less is more. There are programs with so many comments that it obscures the code. Some of us have worked on programs where there were so many comments you could barely find the code!

History comments can make some sense, but then again isn’t that what the version control comment is for? History comments are questionable when you have to page down multiple times just to get to the beginning of the source code. If anything, history comments should be moved to the bottom of the file so that Ctrl-End actually takes you to the bottom of the modification history.


We have all run across comments that are not relevant. Some comments are purely about the  developer’s instantaneous emotional and intellectual state, some are about how clever they are, and some are simply attempts at humor (don’t quit your day job!).

Check out some of these gems (more can be found here):


// I am not sure if we need this, but too scared to delete.

//When I wrote this, only God and I understood what I was doing
//Now, God only knows

// I am not responsible of this code.
// They made me write it, against my will.

// I have to find a better job

try {

}
catch (SQLException ex) {
// Basically, without saying too much, you’re screwed. Royally and totally.
}
catch(Exception ex)
{
//If you thought you were screwed before, boy have I news for you!!!
}

// Catching exceptions is for communists

// If you’re reading this, that means you have been put in charge of my previous project.
// I am so, so sorry for you. God speed.

// if i ever see this again i’m going to start bringing guns to work

//You are not expected to understand this


Self-Documenting Code instead of Comments

We apply science to software by checking the functionality we desire (requirements model) against the behavior of the program (machine code model).

When observations of the final program disagree with the requirements model we have a defect which leads us to change our machine code model.

Of course we don’t alter the machine code model directly (at least most of us); we update the source code which is the last easily modified model. Since comments are not compiled into the machine code there is some logic to making sure that the source code
model be self-documenting. It is the only model that really counts!

Self-documenting code requires that you choose good names for variables, classes, function names, and enumerated types. Self-documenting means that OTHER developers can understand what you have done. Good self-documenting code has the same
characteristic of good comments; it decreases the time it takes to do development.

Practically, your code is self-documenting when your peers say that it is, not when YOU say that it is. Peer reviewed comments and code is the only way to make sure that code will lead to faster development cycles.

Comments gone Wild

Even if all the comments in a program are good (i.e. reduce
development life cycle) they are subject to drift over time. The speed of software development makes it difficult to make sure that comments stay in alignment with the source code. Comments that are allowed to drift become road signs that are no longer relevant to drivers.

Good comments go wild when the developer is so focused on getting a release out that he does not stop to maintain comments. Comments have gone wild when they become misaligned with the source code; you have to terminate them.

No animals (or comments) were harmed in the writing of this blog.

Commented Code
Code gets commented during a software release as we are experimenting with different designs or to help with debugging. What is really not clear is why code remains commented before the final check-in of a software release.

Over my career as a manager and trainer, I’ve asked developers why they comment their code. The universal answer that I get is “just in case”. Just in case what? At the end of a software release you have already established that you are not going to use your commented code, so why are you dragging it around? People hang on to commented code as if it is a “Get Out of Jail Free” card, it isn’t.

The reality is that commented code can be a big distraction. When you leave commented code in your source code you are leaving a land mine for the next developer that walks through it.

When the pressure is on to get defects fixed developers will
uncomment previously commented code to see if it will fix the problem. There is no substitute for understanding the code you are working on – you might get lucky when you reinstate commented code; in all likelihood it will blow up in your face.

Solutions

If your developers are not taking (or given) enough time to put in good comments then they should not write ANY comments. You will get more productivity because they will not waste time putting in bad comments that will slow everyone else down.

Time spent on writing self-documenting code will help you and your successors reduce development life cycles. It is absolutely false to believe that you do not have time to write self-documenting code.

If you are going to take on the hazards of writing comments then they need to be peer reviewed to make sure that OTHER developers understand the code. Unless the code reviewer(s) understands all the comments the code should not pass inspection.

If you don’t have a code review process then you are only commenting the code for yourself. The key principle when writing comments is Non Nobis Solum (not for ourselves alone).

When you run across a comment that sends you on a wild goose
 chase – fix it or delete them. If you are the new guy on the team and realize that the comments are wasting your time – get rid of them; your development speed will go up.

Other articles


Want to see other sacred cows get tipped?Check out:

Moo?

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Uncertainty trumps Risk in Software Development

Successful software development involves understanding uncertainty, and uncertainty only comes from a few sources in a software project.  The uncertainties of a software project increase with the size of the project and the inexperience of the team with the domain and technologies.The focus on this article is on uncertainty and not on risk.  In part 1 we discussed uncertainty and in part 2 we discussed risk, so it should be clear that:

All risks are uncertain, however, not all uncertainties are risks.

For example, scope creep is not a risk (see Shift Happens) because it is certain to happen in any non-trivial project.  Since risk is uncertain, a risk related to scope creep might be that the scope shifts so much that the project is canceled.  However, this is a useless risk to track because by the time it has triggered it is much too late for anything proactive to be done.

It is important to understand the uncertainties behind any software development and then to extract the relevant risks to monitor.  The key uncertainties of a software project are around:

  • requirements
  • technology
  • resources
  • estimating the project deadline

Uncertainty in Requirements
There are several methodologies for capturing requirements:

  • Business requirements document (BRD) or software requirement specification (SRS)
  • Contract-style requirement lists
  • Use cases (tutorial)
  • User stories

Regardless of the methodology used, your initial requirements will split up into several categories:

The blue area above represents what the final requirements will be once your project is completed, i.e. the System To Build.  The initial requirements that you capture are in the yellow area called Initial Requirements.

With a perfect requirements process the Initial Requirements area would be the same as the System To Build area.  When they don’t overlap we get the following requirement categories:

  1.  Superfluous requirements
  2. Missing requirements

Superfluous initial requirements tends to happen in very large projects or projects where the initial requirements process is incomplete.  Due to scope shift the Missing requirements category always has something in it (see Shift Happens).  If either of these two categories contains a requirement that affects your core architecture negatively then you will increase your chance of failure by at least one order of magnitude.

For example, a superfluous requirement that causes the architecture to be too flexible will put the developers through a high learning curve and lead to slow development.

If scalability is a requirement of the architecture but it is missing during the initial architecture then you will discover that it is difficult and costly to add later.


The physical equivalent would be the apartment building here on the right.  The foundation was insufficient to the needs of the building and it is slowly collapsing on one side.  Imagine the cost of trying to fix the foundation now that the building is complete.

I’ve been in start-ups that did not plan for sufficient scalability in the initial architecture; subsequently, the necessary scalability was only added with serious development effort and a large cost in hardware. Believe me, throwing expensive hardware at a software problem late in the life cycle is not fun or a practical solution :-(.


The overlapping box, Inconsistent Requirements, is to categorize known and missing requirements that turn out to be in conflict with other requirements.  This can happen when requirements are gathered from multiple user roles all of whom have a different view of what the system will deliver.

It is much easier and faster to test your requirements and discover inconsistencies before you start developing the code rather than discover the inconsistencies in the middle of development.  When inconsistencies are discovered by developers and QA personnel then your project will descend into fire-fighting (see Root cause of ‘Fire-fighting’ in Software Projects).

The physical equivalent here is to have a balcony specified to one set of contractors but forget to notify another set that you need a sliding door (see right).  When the construction people stumble on the inconsistency you may have already started planning for one alternative only to discover that rework is necessary to get to the other requirement.

Note, if you consistently develop software releases and projects with less than 90% of your requirements documented before coding starts then you get what you deserve (N.B. I did not say before the project starts 🙂 ).   One of the biggest reasons that Agile software development stalls or fails is that the requirements in the backlog are not properly documented; if you Google “poor agile backlogs” you will get > 20M hits.

Requirements Risks
Some risks associated with requirements are:

  • Risk of a missing requirement that affects the core architecture
  • Risk that inconsistent requirements cause the critical path to slip

Uncertainty in Technology
Technical uncertainty comes  from correctly using technology but failing to accomplish the goals as outlined by your requirements; lack of knowledge and/or skills will be handled in the next section (Uncertainty Concerning Resources).  Team resources that don’t have experience with technology (poorly documented API, language, IDE, etc) does not constitute a technical risk it is a resource risk (i.e. lack of knowledge).

Technical uncertainty comes from only a few sources:

  • Defective APIs
  • Inability to develop algorithms


Unforeseen defects in APIs will impact one or more requirements and delay development.  If there is an alternative API with the same characteristics then there may be little or no delay in changing APIs, i.e. there are multiple choices for XML parsing in Java with the same API.

However, much of the time changing to another API will cause delays because the new API will be implemented differently than the defective one. There are also no guarantees that the new API will be bug free.

Mature organizations use production APIs, but even then this does not protect you against defects.  The best known example has to be the Pentium bug from Intel discovered in 1994.  Although the bug did not seem to cause any real damage, any time you have an intermittent problem the source might always be a subtle defect in one of the APIs that you are using.

Organizations that use non-production (alpha or beta) APIs for development run an extremely high risk of finding a defect in an API.  This generally only happens in poorly funded start-ups where the technical team might have excessive decisional control in the choice of technologies.

The other source of technical uncertainty is the teams inability to develop algorithms to accomplish the software goals.  These algorithms relate to the limitations of system resources such as CPU, memory, batteries, or bandwidth concern, i.e.:

  • Performance
  • Memory management
  • Power management
  • Volume of data concerns

Every technical uncertainty is associated with one or more requirements.  The inability to produce an algorithm to satisfy a requirement may have a work-around with acceptable behavior or might be infeasible.

If the infeasible requirements prevents a core goal from being accomplished then the project will get canceled.  If affected requirements have technical work-arounds then the project will be delayed while the work-around is being developed.

Technical Risks
Some risks associated with technology are:

  • Risk that a defective API will cause us to look for another API
  • Risk that we will be unable to find a feasible solution for a core project requirement

Uncertainty Concerning Resources
When using the same team to produce the next version of a software product there is little to no resource uncertainty.  Resource uncertainty exists if one of the following are present:

  • Any team member is unfamiliar with the technology you are using
  • Any member of the team is unfamiliar with the subject domain
  • You need to develop new algorithms to handle a technical issue (see previous section)
  • Any team member is not committed to the project because they maintain another system
  • Turnover robs you of a key individual

Resource uncertainty revolves around knowledge and skills, commonly this includes: 1) language, 2) APIs, 3) interactive development environments (IDEs), and 4) methodology (Agile, RUP, Spiral, etc).  If your team is less knowledgeable than required then you will underestimate some if not all tasks in the project plan.

When team members are unfamiliar with the subject domain then any misunderstandings that they have will cause delays in the project.  In particular, if the domain is new and the requirements are not well documented then you will probably end up with the wrong architecture, even if you have skilled architects.

The degree to which you end up with a bad architecture and a canceled project depends on how unfamiliar you are with the subject domain and technologies being used.  In addition, the size of your project will magnify all resource uncertainties above.

The majority of stand-alone applications are between 1,000 and 10,000 function points.  As you would expect, the amount of the system that any one person can understand drops significantly between 1,000 and 10,000 function points.  The number of canceled projects goes up as our understanding drops because all uncertainties increase and issues fall between the cracks.  N.B. The total % of the system understood by a single person drops precipitously between 1,000 and 10,000 function points.

When there are team members committed to maintaining legacy systems then their productivity will be uncertain.  Unless your legacy system behaves in a completely predictable fashion, those resources will be pulled away to solve problems on an unpredictable basis.  They will not be able to commit to the team and multi-tasking will lower their and the teams productivity (see Multi-tasking Leads to Lower Productivity).

Resource Risks
Some risks associated with resources are:

  • The team is unable to build an appropriate architecture foundation for the project
  • A key resource leaves the project before the core architecture is complete

Uncertainty in Estimation
When project end dates are estimated formally you will have 3 dates: 1) earliest finish, 2) expected finish, and 3) latest finish.  This makes sense because each task in the project plan can finish in a range of time, i.e. earliest finish to latest finish.  When a project only talks about a single date for the end date, it is almost always the earliest possible finish so there is a 99.9% chance that you will miss it.  Risk in estimation makes the most sense if:

  • Formal methods are used to estimate the project
  • Senior staff accepts the estimate

There are numerous cost estimating tools that can do a capable job.  Capers Jones lists those methods, but also comments about how many companies don’t use formal estimates and those that do don’t trust them:

Although cost estimating is difficult there are a number of commercial software cost estimating tools that do a capable job:  COCOMO II, KnowledgePlan, True Price, SEER, SLIM, SoftCost, CostXpert, and Software Risk
Master are examples.

However just because an accurate estimate can be produced using a commercial estimating tool that does not mean that clients or executives will accept it.  In fact from information presented during litigation, about half of the cases did not produce accurate estimates at all and did not use estimating tools.  Manual estimates tend towards optimism or predicting shorter schedules and lower costs than actually occur.   

Somewhat surprisingly, the other half of the case had accurate estimates, but they were rejected and replaced by forced estimates based on business needs rather than team abilities. 

Essentially, senior staff have a tendency to ignore formal estimates and declare the project end date.  When this happens the project is usually doomed to end in disaster (see Why Senior Management Declared Deadlines lead to Disaster).

So estimation is guaranteed to be uncertain.  Let’s combine the requirements categories from before with the categories of technical uncertainty to see where our uncertainty is coming from.  Knowing the different categories of requirements uncertainty gives us strategies to minimize or eliminate that uncertainty.
Starting with the Initial Requirements, we can see that there are two categories of uncertainty that can addressed before a project even starts:
  1. Superfluous initial requirements
  2. Inconsistent requirements

Both of these requirements will waste time if they get into the development process where they will cause a great deal of confusion inside the team.  At best these requirements will cause the team to waste time, at worst these requirements will deceive the team into building the architecture incorrectly.  A quality assurance process on your initial requirements can ensure that both of these categories are empty.

The next categories of uncertainty that can be addressed before the project starts is:

  1. Requirements with Technical Risk
  2. Requirements Technically Infeasible

Technical uncertainty is usually relatively straight forward to find when a project starts.  It will generally involve non-functional requirements such as scalability, availability, extendability, compatibility, portability, maintainability, reliability, etc, etc.  Other technical uncertainties will be concerned with:

  1. algorithms to deal with limited resources, i.e. memory, CPU, battery power
  2. volume of data concerns, i.e. large files or network bandwidth
  3. strong security models
  4. improving compression algorithms

Any use cases that are called frequently and any reports tie up your major tables are sources of technical uncertainty.  If there will be significant technical uncertainty in your project then you are better off to split these technical uncertainties into a smaller project that the architects will handle before starting the main project.  This way if there are too many technically infeasible issues then at least you can cancel the project.

However, the greatest source of uncertainty comes from the Missing Requirements section.  The larger the number of missing requirements the greater the risk that the project gets canceled.  If we look at the graph we presented above:

You can see that the chance of a project being canceled is highly correlated with the % of scope creep. Companies that routinely start projects with a fraction of the requirements identified are virtually guaranteed to have a canceled project.

In addition, even if you use formal methods for estimation, your project end date will not take into account the Missing Requirements.  If you have a significant number of missing requirements then your estimates will be way off.

Estimation Risks

The most talked about estimation risk is schedule risk.  Since most companies don’t use formal methods, and those that do are often ignored, it makes very little sense to talk of schedule risk.

When people say “schedule risk”, they are making a statement that the project will miss the deadline.  But given that improper estimation is used in most projects it is certain that the project will miss its deadline , the only useful question is “by how much?“.

Schedule risk can only exist when formal methods are used and there is an earliest finish/latest finish range for the project.  Schedule risk then applies to any task that takes longer than its latest finish and compromises the critical path of the project.  The project manager needs to try to crash current and future tasks to see if he can get the project back on track.  If you don’t use formal methods then this paragraph probably makes no sense 🙂

Conclusion

The main sources of uncertainty in software development comes from:

  • requirements
  • technology
  • resources
  • estimates

Successful software projects look for areas of uncertainty and minimize them before the project starts. Some uncertainties can be qualified as risks and should be managed aggressively by the project manager during the project.

Uncertainty in requirements, technology, and resources will cause delays in your project.  If you are using formal methods than you need to pay attention to delays caused by uncertainties not accounted for in your model.  If you don’t use formal methods then every time you hit a delay caused by an uncertainty, then that delay needs to be tacked on to the project end-date (of course, it won’t be 🙂 ).

If your project does not have strong architectural requirements and is not too big (i.e. < 1,000 function points) then you should be able to use Agile software development to set-up a process that grapples with uncertainty in an incremental fashion.  Smaller projects with strong architectural requirements should set up a traditional project to settle the technical uncertainties before launching into Agile development.

Projects that use more traditional methodologies need to add a quality assurance process to their requirements to ensure a level of completeness and consistency before starting development.  One way of doing this is to put requirements gathering into its own project.  Once you capture the requirements, if you establish that you have strong architectural concerns, then you can create a project to build out the technical architecture of the project.  Finally you would do the project itself.  By breaking projects into 2 or 3 stages this gives you the ability to cancel the project before too much effort is sunk into a project with too much uncertainty.

Regardless of your project methodology; being aware of the completeness of your requirements as well as the technical uncertainty of your non-functional requirements will help you reduce the chance of project cancellation by at least one order of magnitude.

It is much more important to understand uncertainty that it is to understand risk in software development.


Appendix: Traditional Software Risks
This list of software risks courtesy of Capers Jones.  Risks listed in descending order of importance.

  • Risk of missing toxic requirements that should be avoided
  • Risk of inadequate progress tracking
  • Risk of development tasks interfering with maintenance
  • Risk of maintenance tasks interfering with development
  • Risk that designs are not kept updated after release
  • Risk of unstable user requirements
  • Risk that requirements are not kept updated after release
  • Risk of clients forcing arbitrary schedules on team
  • Risk of omitting formal architecture for large systems
  • Risk of inadequate change control
  • Risk of executives forcing arbitrary schedules on team
  • Risk of not using a project office for large applications
  • Risk of missing requirements from legacy applications
  • Risk of slow application response times
  • Risk of inadequate maintenance tools and workbenches
  • Risk of application performance problems
  • Risk of poor support by open-source providers
  • Risk of reusing code without test cases or related materials
  • Risk of excessive feature “bloat”
  • Risk of inadequate development tools
  • Risk of poor help screens and poor user manuals
  • Risk of slow customer support
  • Risk of inadequate functionality
VN:F [1.9.22_1171]
Rating: 4.0/5 (1 vote cast)
VN:F [1.9.22_1171]
Rating: +1 (from 1 vote)