Category Archives: project management

Ready, Fire, Aim: Why correct requirements are rarely gathered

Posted on by
Reply

missed_target-300x204Connecting with your customers and delivering value depends on understanding your customer’s requirements and selling the correct product or solution that solves your customer’s problems.

Commonly, the requirements gathering process is done hastily or not at all in the rush to get the sale. After all, the faster you can make the sales process go, the faster the money is in the bank — correct?

tailoringThe more that product customization or creation is required, the more it is important understand the customer’s actual problem.  The more time required to build and implement a solution means will not only lead to a failed sale but also to a more disappointed customer and a loss of future revenue.

If you build software, which has long lead times, it is even more important to make sure that what you are building will satisfy the customer’s requirements as changing the final software solution will be nearly impossible and very costly.

Why do we continually misunderstand and sell the wrong solutions and building the wrong software?

Human Nature

Time pressures to make a sale put us under pressure and this stress leads to making quick decisions about whether a product or solution can be sold to a customer.  We listen to the customer but interpret everything he says according to the products and solutions that we have.

Often the customer will use words that seem to match exactly the products we have.  But then after the sale once we have to implement we often find that either we deliver a poor solution to the customer or a solution is infeasible and we must refund the customers money.

However, behind every word that the customer is using there is an implied usage, and understanding that implied usage is where we fail to gather requirements.

For example, suppose the customer says I need a car. Suppose that you sell used cars:

  • the customer asked for a car
  • you sell cars

Ergo problem solved!

  • What if the customer needs an SUV but you don’t have any?
  • What if the customer really needs a truck?
  • What if the customer needs a car with many modifications?
  • What if the car the customer needs has never been built?

We hear the word car and we think that we know what the customer means.  The order-taker sales person will spring into action and sell what he thinks the customer needs. Behind the word car is an implied usage and unless you can ferret out the meaning that the customer has in mind, you are unlikely to sell the correct solution.

If you don’t understand how the customer will to use your solution then you don’t understand the problem.

Comedy of Errors

For products that require customization, the sale will get transferred to professional services that will dig deeper into the customer’s requirements.  At this point you discover that the needs of the customer cannot be met.  This leads to sales people putting pressure on professional services and product management to ‘find a solution’, after all, losing the sale is not an option.

Sometimes heroic actions by the product management, professional services, and software development teams lead to a successful implementation, but usually not until there has been severe pain at the customer and midnight oil burned in your company.  You can eventually be successful but that customer will never buy from you again.

Consequences

As WIlliam Ralph Inge said, There are no rewards or punishments — only consequences.

The consequence of selling the wrong solution to a customer is:

  • Whether you lose the sale or not, the customer loses faith in you
  • The sales person is perceived as incompetent in the rest of the organization especially by professional services and software devleopment
  • Your cost of sales and implementation is much higher than expected
  • Your reputation is damaged

Conclusion

Selling the correct solution to the customer requires that you understand the customer’s problem before you sell the solution.

When customization is required, good sales people engage resources that can capture the customer’s requirements accurately and assess that you can deliver a solution to the customer.

Slowing down to understand the customer requirements and how you will solve his problems is the key.  By understanding the customer’s requirements and producing the correct solutions you become a trusted adviser to the customer.

 

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Do Project Managers need Domain Experience?

Posted on by
Reply

Opinions vary on whether a project manager needs to have domain experience.  Certainly project managers that do not have domain experience will be the first to say that domain experience is not necessary as long as they have access to excellent subject matter experts.

I would advocate a more nuanced position; that is, a project manager does not need domain experience IF his subject matter experts understand the risks and dependencies that are inherent to the domain.

Let’s go through a couple of personal projects that I have been involved with where the project manager did not have domain experience.

Telco Project

I am currently involved in a project that involves a LAN/WAN/WIFI upgrade of a large customer for a large telecommunications company.  The project manager does not have domain expertise in networks and is counting on the subject matter experts to provide him sufficient input to execute the project.

The subject matter experts are so advanced in their knowledge of networks that they no longer understand what beginners (i.e. the project manager) do not know.  They have assumed that when they indicate things to the project manager that he understands what they mean and will take appropriate actions.

The project manager is continually running into situations where he did not understand the implications of certain risks and dependencies.  This has caused a certain amount of rework and delays.

Fortunately, this is not a project with tremendous amounts of risk or dependencies so the project will be late but will succeed.

Mobile Handset Project

In the distant past ,I was part of a team that was building a mobile POS terminal that worked over cellular (GSM, CDMA).  The project manager in this situation did not have domain experience and was counting on the subject matter experts.  In this case, the subject matter experts were very good at general design, but not experts in building cellular devices.

Because the subject matter experts were not specialists, they knew most of the key principles of designing mobile handsets but did not understand all the nuances of handset design.  There were several key issues required by practical handset manufacturing that were overlooked by the generalists and ended up creating such a strong cost over-run that the start-up went out of business.

Sumary

In the first project, the subject matter experts were extremely good, however, the project manager failed to understand the implication of some of their statements and this introduced large delays in the project.

In the second project, the subject matter experts were generalists and did not understand all the risks and dependencies of the project.  The project manager (and start-up) were doomed to fail because “you don’t know what you don’t know”.

Both these projects show that a project can be delayed or fail because a project manager does not have domain experience.

Conclusion

So if a project does not have many uncertainties and dependencies then it is extremely likely that the project manager does not require domain experience and can rely to some degree on his subject matter experts.

However, if the project has complex uncertainties and/or dependencies then a good project manager without domain experience is likely to find himself in a several positions where the consequences of not understanding the uncertainties and dependencies will either introduce serious rework or torpedo the project.

VN:F [1.9.22_1171]
Rating: 5.0/5 (1 vote cast)
VN:F [1.9.22_1171]
Rating: +2 (from 2 votes)

The Documentation Dilemma

Posted on by
Reply

Virtually everyone complains that there is poor documentation for software systems. However, if you ask the same people during a project to document what they are doing then they will tell you that they are too busy and that they will document things later.  The reality is that the documentation rarely gets done, and when it does, it is often useless or too high level.

What is going on here?

This is like when Alice in Wonderland visits the White Queen who offers her jam tomorrow.  But of course, tomorrow never comes.  So when someone says that the documentation will be done later, they are telling you that the documentation will never be done.

Claims that documentation will eventually be done are lies

So the first point to clarify is that just because documentation is done does not mean that documentation exists. For the purposes of this article, the documentation exists if:

The documentation actually helps to understand the program

Let’s motivate the discussion of why documentation is often poor or missing with a common situation. Let start with low level code comments, quite often you run across such code comments:

// Convert float to string
$value = float_to_string( $value );

This is not documentation.  This kind of comment can often be found in code for organizations that insist that a certain number of comments be created.  This comment is worse than useless because it takes up space for no reason whatsoever.  For other reasons why code comments are problematic see Comments are for Losers.

Code comments should not tell us why the code is doing something but rather why the code is doing it.  If other developers can’t read your code and understand what it is doing then that is a different problem.

Similarly at a high level, documentation should explain why something is being done, not how it is being done.  For example, writing down directions to get to a friend’s house does not explain why you are going to your friends house.  Yet most of the documentation that we produce resembles this, it is a documentation of the obvious.

Good documentation focuses on what is being done, not how it is being done

The interesting thing is that developers are fairly disciplined about using version control systems, yet there is a complete lack of discipline (in general) when it comes to documentation.

Of course I expect documentation to be done if the project is poorly planned and there is not enough time to even implement the code.  Projects that are doomed to failure before they start have much bigger problems than documentation to solve.  Projects that are properly planned but are executed with organizations with common bad practices will also likely have a failed project (see Stop It! No… really stop it).

So it stands to reason that only the 30% of successful projects out there should be concerned about documentation.  The other 70% have many other issues to figure out first, like for example, how to execute a successful project (see Failed? You get what you deserve!).

If you don’t have repeatable and reliable processes for successful projects, then documentation is not your biggest problem

One of the reasons that documentation is so poor is that we tend to write it after we have gotten the code working.  Once the code is working, our minds are focused on what it did to get the code working and we then document what we consider to be the difficult part of the implementation.

The only time that you are thinking about what you are trying to do is before you have written the code.  This is clearly the best time to write the documentation.  The problem is that developers move from thinking about how to solve a problem to getting their hands on the keyboard to write code to solve the problem.  In the blink of an eye the programmer switches modes and gets caught up in trying to solve the problem.

So if you seriously want your projects to be documented, then you must document the ‘what’ before your hands hit the keyboard and you are caught up in solving the problem.

Also, documentation is something that needs to be done every day. Waiting until the end of the project and assuming that you will remember what you were thinking about how to achieve the various project goals is virtually impossible.

Good documentation requires just as much (if not more) thinking than what is required to solve the problem.  Much of the time people don’t have a good idea of what problem they are solving.  By truly spending time on figuring out what you have to solve, then documenting it, sounds like it would take longer — but you will be surprised to find out that this actually saves time.

Note, if you don’t produce good documentation then you shouldn’t expect anyone else to either.

Note, this is not associative, i.e. just because you produce good documentation will not entitle you to expect that anyone else does 🙂

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Team Conflict is for Losers

Posted on by
Reply

Loser, smallIt is a guarantee that don’t like someone on your development team and they have behaviors or habits that you might find objectionable:

But as irritating as you find your co-workers, odds are:

You do something that they find annoying…

Annoyances and poor communication can lead to conflicts that range from avoidance to all out war where people get drawn into taking sides.  But consider the cost of team conflict :

Issue Productivity Software Quality
Internal team conflict -10% -15%
Management conflict -14% -19%

The table above is only showing the average result of conflict, some of us have been in situations that get much, much worse.

Software development is not a popularity contest, you don’t have to like everyone that you work with.  However, if you allow your feelings of annoyance escalate into conflict then there is a real cost to your project and ultimately in your stress levels.

All conflicts start with disagreements.  The Communications Catalyst2 talks about the following cycle:Disagree, Defend, Destroy

  • Disagree
  • Defend
  • Destroy

When you disagree with your coworkers then they don’t feel listened to.  They will then defend their position by digging in their heels, then you will dig in your heels and the road to destruction starts. If there are any annoying habits present then the conflict will escalate quickly.

If things get out of hand then people start taking sides and productivity takes a major hit. In the worst conflicts this leads to loss of key personnel, which has been measured to be:

Loss of key personnel, productivity -16%, quality -22%

Losing key personnel who have comprehensive knowledge of business rules and organizational practices tied up in their heads often causes projects to face fault and come to a stand still.

You may feel justified in starting a conflict or escalating one, however, as clever as you think you are, conflict hurts everyone — yourself included.  Just remember:

It is virtually impossible to start a conflict that doesn’t boomerang back and bite you in the @ss!

4 Ways to Avoid or Reduce Conflicts

Things to consider to avoid conflict:

  • Don’t disagree first, signal that the other person has been heard
    • You will rarely agree with everything that someone else says, but start by agreeing with the part that you do agree with.1 This will at least signal that you have heard them and reduce their anxiety that you are not listening to them.
    • Even mechanically echoing everything that they just said is a way to signal that you heard what was said.
    • Once this is done, then talk about what you don’t agree with.
  • Don’t interrupt people.
    • When you are excited and thoughts are springing to mind then you may be tempted to do all the talking and stop listening; get this under control, take a breath, and let others talk.
    • People generally consider it rude when you interrupt and will assume arrogance on your part.  If you are not trying to be arrogant and someone tells you this then wake up — you need to listen.
  • Don’t be frustrated when people don’t understand you
    • If you really know something that others don’t then simply restating your point of view will not improve their understanding.
    • If your friend is lost in a new shopping mall then describing your location will not be helpful in helping him find you.  You need to find out where he is and walk him through the steps of getting to your location.
    • Be open to the idea that there might be something that you are not seeing.  With additional information you might revise your point of view.
  • Don’t automatically assume that someone is insulting you
    • In virtually every case where someone feel insulted this is a knee-jerk reaction to a misunderstanding where no insult was intended.
    • Jumping to conclusions is not good under any circumstance, but is lethal in social interactions.

Managers should be on the lookout for the signs of conflict and clear them up while they are still small.  Most conflicts arise from simple misunderstandings.

You will notice that most organizations will promote people based on their ability to work with others and resolve conflicts over competence.

Learning how to resolve conflicts is often your ticket to an overdue promotion…

References

  1. Carnegie, Dale.  How to Win Friends and Influence People. 1998.
  2. Connolly, Mickey and Rianoshek, Richard.  The Communication Catalyst, 2002.
  3. Jones, Capers and Bonsignour, Olivier.  The Economics of Software Quality.  Addison Wesley.  2011
  4. Kahneman, Daniel. Thinking Fast and Slow. 2011

Side Note

My best friend also works in the tech sector, and despite being friends for almost 25 years we have very few beliefs or habits in common.  There are subjects that we agree on, but then we don’t agree on how they should be handled.  We virtually never take the same action under the same conditions.

Even though we are very different people this has never stood in the way of us being able to do things together.  If you look around you will see radically different people that manage to cooperate and even thrive.

The key to all working relationships especially when the other person is very different from you is respect.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: +1 (from 1 vote)

Infeasible Projects: Executive Ignorance or IT Impotence?

Posted on by
Reply

IDohDoh2nfeasible software projects are launched all the time and teams are continually caught up in them, but what is the real source of the problem?

There are 2 year actual projects for which the executives set a 6 month deadline.  The project is guaranteed to fail but isthis due to executive ignorance or IT impotence?

InfeasibleTimelineThere is no schedule risk in an infeasible project because the deadline will be missed.  Schedule risk only exists in the presence of uncertainty (see Schedule Risk is a Red Herring!!!)

As you might expect, all executives and IT manager share responsibility for infeasible projects that turn into death marches.  Learn about the nasty side effects Death March Calculus.

The primary causes for infeasible projects are:

  • Rejection of formal estimates
  • No estimation or improper estimation methods are used

Rejecting Formal Estimates

Infeasible

This situation occurs frequently; an example would be the Denver Baggage Handling System (see Case Study).

The project was automatically estimated (correctly) to take 2 years; however, executives declared that IT would only have 1 year to deliver.

Of course, they failed1.

The deadline was rejected by executives because it did not fit their desires.  They could not have enjoyed the subsequent software disaster and bad press.

When executives ignore formal estimates they get what they deserve.  Formal estimates are ignored because executives believe through sheer force of will that they can set deadlines.

If IT managed to get the organization to pay for formal tools for estimating then it is not their problem that the executives refuse to go along with it.

Improper Estimation Methods

The next situation that occurs frequently is using estimation processes that have low validity.  Estimation has been extensively studied and documented by Tom DeMarco, Capers Jones, Ed Yourdon, and others.

IceBergImproper estimation methods will underestimate a software project every time. Fast estimates will be based on what you can think of, unfortunately, software is not tangible and so what you are aware of is like the tip of an iceberg.

None of this prevents executives demanding fast estimates from development.  Even worse, development managers will cave in to ridiculous demands and actually give fast estimates.

Poor estimates are guaranteed to lead to infeasible projects (see Who needs Formal Measurement?)

Poor estimates are delivered by IT managers that:

  • Can’t convince executives to use formal tools
  • Give in to extreme pressure for fast estimates

Infeasible projects that result from poor estimates are a matter of IT impotence.

Conclusion

ChildWithIceCreamBoth executive ignorance and IT impotence lead to infeasible projects on a regular basis because of poor estimates and rejecting estimates; so there is no surprise here.

However, infeasible projects are a failure of executives and IT equally because we are all on the same team.  It is not possible for part of the organization to succeed if the other one fails.

IntestinalFortitudePossibly a greater share of problem is with IT management.  After all, whose responsibility is a bad decision — the guys that know what the issues are or the ones that don’t.

If a child wants ice cream before they eat dinner then whose fault is it if you cave in and give them the ice cream?

Unfortunately, even after 60 years of developing software projects, IT managers are either as ignorant as the executives or simply have no intestinal fortitude.

Even when IT managers convince executives of the importance of estimating tools, the estimates are routinely discarded because they do not meet executive expectations.

Rejection of automated estimates: productivity -16%, quality -22%

Until we can get a generation of IT managers that are prepared to educate executives on the necessity of proper estimation and be stubborn about holding to those estimates, we are likely to continue to have an estimated $3 trillion in failures of software projects every year.

End Notes

1For inquiring minds, good automated estimation systems have been shown to be within 5% of time and cost on a regular basis. Contact me for additional information.

References

VN:F [1.9.22_1171]
Rating: 5.0/5 (2 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Schedule Risk is a Red Herring!!!

Posted on by
Reply

Red HerringWe often hear the term schedule risk, however, it is generally a Red Herring. Stating that the schedule might stretch is about as useful as saying that eating can cause you to gain weight.

You may be correct but it gives you no leverage to solve the problem

Schedules slip as a result of a problem, if you want to solve the problem then you must identify the root cause.  Any problem will result in a task taking longer than expected and potentially affecting the schedule.

TwoSidesSameCoinRisk and uncertainty are two sides of the same coin.  Without uncertainty there is no risk.

No Uncertainty = No Risk

A risk is a contingent liability; a risk is a future event that is uncertain that has consequences.

The key words are future and uncertain.

If 6 months remain and the deadline is in 2 months then there is no schedule risk because there is no uncertainty.

6 months late means that the earliest that the critical path items can finish is in 6 months. Just because the project has not hit the deadline and senior staff doesn’t understand the project is late does not qualify the team to talk as if the outcome is uncertain.

It is disingenuous and cowardly to suggest to senior staff that a deadline is possible when you know that it is not.

When the team knows that they are late, they often talk about tasks as being risky simply because they hope that miracles can happen1.

Hope is not a strategy

In fact, Kahneman points out all of us are wired to bet (pray?) on unlikely outcomes when faced with certain losses, i.e. we double down when faced with a loss.  Team members know about the negative consequences of failure and make projects seem possible simply because they want to delay the pain. Even worse, as the situation gets more desperate people will take bigger and bigger risks.

IntestinalFortitudeUsing the term schedule risk when a project is not feasible essentially robs the managers of making a course correction until the point where very little can be done.

At a minimum, money can be saved by winding the project down. Few people have the intestinal fortitude to speak out when they know that a project is late.  Unfortunately, cowardice is very common.

If you take a paycheck then you have an obligation to your organization to tell them when a project is late.

So it makes no sense to talk about schedule risk when:

  • The project is late and you know it
  • The project is not late but you see schedule items slipping

In the latter case you are much better to talk about why things are slipping rather than using the term schedule risk.  By talking about the root cause of the slippage, especially early in a project, can lead to you either solving the problem or adjusting the project deadline.  Either way, you will have a greater chance of ending up with a feasible project.

Related Articles

References

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Seriously. The Devil Made me do It!

Posted on by
Reply

good vs evilJust as eternal as the cosmic struggle between good and evil is the challenge between our two natures. Religion aside, we have two natures, the part of us that:

  • thinks things through; make good or ethical decisions a.k.a. our angelic nature
  • react immediately; make quick but often wrong decisions a.k.a. our devil nature

Guess God left a bug in our brains so that it emphasizes fast decisions over good / ethical decisions.

Quite often we make sub-optimal or ethically ambiguous decisions under pressure

You decide…

SteamingPileSituation: Your manager comes to you and says that something urgent needs to be fixed right away. Turns out the steaming pile of @#$%$ that you inherited from Bob is malfunctioning again.

Of course Bob created the mess and then conveniently left the company; in fact, the code is so bad that the work-arounds have work-arounds.

Bite the bullet, start re-factoring the program when things goes wrong.  It will take more time up front, but over time the program will become stable.

Find another fast workaround and defer the problem to the future.  Find a good reason why the junior member of the team should inherit this problem.

MultiplePathsSituation: You’ve got a challenging section of code to write and not much time to write it.

Get away from the computer, think things through.  Get input from your peers, maybe they have seen this problem before. Then plan the pathways out and write the code once cleanly. Taking time to plan seems counter intuitive, but it will save time.

Naw, just sit at the keyboard and bang it out already.  How difficult can it be?

BlameSituation: The project is late and you know that your piece is behind schedule.  However, you also know that several other pieces are late as well.

Admit that you are late and that the project can’t finish by the deadline.  Give the project manager and senior managers a chance to make a course correction.

Say that you are on schedule but you are not sure that other people (be vague here) will have their pieces ready on time and it could cause you to become late.

Measurement, smallSituation: You have been asked to estimate how long a critical project will take.  You are only been given a short time to come up with the estimate.

Tell the project manager that getting a proper estimate takes longer than a few hours. Without proper estimates the project is likely to be severely underestimated and this will come back to bite you and the project manager in the @$$.

Tell the project manager exactly the date that senior management wants the project to be finished by.  You know this is what they want to hear, why deal with the problem now? This will become the project manager’s problem when the project is late.

The statistics show that we often don’t listen to our better (angelic?) natures very often. So when push comes to shove and you have to make a sub-optimal or less than ethical decision, just remember:

The devil made you do it!

Run into other common situations, email me

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: +1 (from 1 vote)

Project failed? You get what you deserve!

Posted on by
Reply

3 of 10 software projects fail, 3 succeed, and 4 are ‘challenged’1.  When projects fail because you cut corners and exceed your capabilities then — you get what you deserve.  You don’t deserve pity when you do it to yourself.

We estimate that between $3 trillion and $6 trillion dollars are wasted every year in IT.  Most of this is wasted by organizations that are unskilled and unaware that they are ignorant.

Warning this article is long!

However, there are organizations that succeed regularly because they understand development, implement best practices, and avoid worst practices (see Understanding Your Chances).

In fact, McKinsey and Company in 2012 stated:

A study of 5,400 large scale IT projects  finds that the well known problems with IT Project Management are persisting. Among the key findings quoted from the report:

    • 17 percent of large IT projects go so badly that they can threaten the very existence of the company
    • On average, large IT projects run 45 percent over budget and 7 percent over time, while delivering 56 percent less value than predicted

law-unintended-consequencesProjects fail consistently because organizations choose bad practices and avoid best practices and wonder why success is elusive (see Stop It! No… really stop it. to understand the common worst 5 practices)

What is amazing is that failures do not prompt the incompetent to learn why they failed.

RinseAndRepeatEven worse, after the post-fail finger pointing ceremony, people just dust themselves off and rinse and repeat.

The reality is that we have 60 years of experience in building software systems.  Pioneers like Watts Humprey, M.. E. Fagan, Capers Jones, Tom DeMarco, Ed Yourdon, and institutions like the Software Engineering Institute (SEI)  have demonstrated that software complexity can be tamed and that projects can be successful2.

The worst developers are not even aware that there is clear evidence about what works or what doesn’t in software projects.  Of course, let’s not let the evidence get in the way of their opinions.

IngredientsIngredients of a Successful Project

Successful software projects generally have all the following characteristics:

  • Proper business case justification and good capital budgeting
  • Very good core requirements for primary functionality
  • Effective sizing techniques used before executing the project
  • Appropriate project management to the size of the project and to the philosophy of the organization
  • Properly trained personnel
  • Focus on pre-test defect removal

MissingPiecesEvery missing characteristic reduces your chance of success by an order of magnitude.  If you know that one or more of these characteristics are missing then you get what you deserve!

Missing some of these elements doesn’t guarantee failure, but it severely decreases your chance at success.

Let’s go through these elements in order.

This article is very long, so this is a good place to bail if you don’t have time.

Proper Business Case

IfYouBuildItThis is the step that many failed projects skip over, the hard work behind determining if a project is viable or not.

Organizations take the Field of Dreams approach, i.e. “If you build it, they will come...” and skip this step due to ignorance, often resulting from executives who do not understand software (see No Business Case == Project Failure). These are executives that do not have experience with software projects and assume that their force of personality can will software projects to success.

Some organizations claim to build business cases, but these documents are worthless.  I even know of public companies that write the business case AFTER the project has started, simply to satisfy Sarbanes-Oxley requirements.

A proper business case attempts to quantify the requirements and technical uncertainty of a software project.  It does due diligence into what problem is being solved and who it is solving the problem for.  It at least verifies with a little effort that the cash flows resulting from the project will be NPV positive.

Business cases are generally difficult to write because they involve getting partial information.  This can be very difficult if your analysts are substandard (see When BA means B∪ll$#!t Artist).

Very Good Core Requirements

Skeleton Once a project has a proper business case then you need to capture the skeleton of the core requirements.  This is a phase where you determine the primary actors of the system and work out major use case names.

Why expand requirements before starting the project?

Executives have a business to run and need to know when software will be available.  If you don’t know how big your project is then you can’t create an effective project plan.  You don’t want to capture all the requirements so core requirements (i.e. a good skeleton) helps you to size the project without having to get the detailed requirements.

This is why executives like the waterfall methodology. On the surface, this methodology seems to have a predictable timeline — which is what they need to synchronize other parts of the business.  The problem is that the waterfall methodology DOES NOT WORK (see last page).

The only way for managers to get a viable estimate of a software project is to expand the business case into requirements that allow you to determine the project’s size before you start it.

Calculate House SizeThis process is just like determining the cost for a house by the square footage and the quality, i.e. 2500 sq. ft at normal quality (~$200 per sq. ft.) would be approximately $500K, even without detailed blueprints.  Very accurate estimates can be derived by sizing a project using function points.

Effective Estimation

RulersNow that you have core requirements, you can determine the size of the project and get an approximate cost.  You are fooling yourself if you think that you can size large projects without formal estimates (see Who needs formal measurement?)

Just like you can determine the approximate cost of a house if you know the square footage and the quality, you can estimate a software project pretty accurately if you know how many function points (i.e. square footage) and quality requirements of the project3.

There is so much literature available on how to effectively size projects, so do yourself a favor and look it up.  N.B. There are quite a few reliable tools for an accurate estimate of software projects, i.e. COCOMO II, SLIM, SEER-SEM.  See also Namcook Analytics,

If you don’t size a project then your project plan predicts nothing

Of course, you could always try a management declared deadline which is guaranteed to fail (see Why Senior Management Declared Deadlines lead to Disaster)

Appropriate Project Management

AgileManifestoYou must select a project methodology appropriate to the organization.  Many developers are trying to push their organizations towards Agile software development, although many developers are actually quite clueless about what Agile development is.

Agile software development needs buy-in from the top of the organization.  Agile software development will probably do very little for you if you are not doing business cases and gathering core requirements before a project.

Discover how developers who claim that they are ‘Agile’ have fooled themselves into thinking that they are doing Agile development. (see Does Agile hide Development Sins?)

Trained Personnel

IncompetenceManagement often confuses seniority with competence.  After all, if someone has been with the company for 10 years they must be competent, no?  The reality is that most people with 10 years of experience only have 1 year repeated 10 times.  They are no more skilled then someone with 1 year under their belt.

Learn why in general it may be useful to get rid of older developers that are not productive (see No Experience Required!).  Also when it comes to development, you are definitely better off with people that do not rush to write code (see Productive Developers are Smart and Lazy)

Focus on Pre-Test Defect Removal

I’ve written extensively on pre-test defect removal, see Are Debuggers Crutches? for more information.

Conclusion

It is likely that you know all these ingredients that make for a successful project, you’ve just assumed that even though all these characteristics are not present that you just can’t fail.

Quite often projects fail under the leadership of confident people who are incompetent and don’t even know that they are incompetent.  If you want to know why intelligent people often do unintelligent things see Are You are Surrounded by Idiots?  Unfortunately, You Might be the Idiot..

There probably are projects that fail out there because of circumstances out of their control (i.e. natural disasters, etc) but in most failed projects you get what you deserve!

Ingredients

Fallacy of the Waterfall Methodology

The waterfall methodology is widely attributed to Winston W. Royce.

The irony is that the paper he published actually concludes that:

In my experience, however, the simpler method (i.e. waterfall) has never worked on large software development and efforts and the costs to recover far exceeded those required to finance the five step process listed.

That is Mr. Royce said that the waterfall process would never work.  So much for the geniuses that only read the first 2 pages of the paper and then proceeded to create the “waterfall method” and cost organizations trillions of dollars in failed projects each year.

The waterfall methodology was pushed down our throats by ignorant managers that saw that the waterfall seemed to mimic factory processes.  Because this was the process they understood, they icorrectly assumed that this was the right way to develop software.

If any of these guys had bothered to read more than 2 pages from the Royce paper they would have realized that they were making a colossal blunder.

Back to article

1 Challenged means that the project goes significantly over time or budget. In my estimation, ‘challenged’ simply means politically declaring victory on a project that has really failed.

2 This applies to projects that are 10,000 function points or less. We still have problems with projects that are larger than this, but the vast majority of projects are under this threshold.

3 Quality requirements depend on how reliable the project must be. If the risk is that someone might die because of a software malfunction the quality, and therefore cost, must be much higher than if software failures only constitute an annoyance.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Don’t be a Slave to Your Tools

Posted on by
Reply

Abstract SlaveDevelopers attach quickly to tools because they are concrete and have well defined behavior.  It is easier to learn a tool than to learn good practices or methodology.

Tools only assist in solving problems, they can’t solve the problem by themselves. A developer who understands the problem can use tools to increase productivity and quality.

Poor developers don’t invest the time or effort to understand how to code properly and avoid defects.  They spend their time learning how to use tools without understanding the purpose of the tool or how to use it effectively.

To some degree, this is partially the fault of the tool vendors.  The tool vendors perceive an opportunity to make $$$$$ based on providing support for a common problems, such as:

  • defect trackers to help you manage defect tracking
  • version control systems to manage source code changes
  • tools to support Agile development (Version One, JIRA)
  • debuggers to help you find defects

There are many tools out there, but let’s just go through this list and point out where developers and organizations get challenged.  Note, all statistics below are derived from over 15,000 projects over 40 years.1

Defect Trackers

Believe it or not, some organizations still don’t have defect tracking software. I’ve run into a couple of these companies and you would not believe why…

Inadequate defect tracking methods: productivity -15%, quality -21%

So we are pretty much all in agreement that we need to have defect tracking; we all know that the ability to manage more than a handful of defects is impossible without some kind of system.

Automated defect tracking tools: productivity +18%, quality +26%

The problem is that developers fight over which is the best defect tracking system. The real problem is that almost every defect tracking system is poorly set-up, leading to poor results. Virtually every defect tracking system when configured properly will yield tremendous benefits. The most common pitfalls are:

  • Introducing irrelevant attributes into the defect lifecycle status, i.e. creation of statuses like deferred, won’t fix, or functions as designed
  • Not being able to figure out if something is fixed or not
  • Not understanding who is responsible for addressing a defect

The tool vendors are happy to continue to provide new versions of defect trackers. However, using a defect tracker effectively has more to do with how the tool is used rather than which tool is selected.

One of the most fundamental issues that organizations wrestle with is what is a defect?  A defect only exists if the code does not behave according to specifications. But what if there are no specifications or the specifications are bad?  See It’s not a bug, it’s… for more information.

Smart organizations understand that the way in which the defect tracker is used will make the biggest difference.  Discover how to get more out of you defect tracking system in Bug Tracker Hell and How to Get Out.

Another common problem is that organizations try to manage enhancements and requirements in the defect tracking system.  After all whether it is a requirement or a defect it will lead to a code change, so why not put all the information into the defect tracker?  Learn why managing requirements and enhancements in the defect tracking system is foolish in Don’t manage enhancements in the bug tracker.

Version Control Systems

Like defect tracking systems most developers have learned that version control is a necessary hygiene procedure.  If you don’t have one then you are likely to catch a pretty serious disease (and at the least convenient time)

Inadequate change control: productivity -11%, quality -16%

Virtually all developers dislike version control systems and are quite vocal about what they can’t do with their version control system.  If you are the unfortunate person who made the final decision on which version control system is used just understand that their are hordes of developers out their cursing you behind your back.

Version control is simply chapter 1 of the story.  Understanding how to chunk code effectively, integrate with continuous build technology, and making sure that the defects in the defect tracker refers to the correct version are just as important as the choice of version control system.

Tools to support Agile

Sorry Version One and JIRA, the simple truth is that using an Agile tool does not make you agile, see this.

These tools are most effective when you actually understand Agile development. Enough said.

Debuggers

I have written extensively about why debuggers are not the best tools to track down defects.  So I’ll try a different approach here.

One of the most enduring sets of ratios in software engineering has been 1:10:100.  That is, if the cost of tracking down a defect pre-test (i.e. before QA) is 1, then it will cost 10x if the defect is found by QA, and 100x if the defect is discovered in deployment by your customers.

Most debuggers are invoked when the cost function is in the 10x or 100x part of the process.  As stated before, it is not that I do not believe in debuggers — I simply believe in using pre-test defect removal strategies because they cost less and lead to higher code quality.

Pre-test defect removal strategies include:

  • Planning code, i.e. PSP
  • Test driven development, TDD
  • Design by Contract (DbC)
  • Code inspections
  • Pair programming for complex sections of code

You can find more information about this in:

Seldom Used Tools

Tools that can make a big difference but many developers don’t use them:

Automated static analysis: productivity +21%, quality +31%

Automated unit testing: productivity +17%, quality +24%

Automated unit testing generally involves using test driven development (TDD) or data driven development together with continual build technology.

Automated sizing in function points: productivity +17%, quality +24%

Automated quality and risk prediction: productivity +16%, quality +23%

Automated test coverage analysis: productivity +15%, quality +21%

Automated deployment support: productivity +15%, quality +20%

Automated cyclomatic complexity computation: productivity +15%, quality +20%

Important Techniques with No Tools

There are a number of techniques available in software development that tool vendors have not found a way to monetize on. These techniques tend to be overlooked by most developers, even though they can make a huge difference in productivity and quality.

The Personal Software Process and Team Software Process were developed by Watts Humphrey, one of the pioneers of building quality software.

Personal software process: productivity +21%, quality +31%2

Team software process: productivity +21%, quality +31%3

The importance of inspections is covered in:

Code inspections: productivity +21%, quality +31%4

Requirement inspections: productivity +18%, quality +27%4

Formal test plans: productivity +17%, quality +24%

Function point analysis (IFPUG): productivity +16%, quality +22%

Conclusion

There is definitely a large set of developers that assume that using a tool makes them competent.

The reality is that learning a tool without learning the principles that underly the problem you are solving is like assuming you can beat Michael Jordan at basketball just because you have great running shoes.

Learning tools is not a substitute for learning how do do something competently. Competent developers are continually learning about techniques that lead to higher productivity and quality, whether or not that technique is supported by a tool.

References

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

The Programmer Productivity Paradox

Posted on by
Reply

Programmers seem to be fairly productive people.

You always see them typing at their desks; they chafe for meetings to finish so that they can go back to their desks and code. When asked, they will say that there is not enough time to produce the code, and the sooner they can start coding, the sooner they will be done.

So writing code must be the most important thing, correct?

If the average programmer writes about 50 lines of production code a day.  A 50,000 line program would take 1,000 man days to produce.  The 50,000 line listing can be entered by a programmer at about 1,000 lines a day or about 50 man days.

So what the heck are the developers doing for the other 950 days?

Before addressing that issue, lets make a simple observation. Capers Jones has compared many methodologies (RUP, XP, Agile, Waterfall, etc) and programming languages over thousands of projects and determined that programmers write between 325 and 750 lines of code (LOC) per month, which is less than the 1,000 LOC per month suggested above1.  Even if programmers do not average 50 lines of code per day, the following is clear2.

  • Methodology does not explain the apparent productivity gap
  • No language accounts for the apparent productivity gap

The reality is that only a fraction of a developer’s time is actually spent writing production code. If a developer is typing in code all the time then they are really trying different combinations of code until they finally find the combination of code that works.

Or more correctly, the combination that seems to match the requirements until either QA or the business analyst comes back and lets them know there is a problem.

That is why developers that plan their code before using the keyboard tend to outperform other developers. Not only do only a few developers really plan out their code before coding but also years of experience do not teach developers to learn to plan.  In fact studies over 40 years show that developer productivity does not change with years of experience. (see No Experience Required!)

Years of experience do not lead to higher productivity

Interestingly enough, there are methodologies that have been around for a long time that emphasize planning code.  Watts Humphrey is the created of the Personal Software Process (PSP)3.  Using PSP has been measured to:

PSP can raise productivity by 21.2% and quality by 31.2%

If you are interested there are many other proven methods of raising code quality that are not commonly used (see Not Planning is for Losers).

If your developers at their keyboard and not planning at a white board then odds are that your productivity is not as high as it could be.

Bibliography

1 The The Mythical Man Month is even more pessimistic suggesting that programmers produce 10 production lines of code per day
2 Jones, Capers and Bonsignour, Olivier.  The Economics of Software Quality.  Addison Wesley.  2011
3 Watts, Humphrey.  Introduction to the Personal Software Process, Addison Wesley Longman. 1997

Other articles in the “Loser” series

Want to see more sacred cows get tipped? Check out:
Moo?

Make no mistake, I am the biggest “Loser” of them all.  I believe that I have made every mistake in the book at least once 🙂

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)