The Programmer Productivity Paradox

Posted on May 13, 2014 by Dalip Mahal

Programmers seem to be fairly productive people.

You always see them typing at their desks; they chafe for meetings to finish so that they can go back to their desks and code. When asked, they will say that there is not enough time to produce the code, and the sooner they can start coding, the sooner they will be done.

So writing code must be the most important thing, correct?

If the average programmer writes about 50 lines of production code a day. A 50,000 line program would take 1,000 man days to produce. The 50,000 line listing can be entered by a programmer at about 1,000 lines a day or about 50 man days.

So what the heck are the developers doing for the other 950 days?

Before addressing that issue, lets make a simple observation. Capers Jones has compared many methodologies (RUP, XP, Agile, Waterfall, etc) and programming languages over thousands of projects and determined that programmers write between 325 and 750 lines of code (LOC) per month, which is less than the 1,000 LOC per month suggested above¹. Even if programmers do not average 50 lines of code per day, the following is clear².

Methodology does not explain the apparent productivity gap
No language accounts for the apparent productivity gap

The reality is that only a fraction of a developer’s time is actually spent writing production code. If a developer is typing in code all the time then they are really trying different combinations of code until they finally find the combination of code that works.

Or more correctly, the combination that seems to match the requirements until either QA or the business analyst comes back and lets them know there is a problem.

That is why developers that plan their code before using the keyboard tend to outperform other developers. Not only do only a few developers really plan out their code before coding but also years of experience do not teach developers to learn to plan. In fact studies over 40 years show that developer productivity does not change with years of experience. (see No Experience Required!)

Years of experience do not lead to higher productivity

Interestingly enough, there are methodologies that have been around for a long time that emphasize planning code. Watts Humphrey is the created of the Personal Software Process (PSP)³. Using PSP has been measured to:

PSP can raise productivity by 21.2% and quality by 31.2%

If you are interested there are many other proven methods of raising code quality that are not commonly used (see Not Planning is for Losers).

If your developers at their keyboard and not planning at a white board then odds are that your productivity is not as high as it could be.

Bibliography

¹	The The Mythical Man Month is even more pessimistic suggesting that programmers produce 10 production lines of code per day
²	Jones, Capers and Bonsignour, Olivier. The Economics of Software Quality. Addison Wesley. 2011
³	Watts, Humphrey. Introduction to the Personal Software Process, Addison Wesley Longman. 1997

What the Heck are Non-Functional Requirements?

Posted on December 2, 2013 by Dalip Mahal

What the Heck are Non-Functional Requirements? Simply put, if functional requirements create code that will address the needs of the end-users (customers), then non-functional requirements address the needs of the people who install, operate, and configure the code.

Those people are the operations personnel and help desk personnel in whatever organization that uses your software. Every developer needs to be aware of what those non-functional requirements are and why operations personnel and help desk personnel are customers that are just as important as the end-users.

Functional Requirements

Functional requirements are baked into the code that developer’s deliver (interpreted or compiled). Events from input devices (network, keyboard, devices) trigger functions to convert input into output — all functions have the form:

This is true whether you use an object-oriented language or not. Non-functional requirements involve everything that surrounds a functional code unit. Non-functional requirements concern things that involve time, memory, access, and location:

Performance
Availability
Capacity
Continuity
Security

Non-functional requirements are slightly different between desktop applications and services; this article is focused on non-functional requirements for services.

If you have any knowledge of ITIL you will recognize that the last 4 items deal with the warranty of a service. In fact, the functional requirements involve the utility of a service, the non-functional requirements involve the warranty of a service.

Availability

Availability is about making sure that a service is available when it is supposed to be available. Availability is about a Configuration Item (CI) in the environment of the operations center that specifies how the code is accessed. Availability is decided independently of the code and is at best part of the Service Design Package (SDP) that is delivered to the operations department, at worst it is simply code dumped on the operations personnel.

Developer’s need to be aware of the difficulty of creating the CI for the operations personnel. If a CI is manually created then there will always be a potential for an error when the service is installed or updated. The requirement to create a CI is a non-functional requirement and the ability to minimize errors is another non-functional requirement.

Developer’s need to be aware of single-points of failure (i.e. services hard-coded to a specific IP) which causes fits in operations that are not running virtual machines (VM) that can have virtual IPs . The requirement to create code that is not reliant on static IPs or specific machines is a non-functional requirement. Availability is simplified in operations if the code is resilient enough to allow itself to easily move (or be replicated) among servers.

Availability non-functional requirements include:

Ability to easily make the CI
Automatic installation of CI or mechanisms
Ability to detect and prevent manual errors for a CI
Ability to easily move code between servers

Capacity

Capacity is about delivering enough functionality when required. If you ask a web service to supply 1,000 requests a second when that server is only capable of 100 requests a second then some requests will get dropped. This may look like an availability issue, but it is caused because you can’t handle the capacity requested.

Internet services almost always can’t provide enough capacity with a single machine and operations personnel need to be able to run multiple servers with the same software to meet capacity requirements. The ability to run multiple servers without conflicts is a non-functional requirement. The ability to take a failing node and restart it on another machine or VM is a non-functional requirement.

Capacity non-functional requirements include:

Ability to run multiple instances of code easily
Ability to easily move a running code instance to another server

Continuity

Continuity involves being able to be robust against major interruptions to a service, these include power outages, floods or fires in an operational center, or any other disaster that can disrupt the network or physical machines.

Where availability and capacity often involve redundancy inside a single operation center, continuity involves geographic and network redundancy. Continuity at best involves having multiple servers that can work in geographically distributed operation centers. At worst, you need to be able to have a master-slave fail over model with the ability to journal transactions and eventually bring the master back up.

Security

Security non-functional requirements concern who has access to functions and preventing the integrity of data from being corrupted.

Where access is concerned, how difficult will it be for operations personnel or help desks to set up security for users? Developer’s build in different levels of access into their applications without considering how difficult it will be for a 3rd party (help desk or operations) to set-up end users. The ease of setting up security is a non-functional requirement.

Data integrity is another non-functional requirement. Developer’s need to consider how their applications will behave if the program encounters corrupted data due to machine or network failures. This is not as important an issue in environments using RAID or redundant databases.

What Happens When You Forget Non-Functional Requirements

Commonly start-ups are so busy setting up their services that the put non-functional requirements on the back burner. The problem is that there are non-functional requirements that need to be designed into the architecture when software is created.

For example, it is easy to be fooled into building software that is tied to a single machine, however, this will not scale in operations and cause problems later on. One of the start-ups I was with built a server for processing credit/debit card transactions without considering non-functional requirements (capacity, continuity). It cost more to add the non-functional requirements than it cost to develop the software!

Every non-functional requirement that is not thought through at the inception of a project will often represent significant work to add later on. Every such project is a 0 function point project that will require non zero cost!

Generally availability, capacity, and continuity is not a problem for services developed with cloud computing in mind. However, there are thousands of legacy services that were developed before cloud computing was even possible.

If you are developing a new service then make sure it is cloud enabled!

Operations People are People Too

Make no mistake, operations and help desk personnel are fairly resourceful and have learned how to manage software where non-functional requirements are not handled by the code. Hardware and OS solutions exist for making up for poorly written software that assumes single machines or does not take into account the environment that the code is running in, but that can come at a fairly steep cost in infrastructure.

The world has moved to services and it is no longer possible for developers to ignore the non-functional requirements involved with the code that they are developing. Developer’s that think through the non-functional requirements can reduce costs dramatically on the bottom line and quality of service being delivered.

The guys that run operational centers and help desks are customers that are only slightly less important than the end-user. Early consideration of the non-functional requirements makes their lives easier and makes it much easier to sell your software/services.

It is no longer possible for competent developers to be unaware of non-functional requirements.

Who should set defect priority?

Posted on November 19, 2013 by Dalip Mahal

Surprisingly, defect priority should not be set by QA. QA are generally the owners of the defect tracking system and control it, but this is one attribute that they should not control. The defect tracker is a shared resource between QA, engineering, engineering management, and product mangement and is a coordinating mechanism for all these parties.

Commonly people mix up priority and severity, for example, there may be a severe defect that causes the software either not to install or to cease functioning.

It is common for new releases to have various installation problems when it initially gets to QA. This blocks QA so they mark the defects with a high severity and high priority. This issue has a high severity and needs to be addressed right away, but remember bug tracking systems are append only — once this defect gets into the system, it will never get out. This kind of issue should be escalated to the engineers and engineering management because it makes little sense to clog the defect tracking system with it.

Now there may be intermittent issues that cause the software to fail and you may assume that this defect would be high priority, but if this defect occurs very rarely and would cost too much to fix then this defect may be a low priority. Once again the priority of an intermittent severe issue can not be determined by QA.

Similarly, there may be many cosmetic or minor defects where fixing them might make a huge difference in the user experience and reduce support calls. Even though these defects are minor, they may be easy to fix and save you serious money. Once again, this can not be decided by QA.

Therefore, QA should reserve the right to set the initial severity, and may have an internal field for QA priority, but the priority of a defect should be determined by a product manager (PM) who has a more complete understanding of the overall context of the product. The priority of a defect is a business issue, not an engineering or QA issue.

Ideally, the product manger will set the priority of a defect during a defect triage session where representatives from engineering, engineering management, and QA are present. As you go through each new defect each person can present their logic for what the priority should be. The PM should then set the priority defect and assign the release it should be fixed in (i.e. this release, minor release, major release, won’t fix). Ideally there are extra fields in the defect record for both QA and development to put their priority beliefs.

It is important that the status of a defect not include any of the following or your defect tracker will go to hell (see Bug Tracker Hell and How to Get Out):

Priority
Severity
Fix version

It is very hard to generate meaningful reports when these attributes creep into the defect status. You know that this has happened if you have a multi-page training manual for entering defects into the system. Some of the statuses that make sense initially but turn the defect system into a nightmare are:

FAD (functions as designed)
WontFix
NextVersion
CantReproduce

Of course, if you don’t have regular bug triage sessions with all the parties mentioned above then you are probably sand-bagged in fire-fighting.

Enhancements don’t belong in the bug tracker

Posted on October 26, 2013 by Dalip Mahal

As development progresses we inevitably run into functionality gaps that are either deemed as enhancements.

These issues often get captured by QA in the bug tracker and assigned to a developer.

Enhancements should not be managed from the bug tracker

The life cycle of a defect and the life-cycle of a enhancement are two entirely different things. A defect is a difference between a stated requirement and the code. If there is no documentation there is no code defect (see It’s not a bug, it’s…) — in fact, most enhancements will eventually be coded by some developer; they just should not be managed from the bug tracker.

Defect Life-cycle

The defect life-cycle is well known:

Defect is identified as a departure from the requirements
Defect is assigned to a developer
Defect is corrected
Correct is verified
- If not corrected re-open and re-assign to developer
The defect is closed

This is the incorrect way to manage enhancements. When a functionality gap is determined by QA and it is not covered by the requirements then we have an issue. It is rarely the case that the issue can be resolved by the developer.

Enhancement Life-cycle

If enhancements are assigned to a developer then they are likely to try to resolve the issue. The problem is that “enhancements” determined at the QA level may be phantom problems caused by either:

Insufficient requirements
Correct requirements but incorrect test plans

Enhancements may or may not become code changes. Even when enhancements turn into code change requests they will generally not be implemented as the developer or QA think they should be implemented.

Enhancements are really requirement defects. Enhancements should be logged as such in the bug tracker and assigned to the person in charge of requirements (business analyst or product manager). Those individuals should be responsible to track down how these issues should be handled.

If the requirements are correct and the test plans are defective then it should be logged as a test defect. This is tricky because QA often controls the bug tracker and will not log errors that they have made.

At a minimum, the implementation of requirements and test defects can do several positive things for you:

It removes the responsibility to find a solution from development.
It makes it clear how many defects are in the requirements or test plans.
It reduces stress; no developer wants to be blamed for an issue that is not his.
Many enhancements call for updated project plans and pushing back the deadline.

Put Responsibility Where it Belongs

The creation of requirements and test defects in the bug tracker goes a long way to cleaning up the bug tracker. In fact, requirements and test defects represent about 25% of defects in most systems (see Bug Tracker Hell and How to Get Out!). The percentages break down as follows:

Requirements defects: 9.58%
Testing defects: 15.42%

The creation of requirement and test defects in the bug-tracker alleviate pressure on the engineering department and redirect it to either the product manager or QA. Eventually enough data will accumulate in the bug-tracker to get management’s attention.

At a minimum, these categories should help reduce the amount of fire-fighting in late projects (Root cause of ‘Fire-Fighting’ in Software Projects)

Rating: 0.0/5 (0 votes cast)

Rating: 0 (from 0 votes)

It’s not a bug, it’s…

Posted on October 22, 2013 by Dalip Mahal

When does a bug become a bug?
Who decides that it is a bug?

How many legs does a lamb have if I say the tail is a leg? The answer is 4, just because I say the tail is a leg does not make it a leg!

Bugs should be obvious, but we say It’s not a bug, it’s a feature because often it isn’t obvious. Watson Humphrey felt that we should use the term defect and not bug because most people don’t take bugs seriously, so let’s use the term defect instead.

So when does a defect become a defect?

When quality assurance tells you that you have a defect?
When product management says that it is a defect?
When the customer says that it is a defect?

The answer is: none of the above.

Now it might turn out that there is a problem and that code needs to change, but a defect only exists if:

code behaves differently than the requirements specification

This is important because most systems are under specified (if they are specified at all) and so when code misbehaves it is only a defect if the code behavior differs from the specification. We call defects undocumented features because we know that the problem is that the requirements were never written.

Incomplete and Inconsistent Requirements

Many organizations do not create sufficiently complete requirements before starting development, either because they don’t know how to capture requirements properly or because they don’t have resources capable of capturing complete requirements. Incomplete (and inconsistent) requirements and unrealistic deadlines often force developers into making decisions about how to implement features. The end result is that developers are regularly told that they have defects in their code.

While this process is common, it is destructive. When requirements are under specified and inconsistent developers end up needing to perform serious rework. The rework will can require dramatic changes that will impact the architecture of the code.

The time required to find a work around (if it is possible) is rarely included in the project plan. Complicating matters is that the organizations that are reluctant to spend time creating requirements also tend to underestimate their projects. This puts tremendous pressure on the engineering department to deliver; this promotes the 5 worst practices in software development (see Stop It! No… really stop it.)

Only 54% of Issues are Resolved by Engineers

The attitude that all defects must be resolved by the engineering department is severely misguided. Analysis by Capers Jones of over 18,000+ projects shows that only about 54% of all defects can be resolved by the engineers! (only the 3 highlighted rows below)

Defect Role Category	Frequency	Role
Requirements defect	9.58%	BA/Product Management
Architecture or design defect	14.58%	Architect
Code defect	16.67%	Developer
Testing defect	15.42%	Quality Assurance
Documentation defect	6.25%	Technical Writer
Database defect	22.92%	Data base administrator
Website defect	14.58%	Operations/Webmaster

This means that precious time will be wasted assigning issues to developers that they can not resolve. The time necessary to redirect the issue to the correct person is a major contributing factor to fire-fighting

Getting Control of the Defect Process

For most organizations fixing the defect process involves understanding and categorizing defects correctly. Organizations that are not tracking the different sources of defects probably have a bug tracker that has gone to hell. Here is how you can fix that problem, see Bug Tracker Hell and How To Get Out!

At a minimum you need to implement the requirements defect, once you identify issues that are caused by poor requirements it will shine the white hot light of shame onto the resources that are capturing your requirements. Once you realize how many requirements defects exist in your system you can begin to inform senior management about the requirements problem.

Reducing Fire-Fighting

The best way to reduce fire fighting is to start writing better requirements (or writing requirements 🙂 ). To do so you need to figure out which of the following are broken:

Not enough time is allocated to the requirements phase
Unskilled people are capturing your requirements

In all likelihood both of these issues need to be fixed in your organization. When requirements are incomplete and inconsistent you will have endless fire-fighting meetings involving everyone (see Root cause of ‘Fire-Fighting’ in Software Projects)

Stand your ground if someone tells you that you have coded a defect when there is no documentation for the requirement.

Rating: 0.0/5 (0 votes cast)

Rating: 0 (from 0 votes)

Size Matters

Posted on September 30, 2013 by Dalip Mahal

Some say that software development is challenging because of complexity. This might be true, but this definition does not help us find solutions to reduce complexity. We need a better way to explain complexity to non-technical people.

The reality is that when coding a project size matters. Size is measured in the number of pathways through a code base, not by the number of lines of code. Size is proportional to the number of function points in a project.

There are many IT people that succeed with programs of a certain size and then fail miserably when taking on programs that are more sophisticated. Complexity increases with size because the number of pathways increase exponentially in large programs.

Virtually anyone (even CEOs 🙂 ) can build a hello, world! application; an application that only has a single pathway through it and is as simple as you can get. Some CEOs write the simple hello, world! program and incorrectly convince themselves that development is easy. Hello, world! only has a single pathway through it and virtually anyone can write it.

main() {
printf( “hello, world” );
}

If you have an executive that can’t even complete hello,world then you should take away his computer 🙂

Complexity Defined

As programs get more sophisticated, the number of decisions that have to be made increase and the depth of the call tree increases. Every non-trivial routine will have multiple pathways through it.

If your average call depth is 10 with an average of 4 pathways through each routine then this represents over 1 million pathways. If the average call depth is 15 then it represents 107 million pathways.

Increasing sophisticated programs have greater call depth than ever and distributed applications increase the call depth even because the call depth of a system is additive. This is what we mean by complexity; it is impossible for us to test all of the different pathways in a black box fashion.

Now in reality every combination of pathways is not possible, but you only have to leave holes in a few routines and you will have hundreds, if not thousands, of pathways where calculations and decisions can go wrong.

In addition, incorrect calculations or decisions higher up in the call tree can lead to difficult to find defects that may blow up much further away from the source of the problem.

What are Defects?

Software defects occur for very simple reasons, an incorrect calculation is performed that causes an output value to be incorrect. Sometimes there is no calculation at all because input data is not validated to be consistent and that data is either stored incorrectly or goes on to cause incorrect calculations to be performed.

We only recognize that we have a defect when we see an output value and recognize that it is incorrect. More likely QA sees it and tells us that we are incorrect.

Basically we follow a pathway that is correct through nodes 1, 2, 3, 4, and 5. At point 6 we make a miscalculation calculation, and then we have the incorrect values at points 7 and 8 and discover the problem at node 9.

So once we have a miscalculation, we will either continue to make incorrect calculations or make incorrect decisions and go down the wrong pathways (where we will then make incorrect calculations).

Not all Defects are Equal

It is clear that the more distance there is between a miscalculation and its discover will make defects harder to detect. The longer the call depth the greater the chance that there can be a large distance between the origin and detection, in other words:

Size Matters

Today we build sophisticated systems of many cooperating applications and the call depth is exponential with the size of the system. This is what we mean by complexity in software.

Reducing Complexity

Complexity is reduced for every function where:

You can identify when inconsistent parameters are passed to a function
All calculations inside of a function are done correctly
All decisions through the code are taken correctly

The best way to solve all 3 issues is through formal planning and development.Two methodologies that focus directly on planning at the personal and team level are the Personal Software Process (PSP) and the Team Software Process (TSP) invented by Watts Humphrey.

Identifying inconsistent parameters is easiest when you use Design By Contract (DbC) , a technique that was pioneered by the Eiffel programming language. It is important to use DbC on all functions that are in the core pathways of an application.

Using Test Driven Development is a sure way to make sure that all calculations inside of a function are done correctly, but only if you write tests for every pathway through a function.

Making sure that all calculations are done correctly inside a function and that correct decisions are make through the code is best done through through code inspections (see Inspections are not Optional and Software Professionals do Inspections).

All techniques that can be used to reduce complexity and prove the correctness of your program are covered in Debuggers are for Losers. N.B. Debuggers as the only formalism will only work well for systems with low call depth and low branching.

Conclusion

Therefore, complexity in software development is about making sure that all the code pathways are accounted for. In increasingly sophisticated software systems the number of code pathways increases exponentially with the call depth. Using formal methods is the only way to account for all the pathways in a sophisticated program; otherwise the number of defects will multiply exponentially and cause your project to fail.

Only projects with low complexity (i.e. small call depth) can afford to be informal and only use debuggers to get control of the system pathways. As a system gets larger only the use of formal mechanisms can reduce complexity and develop sophisticated systems. Those formal mechanisms include:

Personal Software Process and Team Software Process
Design by Contract (via Aspect Oriented Programming)
Test Driven Development
Code and Design Inspections

Rating: 0.0/5 (0 votes cast)

Rating: 0 (from 0 votes)

Are we there yet?

Posted on September 18, 2013 by Dalip Mahal

We associate “are we there yet?” with kids asking incessantly if a long trip is almost over. It is generally funny, however, it is less funny on a project that should be complete.

Projects follow distinct phases:

Basic requirements are collected
Project plan and end date are established
Development starts
Projects track to the project plan

Often, tasks start off well until they all level off at 90-95% complete and get stuck. Management was satisfied with progress until progress stall, they see frantic activity picking up and they wonder “are we there yet?.”

Like a family vacation, this trip is sometimes not even close to being finished. You expect that if 9,000 hours have been spent on a project estimated at 10,000 hours that you would be 90% done. Surprisingly this is not the case for 7 projects out of 10 — have you ever worked on a project where a 90% project plan meant the project was 90% done?

Using a project plan to estimate completion is useful if there is a direct correlation between the project goal and the project plan. You often discover that the goal and the plan differ late in a project. Project plans and results differ because

Requirements and tasks are missing
The project is incorrectly estimated
Work is performed on tasks that do not advance the project

Requirements and Tasks are Missing

Clearly missing requirements mean that more work will be necessary to get to the goal, but the time for this work is rarely added to the project deadline.

A relative of missing requirements is missing tasks, this occurs when the work breakdown structure is incomplete and more subtasks are necessary to complete a task than estimated.

In both cases, if there are 2,000 hours of missing requirements and tasks then a project initially forecasted for 10,000 hours should move the deadline to 11,000 hours. Therefore if 9,000 hours are done then you are only 75% complete.

Unfortunately, weak IT leadership, internal politics, and embarrassment over poor estimates will not move the deadline and teams will have pressure put on them by overbearing senior executives to get to the original deadline even though that is not possible.

The Project is Incorrectly Estimated

There is much literature about how accurate estimates are possible and necessary to successful projects. Weak and uniformed IT leadership will cave in to senior management demands for project deadlines without formal estimates. A typical interaction looks as follows:

CEO: We need feature X, how much will it cost and how long will it take to built?

VP Engineering: Well we need to define feature X properly, see how it will be implemented, determine if we have the necessary skill sets, and see what the impact to our other operations will be. It will take time to do do this work.

CEO: We don’t have time for formal estimates. How hard can it be to add feature X? But next Friday, I will need a ballpark estimate for time and cost.

VP Engineering: I’ll see what I can come up with for next week.

Weak IT executives allow themselves to be bullied all the time by other executives that have no idea what is involved in IT projects. The end result is an underspecified project that will be underestimated in time and cost (see Why Executive Declared Deadlines lead to Disaster)

The more inaccurate the requirements the more extra work there is to do to get to the target. That is why short requirements processes lead to strongly shifting requirements and cancelled projects (see Shift Happens). In fact, the degree of requirements shift is equal to the chance of a project being cancelled.

Work is Performed that Does Not Advance the Project

Even if a project is correctly specified, there are several activities that will be performed that will not advance the project:

Some requirements can not be implemented as specified and time will be spent researching and implementing work arounds
Some requirements will be ambiguously specified and be implemented incorrectly and need to be redone
Some requirements will be inconsistent and require time and analysis to establish consistent requirements
Infrastructure might need to be refactored when you discover that it will not support the code created later

Rework Work executed on these activities will not advance your project and should not be counted in the total of completed hours. So if 2,000 hours have been spent on activities that don’t advance the project then if 9,000 hours have been done on a 10,000 hour project then you have really done 7,000 hours of the 10,000 hour project and you are only 70% done.

Not Changing the Deadline Leads to Bad Practices

Whether a project is off course because of missing activities or non-productive activities, not changing the project end date will lead to schedule pressure as the project advances and people slowly start to realize that you are not going to make it.

When it becomes clear that a project can’t make it’s original deadline many organizations will start common but deadly practices. Excessive schedule pressure often leads to the following bad practices (see Stop It! No… Really stop it.):

Friction within the team
Friction amongst the managers
Inadequate communication with stakeholders
Layoffs of key personnel

Solutions

There are only a few cures for the ‘Are we there yet?‘ problem:

IT management with the intestinal fortitude to hold out for the creation of proper work breakdown structures and formal estimates
Proper requirement processes that yield complete, consistent, and concise requirements
Proper change management processes to alter the project deadline when missing tasks and non-productive activities are encountered

Failure to comply with these 3 principles means that you will continue to be subject to chaotic environments where 7 out of 10 projects fail (see Executives: Understanding your Chances of a Successful Project)

Rating: 0.0/5 (0 votes cast)

Rating: 0 (from 0 votes)

Defects are for Losers

Posted on April 25, 2013 by Dalip Mahal

A developer is responsible for using any and all techniques to make sure that he produces defect free code. The average developer does not take advantage of all of the following opportunities to prevent and eliminate defects:

Before the code is written
As the code is written
Writing mechanisms for early detection
Before the code is executed
After the code is tested

The technique that is used most often is #5 above and will not be covered here. It involves the following:

Code is delivered to the test department
The test department identifies defects and notifies development
Developer’s fire up the debugger and try to chase down the defect

Like the ‘rinse and repeat‘ process on a shampoo bottle, this process is repeated until the code is cleaned or until you run out of time and are forced to deliver.

The almost ubiquitous use of #5 leads to CIOs and VPs of Engineering assuming that the metric of one tester to two developers is a good thing. Before assuming that #5 is ‘the way to go‘ consider the other techniques and statistical evidence of their effectiveness.

Before the Code is Written

A developer has the most options available to him before the code is written. The developer has an opportunity to plan his code, however, there are many developers who just ‘start coding’ on the assumption that they can fix it later.

How much of an effect can planning have? Two methodologies that focus directly on planning at the personal and team level are the Personal Software Process (PSP) and the Team Software Process (TSP) invented by Watts Humphrey.

PSP can raise productivity by 21.2% and quality by 31.2%

TSP can raise productivity by 20.9% and quality by 30.9%

Not only does the PSP focus on code planning, it also makes developers aware of how many defects they actually create. Here are two graphs that show the same group of developers and their defect injection rates before and after PSP training.


Before PSP training	After PSP training

The other planning techniques are:

Decision tables
Proper use of exceptions

Both are covered in the article Debuggers are for Losers and will not be covered here.

As the Code is Written

Many developers today use advanced IDEs to avoid common syntax errors from occurring If you can not use such an IDE or the IDE does not provide that service then some of the techniques in the PSP can be used to track your injection of syntax errors and reduce those errors.

Pair Programming

One technique that can be used while code is being written is Pair Programming. Pair programming is heavily used in eXtreme Programing (XP). Pair programming not only allows code to be reviewed by a peer right away but also makes sure that there are two people who understand the code pathways through any section of code.

Pair programming is not cost effective overall (see Capers Jones). For example, it makes little sense to pair program code that is mainly boiler plate, i.e. getter and setter classes. What does make sense is that during code planning it will become clear which routines are more involved and which ones are not. If the cyclomatic complexity of a routine is high (>15) then it makes sense for pair programming to be used.

If used for all development, Pair Programming can raise productivity by 2.7% and quality by 4.5%

Test Driven Development

Test driven development (TDD) is advocated by Kent Beck and stated in 2003 that TDD encourages simple designs and inspires confidence. TDD fits into the category of automated unit testing.

Automated unit testing can raise productivity by 16.5% and quality by 23.7%

Writing Mechanisms for Early Detection

Defects are caused by programs either computing wrong values, going down the wrong pathway, or both. The nature of defects is that they tend to cascade and get bigger the further in time and space between the source of the defect and the noticeable effects of the defect.

Design By Contract

One way to build checkpoints into code is to use Design By Contract (DbC), a technique that was pioneered by the Eiffel programming language It would be tedious and overkill to use DbC in every routine in a program, however, there are key points in every software program that get used very frequently.

Just like the roads that we use have highways, secondary roads, and tertiary roads — DbC can be used on those highways and secondary roads to catch incorrect conditions and stop defects from being detected far away from the source of the problem.

Clearly very few of us program in Eiffel. If you have access to Aspect Oriented Programming (AOP) then you can implement DbC via AOP. Today there are AOP implementations as a language extension or as a library for many current languages (Java, .NET, C++, PHP, Perl, Python, Ruby, etc).

Before the Code is Executed

Static Analysis

Most programming languages out there lend themselves to static analysis. There are cost effective static analysis for virtually every language.

Automated static analysis can raise productivity by 20.9% and quality by 30.9%

Inspections

Of all the techniques mentioned above, the most potent pre-debugger technique is inspections. inspections are not sexy and they are very low tech, but the result of organizations that do software inspections borders on miraculous.The power of software inspections can be seen in these two articles:

Code inspections can raise productivity by 20.8% and quality by 30.8%

Design inspections can raise productivity by 16.9% and quality by 24.7%

From the Software Inspections book on p.22.

In one large IBM project, one half million lines of networked operating system, there were 11 development stages (document types: logic, test, user documentation) being Inspected. The normal expectation at IBM, at that time, was that they would be happy only to experience about 800 defects in trial site operation. They did in fact experience only 8 field trial defects.

Evidence suggests that every 1 hour of code inspection will reduce testing time by 4 hours

Conclusion

Overworked developers rarely have time to do research, even though it is clear that there is a wealth of information available on how to prevent and eliminate defects. The bottom line is that if your are only using technique #5 from the initial list, then you are not using every technique available to you to go after defects.My opinion only, but:

A professional software developer uses every technique at his disposal to prevent and eliminate defects

NO Experience Necessary!!!

Posted on March 19, 2013 by Dalip Mahal

Did you know that we have never found a relationship between a developer’s years of experience and code quality or productivity?

The original study that found huge variations in individual programming productivity was conducted in the late 1960s by Sackman, Erikson, and Grant (1968).

This study has been repeated at least 8 times over 30 years and the results have not changed! (see below)

Sackman et al studied professional programmers with an average of 7 years’ experience and found that:

the ratio of initial coding time was about 20 to 1
the ratio of debugging times over 25 to 1
program execution speed about 10 to 1
program size 5 to 1

They found no relationship between a programmer’s number of years of experience and code quality or productivity. That is there was NO correlation between experience and productivity (i.e. ability to produce code) and there was NO correlation between experience and quality (i.e. minimizing defects) .

Think about that for a minute…

That is the worst programmers and the best programmers made distinct groups and each group had people of low and high experience levels. Whether training helps developers or not is not indicated by these findings, only that years of experience do not matter.

Without considering legality, this means that it is simpler to get rid of expensive poor performers with many years of experience and hire good performers with few years of experience!

Results Have Been Confirmed for 30 Years!

There were flaws in the study, however, even after accounting for the flaws, their data still shows more than an order of magnitude difference between the best programmers and the worst, and that difference was not related to experience. In years since the original study, the general finding that “There are order-of-magnitude differences among programmers” has been confirmed by many other studies of professional programmers (full references at the end of the article):

Curtis 1981
Mills 1983
DeMarco and Lister 1985
Curtis et al. 1986
Card 1987
Boehm and Papaccio 1988
Valett and McGarry 1989
Boehm et al 2000

Technology is More Sophisticated, Developers are not

You might think that we know much more about software development today than we knew in 1968, after all today:

we have better computer languages
we have more sophisticated technology
we have better research on effective software patterns
we have formal software degrees available in university

It turns out that all these things are true, but we still have order of magnitude differences among programmers and the difference is not related to years of experience. That means that there is some other x-factor that drives productive developers; that x-factor is probably the ability to plan and make good decisions.

The bad news is that if you are not a productive developer writing quality code then you will probably not get better simply because of years of experience.

Developers face making decisions on how to structure their code every day. There is always a choice when it comes to:

laying out code pathways
packaging functions into classes
packaging classes into packages/modules

Because developers face coding decisions, many of which are complex, the best developers will plan their work and make good decisions. Bad developers just ‘jump in’; they assume that they can always rewrite code or make up for bad decisions later. Bad developers are not even aware that their decision processes are poor and that they can become much better by planning their work.

Solution might be PSP and TSP

Watts Humphrey tried to get developers to understand the value of estimating, planning development, and making decisions in the Personal Software Process (PSP) for individuals and the Team Software Process (TSP) for teams, but only a handful of organizations have embraced it. Capers Jones has done analysis of over 18,000 projects and discovered that¹:

PSP can raise productivity by 21.2% and quality by 31.2%
TSP can raise productivity by 20.9% and quality by 30.9%

All of these findings should have a profound effect on the way that we build our teams. Rather than having large teams of mediocre developers, it makes much more sense to have smaller teams of highly productive developers that know how to plan and make good decisions. The PSP and TSP do suggest that the best way to rehabilitate a poor developer is to teach them how to make better decisions.

Be aware, there is a difference between knowledge of technologies which is gained over time and the ability to be productive and write quality code.

Conclusion

We inherently know this, we just don’t do it. If the senior management of organizations only knew about these papers, we could make sure that the productive people get paid what they are worth and the non-productive people could seek employment in some other field. This would not only reduce the cost of building software but also increase the quality of the software that is produced.

Unfortunately, we are doomed to religious battles where people debate methodologies, languages, and technologies in the foreseeable future. The way that most organizations develop code makes voodoo look like a science!

Eventually we’ll put the ‘science’ back in Computer Science, I just don’t know if it will be in my lifetime…

Check out Stop It! No… Really stop it. to learn about 5 worst practices that need to be stopped right now to improve productivity and quality.

Bibliography

Boehm, Barry W., and Philip N. Papaccio. 1988. “Understanding and Controlling Software Costs.” IEEE Transactions on Software Engineering SE-14, no. 10 (October): 1462-77.

Boehm, Barry, et al, 2000. Software Cost Estimation with Cocomo II, Boston, Mass.: Addison Wesley, 2000.

Card, David N. 1987. “A Software Technology Evaluation Program.” Information and Software Technology 29, no. 6 (July/August): 291-300.

Curtis, Bill. 1981. “Substantiating Programmer Variability.” Proceedings of the IEEE 69, no. 7: 846.

Curtis, Bill, et al. 1986. “Software Psychology: The Need for an Interdisciplinary Program.” Proceedings of the IEEE 74, no. 8: 1092-1106.

DeMarco, Tom, and Timothy Lister. 1985. “Programmer Performance and the Effects of the Workplace.” Proceedings of the 8th International Conference on Software Engineering. Washington, D.C.: IEEE Computer Society Press, 268-72.

¹Jones, Capers. SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.

Mills, Harlan D. 1983. Software Productivity. Boston, Mass.: Little, Brown.

Valett, J., and F. E. McGarry. 1989. “A Summary of Software Measurement Experiences in the Software Engineering Laboratory.” Journal of Systems and Software 9, no. 2 (February): 137-48.

Rating: 4.3/5 (6 votes cast)

Rating: +1 (from 1 vote)

Testing departments are for Losers

Posted on January 7, 2013 by Dalip Mahal

I understand that this is a very strong statement and I’m definitely not trying to insult anyone; so apologies in advance. I’m trying to challenge the belief that testing is mandatory and that there should be one testing resource for every two developers. Quality development is about quality assurance and zero defects not about having testing departments.

One testing resource for every two developers is not a good solution

Quality Assurance (QA) is about making a positive statement about product quality. QA is about positive assurance, which is stating, “We are certain that there are few, if any, defects in this product.” Airplanes are a product with quality assured, the manufacturers will stand by their quality and the statistics back them up. Contrast this with this article or this article which wonders what would happen if Microsoft made airplanes — would you fly in them?

The reality is that most testing departments simply discover defects and forward them back to the engineering department to fix. By the time the software product gets released we are basically saying, “We got rid of as many defects as we could find before management forced us to release this product, however, we really have no idea how many other defects are in the code”. This is not assuring quality; at best you get negative assurance out of this.

Everyone understand that buggy software kills sales (and start-ups :-)), however, testing is often an after thought in many organizations. When software products take longer than expected they are forwarded to the testing department. The testing department is often expected to test and bless code in less time than allocated.

To compound problems, many testing departments don’t even receive proper requirements against which to test the code and/or sufficient tools to work with. Large testing departments and/or large amounts of manual testing are not healthy or efficient.

Humphrey Watts was emphatic that calling defects “bugs” trivializes the issue and downplays the negative impact that defects cause on a development organization.

Calling defects “bugs” trivializes an important issue

Goblins and Elves

Defects are not introduced into software by goblins and elves. Defects are injected into the code by developers that:

don’t understand the requirements or architecture
misunderstand how to use their peer’s components
misunderstand 3rd party libraries
having a bad day because of home troubles or work environment
are careless because someone else will test their code

Defects are injected by the team

No one is more aware of how code can break down than the developer who writes it. Any line of code that is written without concentration and planning becomes a potential defect. It is impossible for testers to understand every pathway through the code and make sure that every possible combination of variables is properly taken care of.

There are many techniques that can increase code quality and dramatically reduce the amount of testing that is necessary:

test driven development (TDD)
database driven testing
design by contract (DbC)
inspections
pair programming
minimizing cyclomatic complexity
using static and dynamic code tools
proper code planning techniques

Test Driven Development

Properly written tests require a developer not only to think about what a code section is supposed to do but also plan how the code will be structured. If you know that there are five pathways through the code then you will write five tests ahead of time. A common problem is that you have coded n paths through the code when there are n+1 conditions.

TDD is white box testing and can reach every pathway that the developer codes. TDD is proactive and can test pathways from end to end, it does not just have to be used for unit testing. When TDD is hooked up to a continuous integration engine then defects are located and fixed before they make it to testing.

Database Driven Testing

Using actual test data to test existing routines during development is an excellent way to make sure that there are fewer production problems. The test data needs to be a copy (or subset) of production data.

Database driven testing can also be hooked up to a continuous integration engine and prevent defects from getting to testing.

Design By Contract

The Eiffel programming language introduced design by contract (DbC). DbC is orthogonal to TDD because its goal is to ensure that the contract defined by the preconditions and postconditions for each function call is not violated. DbC can be used in virtually any language for with their is an Aspect Oriented Programming (AOP) solution.

During development, the minute a developer violates the expected contract of any function (his or a peers) then the developer will get feedback to fix the problem before it gets to testing.

Inspections

Since the 1970s we have statistical evidence that one of the best ways to eliminate defects from code is through inspections. Inspections can be applied to the requirements, design, and code artifacts and projects that use inspections can eliminate 99% of the defects injected into the code. Se Inspections are not Optional and Software Professionals do Inspections.

Each hour of inspections will save you 4 hours of testing

Pair Programming

Pair programming can be selectively used to prevent and eliminate defects from code. When developers work in pairs they not only review code as quickly as possible but also learn productivity techniques from each other. Pair programming should only be done on complex sections of code.

Pair programming not only eliminates defects but allows developers to get enough feedback that they can prevent defects in the future.

Minimizing Cyclomatic Complexity

There is evidence that routines with high cyclomatic complexity will have more latent defects than other routines. This makes sense because the number of code pathways goes up dramatically as cyclomatic complexity increases and increases the chance that the developer does not handle all of them. In most cases, testing departments can not reproduce all of the pathways in routines of high cyclomatic complexity.

Use Dynamic and Static Code Checking

There are many code problems caused by a careless use of pointers and other powerful language constructs. Many of these problems can be detected by having the development team use dynamic and static code checking problems.

Proper Code Planning Techniques

There are developers that try to write code at the keyboard without planning, which is neither efficient nor effective. This is like having to do errands in 5 locations and driving to the locations randomly — you might get your errands done, but odds are it won’t be efficient.

Watts Humphrey talks directly to the idea of planning in the Personal Software Process. In addition techniques like diagramming with UML or using decision tables can go a long way to thinking through code structure before it is implemented.

Conclusion

Developers are the ones who inject defects into the code and therefore they are the best line of defense to remove them. The developer has the best information on what needs to be tested in his code at the time that he writes it. The longer it takes for testing or a customer to discover a code defect the longer the developer will spend in a debugger chasing down the problem.

Developers need to be trained in preventing and eliminating defects. Developers who learn to get the code correct the first time will reduce and eliminate effort in testing.

The goal of development should be to catch defects early; this is the only way to assure quality. Hence quality assurance starts and finishes in the development department, not the testing department.

Rating: 4.7/5 (3 votes cast)

Rating: -2 (from 2 votes)

Bibliography

Other articles in the “Loser” series

Functional Requirements

Availability

Capacity

Continuity

Security

What Happens When You Forget Non-Functional Requirements

Operations People are People Too

Defect Life-cycle

Enhancement Life-cycle

Put Responsibility Where it Belongs

Incomplete and Inconsistent Requirements

Only 54% of Issues are Resolved by Engineers

Getting Control of the Defect Process

Reducing Fire-Fighting

Complexity Defined

What are Defects?

Not all Defects are Equal

Reducing Complexity

Conclusion

Requirements and Tasks are Missing

The Project is Incorrectly Estimated

Work is Performed that Does Not Advance the Project

Not Changing the Deadline Leads to Bad Practices

Solutions

Before the Code is Written

As the Code is Written

Pair Programming

Test Driven Development

Writing Mechanisms for Early Detection

Design By Contract

Before the Code is Executed

Static Analysis

Inspections

Conclusion

Other articles in the “Loser” series

Results Have Been Confirmed for 30 Years!

Technology is More Sophisticated, Developers are not

Solution might be PSP and TSP

Conclusion

Bibliography

Test Driven Development

Database Driven Testing

Design By Contract

Inspections

Pair Programming

Minimizing Cyclomatic Complexity

Use Dynamic and Static Code Checking

Proper Code Planning Techniques

Conclusion