Estimation challenges, gotcha’s and expectations from developer

What is an estimate? (From Wikipedia)

Estimation (or estimating) is the process of finding an estimate, or approximation, which is a value that is usable for some purpose even if input data may be incomplete, uncertain, or unstable.

Estimate in software development terms (From Wikipedia)

Software development effort estimation is the process of predicting the most realistic amount of effort (expressed in terms of person-hours or money) required to develop or maintain software based on incomplete, uncertain and noisy input.

In essence, estimation is the essential crux of software development, which is required for following purposes –

Evaluation of the total cost of the project
Decide on number of engineers (developers, testers and/or business analysts) required on the project
Decide tentative release date of the product
Decide tentative release date to deliver a feature (or set of features) into existing product

Estimates can be broadly categorized like below and other variants also exists –

Estimate based on Work breakdown structure
Guestimate
Predictions and projections
Estimate based on past history of the project and team strength

Estimation: Probability vs Actual Time

My experience and observations

When I am asked to give estimate on behalf of team, I feel that it is injustice to the team, why?

Here are some facts –

On one particular task, time taken by one year experience developer is different than that of 8 year experienced developer. Even same is true for two equally experienced developers
A new developer when joins existing project he must get familiar with the codebase first to get productive
I can’t control dependencies on other teams or their delivery schedule
Initially there is lack of clarity on requirements front
When project is going on from years and uses legacy and unsupported products, it is a challenge to estimate and find a developer to work on
Generally it is difficult to break down tasks at the minute level at the start of the project
I don’t know the technology that may be used in the project (I may know, however what if my team has not yet formed or team members have only specific skill set that is useless for the project)

Going further let’s stick to backlog level estimates, as project level estimates deserves its own discussion.

Estimates based on WBS, even though may not be accurate, they at least identify dependencies, tasks etc. which help in further planning.

When someone gives his/her own estimates

They never match, task is either vastly under estimated or overestimated
Not much efforts are put-in for task breakdown
Unit testing efforts are generally ignored
Documentation is not considered as an integral part of estimates
Testing (by QA) efforts are not considered by developers
Code review and review comments implementation finds no place while estimating

I have seen that an 8 hour estimated task being worked upon across several sprints and still not being complete. I agree that estimates can’t be accurate, however things like this can cause much damage to delivery schedule. There is either utter ignorance in terms of work breakdown or incompetence/lack of skills/vague user story; whatever the reason may be, this is not at all justifiable when there are strict deadlines to adhere too.

Bottom line is – despite being moved to agile methodology, there is still need of estimation to decide scope, size and budget of the project.

Estimates are needed by everyone – Product Owner, Management, Customer etc. for above mentioned purposes.

Below are main challenges in the path of estimation

Unknown unknowns (term coined by Donald Rumsfeld) – I don’t know what I need to know. These needs to be explored and mitigated.

Known unknowns – I know what I need to know however I don’t know the details. Example: I need to write UI in WPF, however I only know WinForms.

Known knowns – Easy to estimate, however still there is scope of lot of deviation.

Procrastination by stakeholders.

Lack of knowledge of the project.

Agile estimation to the rescue?

Agile teams have an interesting way of estimation i.e. in terms of story point estimation, by using methods such as planning poker, T-Shirt sizing and many more. However, with changes in team dynamics, it is difficult to move towards accuracy.

Moreover these are estimates for the team and not for an individual developer/tester. When a backlog needs to be worked upon, one must identify all the tasks that are required for the backlog.

Underestimation:

My observation is – either generic tasks are added to work on the backlog or only few tasks are added. Sometimes even test cases are also missed and added on the last day. Whatever the reason may be – procrastination/lack of knowledge/utter ignorance; when tasks are not broken down, things generally don’t go as planned and there is task slippage.

Overestimation:

Sometimes tasks are vastly overestimated, and when they complete, rest of the time is filled with other useless non-productive activities.

Expectations from individuals or teams

Great things are not achieved by luck, one needs to work towards achieving the goals to make it happen. I think, an estimation is a step forward towards achieving the goal.

Questions to ask yourself

Have I included unit testing/test case estimates? (Shouldn’t I write VSTS test cases? Or Coded UI test for my UI?)
How many features I am going to cover during implementation of the backlog? Can they be broken down further? Does backlog/user story can be further split (if cannot fit into one sprint)?
What am I going to do as documentation for the backlog?
Am I absolutely sure about done criteria of the feature?
Are there any known unknowns?
What about unknown unknowns? How to investigate?
Is there any dependency? (If yes, then is it internal or external?)
Can I set self-deadline for every task?
Is there any need to revise estimates as I make progress?

Some of the aspects that should be considered during estimation –

Localization of the software
Security in the software
Design / extensibility / API modularity
Any unexpected events such as individual PC breakdown, build environment crash etc.
Unplanned activities that may pop up in between
Skillset of the person who will work on the task

Conclusion

Estimations are very much important, and we as a software developer are compelled to provide these all the time. So we must take proactive steps to eliminate all the roadblocks (as far as possible) and come up with estimates and then refine estimate as we work on the tasks.

Blog