Editor’s Note

I am often asked by client’s to manage software development projects. One of the aspects that most bothers a client is the cost of bespoke development, particularly because of the reputation major software development projects have of going over budget. This is often down to poor estimating, or even worse, of trying to use traditional contract pricing methodologies with an Agile development project. 

This article, written by Paul Barnes, Head of Projects at TOPTAL, provides an excellent analysis of this situation and describes, in detail, the approach Toptal take with their development projects.  It aligns very closely with my own experiences, which is why I am sharing it here. I hope you find it interesting and informative.

Paul Every, Solitaire Consulting.

Introduction

One of the hardest things to do in software development is to determine how long and how much it will take to deliver a new software product. Should it be so hard? The answer is not straightforward.

Software costs estimation is inherently difficult, and humans are terribly bad at predicting absolute outcomes. No two projects are the same;each is unique in what it sets out to achieve, and unique in the myriad of parameters that form its existence. Often, what appears to be a simple problem on the surface is much harder or technically challenging to implement in reality. And, undoubtedly, there will be ‘unknowns’ with the project that can only be identified when they arise.

Additionally, no two people are the same, whether you’re a customer, a developer or a user. We come preloaded with our own set of knowledge, experiences, values, expectations, attitude to risk, and ability to adapt.

Writing good quality software is bread and butter for senior engineers; creating awesome software products can be a much harder endeavour, for all involved.

So, how do you go about estimating the size, duration and cost of a project? Let’s explore Agile project estimation and software development costs, and how we do it at Toptal.

Traditional Contract Pricing and Estimation

Traditionally, using non-Agile practices, software projects have sought to fix functionality or scope, and to let time and cost be a variable. This causes problems:

1. How do you know that the functionality you fix at the outset of a project really is the functionality that serves your business or customers best? More often than not, functionality or scope will change, which is why we hear about ‘scope creep,’ the outcome of desired needs being identified through the lifecycle of a project and being determined as necessary or compulsory
2. When cost becomes a variable we lose control over the return on investment (ROI) that we’re seeking to achieve. Increased cost is often a product of unidentified risks or changing requirements, which means we have to add team members to do more work in the same time frame, or keep team members longer. Neither is desirable
3. When time is a variable, we lose control over the position in our market. Perhaps we miss an important industry date or our competitors get their product out before us, thus losing any competitive advantage our project may have had.

There are many other outcomes of variable time and cost, which are often negative and undesirable.

Of course, many customers and organisations seek to fix all three components of this ‘magic triangle’. Unfortunately, it’s nigh on impossible to realistically achieve. There are too many elements that conspire to unsettle this ideal, which ultimately end in products that don’t meet a need, take too long to benefit its customers or cost too much to realize business value.

Agile Contracts for Software Projects

Cost is a product of time and people (team members). Add more time, and you add cost for employing people for longer. Add more team members, and you increase cost to deliver the same business value. The things we really want to avoid. This is why Agile principles believe in fixing time and team members and allowing scope to be the variable component.

Which sounds better and increases stakeholder confidence, fixed cost or variable cost?

Of course, it’s important that a product delivers on its promises and the needs of its customers. It’s no good spending an exact amount of time and an exact amount of money, if at the end, you have a product that nobody wants or can use effectively.

So Agile contracts focus on the following:

• Fixed price work packages – The whole project is broken down into logical ‘mini’ releases which contribute to the full product outcome. Each release is a work package that is priced accordingly. As a work package is completed, future work packages are re-estimated based on what we have learnt from the previous one. This avoids unnecessary contingency and allows for a level of re-prioritisation and new/revised features to be defined by the customer.
• Early termination – This allows the customer to seek to terminate the project early if enough of the product has been delivered and there is no further ROI to be achieved by retaining a project team that will only continue to deliver marginal gains. This clause is typically allowed at any time and is valid as long as the project team and customer have maintained a strong, trusting and close working collaborative relationship. The benefit for the customer is that the project will finish early, having delivered all the valuable features necessary to make the product viable. In return, the supplier is paid 20 percent of the remaining contract value and offsets the risk of retaining staff.
• Flexible changes – Change is a theme that runs strong through the veins of Agile software delivery. We expect to not know everything we need to make a product successful from the outset. So, we promote change, based on relevant data and feedback, to ensure that the right product is delivered. At the end of an iteration, changes can be swapped out for old features no longer deemed necessary or a priority. As long as the change is of equal value, there is no further cost. If the change is of lower value, additional work can be identified or pulled forward from the remaining backlog. This clause is valid as long as the project team and customer have maintained a strong, trusting and close working collaborative relationship throughout the project.
• Additional work – Through the life of a project, more features may be identified that would not be achievable under the existing fixed price contract. For this scenario, either additional newly priced work packages can be added to the end of the project or revert to time and materials.
• Ranged estimates – There are two ways that estimates can be ranged in an Agile project contract: a range of duration or a range of features. A range of duration allows for an estimate to say that the project or work package will take 12 to 16 weeks for a given set of scope. However, adding duration adds cost as you keep project team members for longer, or it means they can’t be released to work on other development projects. At Toptal, we prefer to range features across a range of story points, keeping the scope as the variable but promising to deliver a minimum level of value to the customer within the fixed time frame of the work package or overall project.

It’s worth remembering that you can always add more scope later. Perhaps you’ve started to earn revenue, you’ve increased users or reduced costs. Either way, it’s much easier to ask for more money and time if you’ve already demonstrated a return or improvement and are delivering business value.

Our approach to software costs and pricing

At Toptal we work closely with our customers and engineers to employ techniques that promote stakeholder confidence in project duration and costs. We work at continually elaborating and adapting planning from an initial high level down to more granular detail when it is appropriate and necessary to avoid waste and to enable managed change.

The following steps are taken in elaborating an estimate and fixed price project:

1. Initial high level scope

At the outset of a project, we know least about it’s eventual outcome. It’s folly to imagine it’s possible to know exactly what features our customers and users need from the beginning.

So, we start with a project charter and a high level set of “epic” features that guide the direction of the project, based on a sound vision and objectives

a. Vision and Objective setting What do we need to build ? What do you need to achieve and what are your business objectives? Understanding these questions allows us to set the scale of the project. Do you need a prototype to test an initial idea, concept or technology? Have you identified a clear proposition that has been tested with your market and are you ready to build your first Minimal Viable Product (MVP)? Or, are you scaling your existing business or product to take it to the next level?

b. High level “epic” features Without going into too much detail, you’ll want to define the features that your product has to fulfil your customer’s needs. This is a structured “shopping list” that describes the bare bones of your product; often these are referred to as “User Stories” or epics

c. MoSCoW analysis MoSCoW analysis is a technique that, put simply, helps to identify what is really necessary to make the product viable and what is a nice to have. Those that are identified as a “Must” satisfy what will encourage users to engage and adopt your product. Those features identified as a “Should” will surprise and delight your customers but could be built later. The “Could” items often add no significant business value, may not increase the return and are the lowest of your priorities. The “Won’t” features could well be important one day but are out of scope for this project iteration. However, identifying these now can help to set in mind the potential scale and size of the product for the future.

2. Proposal

To make a decision on whether to proceed with a project, it’s necessary to base that decision on well informed data: cost and duration. These are the minimums you need to ask yourself: What will it take to create the product we want ? When can we launch? Does this align with our business strategy and finances?

With the details above, we’re in a position to provide a proposal. Our engineers are handpicked for the specific project requirements and work together with a project manager to derive at least one technical solution, an estimated duration that delivers the known scope and an estimated cost to complete the project.

As we mentioned before, at the outset of a project we know least about what will be delivered. We deliberately keep the features and scope vague, since to do otherwise suggests we know exactly what is required. An estimate at this stage would be the least accurate, but gives guidance on whether it’s worth proceeding with the project.

The proposal is the first tool in elaborating the duration and cost of a project. Once a proposal is accepted, we can move forward to provide a fixed priced quote.

3. Release Planning

The next level of estimate elaboration is to create a release plan that will deliver a range of features in a given timeframe. We derive this from a list of features, the size of the project, how quickly our team can develop quality software that meets a customer’s expectations and techniques for managing the risk of uncertainty.

a. Product Backlog The product backlog is simply an ordered list of “Epics” or “User Stories” that represents the features required for a product. This list starts life as the epics discussed earlier, but between the assigned project team, project manager and customer, we now break these down into more meaningful items. Each of the items represents a portion of business value to the customer. We don’t go into more detail at this stage, we don’t need to know the acceptance criteria, we don’t need to know if a button is blue or green, we just need to know there’s a button that allows some task to be performed.

b. Estimation Now that we have our list of features described as user stories, the team estimate these discrete items of features using a technique called “Planning Poker.” This is a useful technique that ensures quick, reliable results based on expert opinion and analogous sizing. Planning Poker assigns an agreed number to each item representing its size and complexity. This is called a story point. Other Agile estimation techniques and sizes, such as ‘ideal days’, are available.

The end of this process will determine the size of the project and dependencies between features. The size is determined by adding up all the story points from the items in the product backlog. If that number equals 120, then the size of our project is 120 story points.

c. Prioritisation Now that we have a backlog and a size for the project, we’re in a position to prioritise it with the customer. This is really about identifying what is most valuable to the customer in order to achieve the desired results. The item at the top of the list is considered the most important, the second item is less important than the first, and so on through the list. No two items can be as important as another, each item’s priority is of relative importance or value to each of the other items.

This approach to prioritisation is an important milestone in planning a software project. We now know what is important to the customer and in which order to complete work, taking care of dependencies, to deliver a product that meets expectations.

d. Release Planning To date, we’ve determined what we believe the product to be and how big it is.

Now, we determine how long it will take to deliver a releasable product. The customer and team, including the designers, engineers, testers, scrum master and project manager, work together to identify what can be achieved and how quickly work can be done to create a release plan.

The release plan also gives insight into how the project will align with a customer’s strategic plans.

And finally, this plan ensures the project team has a guiding light that leads the way and defines a logical endpoint to development.

Before we start, we ensure we understand the agreed definition of “done.” This is a given set of criteria that a customer will accept as complete and also meets all of the engineering requirements to be considered releasable.

To take a basic scenario, we take the total number of story points we got from sizing our backlog and divide that by our teams anticipated velocity. (NB velocity is normally expressed as a range, but for simplicity we’ll use a single number here.) We work in two week iterations so our velocity will be reflected by how much we can complete in a two week cycle with the available capacity of the team. If our story points totalled 120 and we anticipate completing 20 points per iteration, the total development duration would be 12 weeks or 6 iterations. We add to that a sprint 0 of 2 weeks and a release preparation sprint of two weeks. In total, our project length is 16 weeks. There are techniques we can use that would help build an appropriate risk buffer into our planning, which we’ll discuss later. But in short, we use the buffer to manage the risk of uncertainty and to come to a minimum agreement of expected story points to be delivered. The outcome might be a range of 90 to 150 story points delivered, 90 being the minimum that would be acceptable to create a viable product.

Alternatively, if the project must be complete by a given date, in say 10 weeks, the team would determine how much of the backlog can be completed in that time. If we anticipate 20 story points per sprint, plus Sprint 0 and a release sprint, we would be targeting 60 points completed by the end of the project. Again, we would look to manage risk by adding an appropriate buffer, which might result in a target of 45 to 75 story points completed and ready to release. The 45 story points would align with the minimum acceptable to deliver a viable and valuable product. This is one scenario where you might expect to add a team member to increase velocity, if appropriate.

Of course, all of the above is supported by good quality communication and collaboration between all parties to derive a release plan that is achievable, realistic and acceptable to the customer.

4. Fixed Price Contract

Once a release plan is agreed upon, we’re able to create a quote for a fixed price project contract. As mentioned previously, it’s advisable to keep the project duration and team fixed and the scope variable.

The quote for a fixed price contract is delivered along with a statement of work and agreed payment schedule.

As long as there is trust, communication, collaboration and a readiness to enter into the spirit of an Agile software project, all of the steps above allow us to deliver a quote with a realistic degree of confidence that a project will be delivered on time and on budget. Of course, there will be occasions where a project is delivered early or late and we deal with these variations with the utmost transparency.

Estimation Techniques

Agile planning and estimation is supported by a number of techniques that a development team can use to gain confidence in their size, effort, duration and cost. Here are some of the ones our teams use to estimate the size and cost of a software project.

>>> Click here to read about these techniques and the rest of this article on the Toptal Engineering Blog

Some closing thoughts

Agile can be a very difficult concept to fully grasp and adopt. This is no less true when managing sensitive topics such as price, scope and duration. Unfortunately, I know first hand that demanding clients want all things fixed up front and are eager to blame the supplier when it all starts to unravel. Equally, I’m aware of vendors that dig their heels in, become unresponsive and fail to respond to customer needs. Taking an Agile path is fundamentally built on trust, good relationships and stellar communication. These must be values held by both parties in order to maintain a healthy project for the equal benefit, satisfaction and success for all involved. Keeping an open mind and constructive attitude to collaboration and negotiation is the best way to avoid relationships going sour.

I’ve worked with clients that have found it hard to embrace the adaptive nature of Agile and to relinquish a command-and-control attitude. It’s hard to let go and put all your faith and trust in a team you don’t know. Often, clients may wish to create all the requirements up front as a specification of what will be delivered. This gives them a feeling of confidence that the scope of a project is well-defined. But ultimately, this fails to materialise as a successful approach. If we’re locked down to scope and unrealistic demands in a contract, problems arise very quickly.

We know, when taking this approach in traditional methods, that scope changes, unknowns are uncovered or what we thought the customer wanted is no longer true or way off the mark. Taking an adaptive approach to pricing, planning and scope allows customers to truly identify their product to be exactly what their market needs. It enables a vendor to be responsive, imaginative and efficient too. Harnessing collaboration between customer and vendor over contract negotiation is key.

Vendors need to be honest and customers need to be realistic about what can be achieved from the outset. A vendor that promises unrealistic targets and then increases costs may win the initial contract, but will soon lose favour from a disgruntled customer. Too often, relationships break down due to a lack of trust or confidence between parties. Trust must be built from the outset and maintained throughout the course of a project. A vendor must be flexible and cooperate with changing needs. Customers always want more; it’s a natural consequence of doing business. There must be a an equal and beneficial value exchange between both sides. For customers, they’re looking to create value for their business. For vendors, they should be looking to create value by forming long-lasting relationships with customers. Observing the Agile Manifesto’s values and guiding principles is a sound basis for forming strong, balanced and long relationships.

Summary

I hope this has given you some insight into planning, estimating and defining a price for an Agile software project. All of the approaches and techniques above are designed to build trust in a team and to build confidence in customers’ minds on how long and how much it will take to build a software product. And ultimately, to build confidence in making a decision to proceed.

Follow these guidelines and you’ll be sure to find a satisfactory route to bring your software product to life.