Photo by Sarah Cervantes on Unsplash

Estimation accuracy is not what you should be looking for

Why accurate estimation in product development is so unreliable and so misunderstood

15 years ago, I moved my family to Boston, and with a new promotion to a Principal Consultant position at Microsoft under my belt, I went out into the U.S. market to help various enterprises with software development problems.

My customers at that time were classified as “federal civilian” which means nationwide non-D.O.D customers, which includes several well-known government agencies, like: The Centers for Disease Control and Prevention (C.D.C), the Department of the Interior (D.O.I), the United States Department of Agriculture (U.S.D.A), North American Space Agency (N.A.S.A), the Veterans Administration (The V.A), etc., and ironically, also included all the defense contractors (e.g., Lockheed Martin, Northrop Grumman, General Dynamics, Raytheon, Boeing, etc.)

In Boston, I was staging to lead a major multi-year program with Raytheon (the actual reason I was moved to Boston), but the program start would be delayed for several months — damn corporate lawyers! So, like any high-priced consultant, I could NOT be sitting on the bench and not billing clients, I had to be put to work on other engagements around the nation with some of the other customers we had. Plenty to do!

At the time, I was a specialist in software teamwork, developer tooling, engineering excellence & architectural patterns & practices. At that time I was started studying and practicing agile and lean methods on other engagements in Western Europe, and at that stage (in the U.S. in late 2007), there was quite a growing demand for agile “trainers”.

There was no such thing as an “agile coach” in the job market at that time, that came much later.

For this interim period, I decided to teach and train various software teams on these methods and mindsets in agility, as well as teach modern engineering practices.

Back then, saying your team was “agile”, or advertising that your company had transformed to “agile”, was not a thing. That kind of rubbish, and those outright lies, were yet to come.

At this time, the engagements I was assigned to were mostly stealth initiatives by companies who found themselves in peril and desired to leap ahead of the competition.

Of course, I was teaching scrum and agile mindsets, and, of course, I was dealing with many senior organizational decision-makers, trying to educate them on how things worked this way, and how to integrate it and support it into their existing businesses. Most importantly, what needs to change to make it stick and sustain it in the long term? And boy, that part was not fun at all!

Training teams of developers, designers, testers, and their immediate project leads was relatively straightforward. Scrum is pretty easy to prescribe and workshop with people keen to learn and change; they can learn it and adapt very quickly. But dealing with their I.T. managers and stakeholders up the line, most often in all cases: male, pale, and stale men in their late forties and mid-fifties (without exception), was a whole other problem in of itself. These managers didn’t really want to buy into any of this stuff.

They weren’t considering that “agility” meant that they needed to change at all!

In many cases, they had been forced to accept (by their bosses) that their ways were no longer working, and something had to be done about it. And what better way to help them deal with that than to have a principal consultant from Microsoft come and help them out?

They loved it at the start, [them, thinking] me whipping their people into shape and retraining them, [and fixing them]. But, they later came to hate it in due course because, at the end of the day, it was them that had to change the most, not so much the people working under them. Ultimately, in this specific kind context of software development (corporate I.T.), most of the work actually flows top-down, prescribed from one decision-maker to a project manager and on to software people. There are no real software “teams” to speak of in this context. We call these “feature teams” today.

Thus, those managers and stakeholders dishing the work needed to adapt their thinking to sustain any meaningful agile change in the organization. To have agility, you need to sense and respond, not jump to conclusions based on limited information far away from your customers/clients and then force your confirmation-biased, presumed solutions down the line, expecting those people who make it a reality to take responsibility for their success. Anyway, I digress, grrrrrr!

I stopped doing that kind of work after many years after seeing this occur and re-occur in corporate I.T. businesses all over the world. They are all the same.

As I said above, the biggest challenge I had was trying to educate the managers and senior stakeholders in these businesses and get them to understand some very basic fundamentals of software development that they presided over. I was actually pretty sharp at it for the most part. I had to educate them on things like: estimation pitfalls, iterative work, cross-collaboration, discovery work, breaking work down into smaller pieces, tracking progress, technical practices, quality, technical debt accumulation, economies of scale, you name it, and all the other really hard things to grasp for managers who had never written a line of code in their lives.

Actually, honestly, the hardest managers to teach this to, were managers who were promoted (from programming or other technical roles) far too early in their careers before they truly learned the long-term causes and effects of these things in the software world — even though they may have written code for some time in the past. They thought they had learned it all, already in just a hand ful of years!

(Overconfidence and Dunning Kruger quickly comes to mind).

I have to say that building the materials to spell this kind of stuff out to them and building the materials and workshops and exercises to give them new models of thinking about this stuff, back then, used to keep me up at night. How to make it relatable and impactful to them? At the time, there wasn’t a lot of help out there published on this stuff.

Anyway, I digress. To conclude this story, I was pretty successful at it back then with those managers and those software teams, as my Cust-Sat surveys later confirmed. But, I had to move on to different contexts where these practices and mindsets were already accepted as the norm, and that’s where I've pretty much been since.

Since then, except for a few future consulting engagements that I would endure like broken shards of glass in your eye, in N.Z later on, I haven't had to reach back into my career to deal with explaining these fundamentals underpinnings of software development in detail, for a very long time.

So, confronting these challenges nowadays just makes me feel like I am in a time warp, and worse, quite naked, since I’ve long forgotten all the training aids and exercises and models I was using back then to persuade and educate those who need them the most.

As such, I’ve been looking for a better analogy to help non-deeply technical people understand this stuff for years. I think I may have finally found a useful analogy that most people can relate to. Hopefully, it works for you.

TL;DR

In this article, we discuss the most damaging question in product development: “How long will it take?” While this may seem like a simple, benign inquiry, unintentionally, it creates enormous pressure, drives low-quality outcomes, erodes trust, and ultimately drives very capable people away from those asking it. In everyday life, estimates (such as for cutting the lawn, cooking, or driving) are relatively predictable and easy to make because they involve tasks we’ve done before that are easily measurable and vary very little. Not much risk in making or breaking these benign estimates. In product development, however, the complexity is much higher due to unknown variability, dependencies, and constant change due to uncovering more work, making accurate estimates nearly impossible. Worse, when these estimates are taken at face value and are not met (for whatever reason, reasonable or not) because the stakes are also often very high, expectations are shattered, often leading to finger-pointing and dire consequences and severe backlash.

The article uses the analogy of preparing meals for two different dinner parties under pressure to illustrate how estimation differs between everyday tasks and product development. In product development, each “meal” (feature) is unique and varies in size, complexity, and process, with no clear “recipes” at hand. Furthermore, these features must integrate seamlessly with all the existing ones before it, which introduces even more variability, unknown work, and uncertainty, that is also not encountered with activities like cooking.

Traditional estimation techniques, which seek precise answers, often lead to frustration, wasted resources, and poor outcomes. Instead, frequent iterative pushes increase certainty and confidence and allow teams to learn, adapt, and deliver incrementally to reach the quality outcome they sought in the beginning. The key takeaway is that in product development, certainty in time estimates is an illusion, and businesses must embrace the inherent complexity and uncertainty in the process to achieve better overall results.

Photo by Oxana Melis on Unsplash

The scariest question in Software Development

One of those core fundamental understandings that everyone working in software tech companies, like startups and product companies (quite a different context than corporate I.T.), from Sales to Finance, needs to be familiar with is the very nature of the work that product teams are doing. Why? Because this work is actually what drives all the outcomes of the business, they need to recognize that this work is not “done on a factory production line, producing widgets”.

Not understanding this fundamental and profound concept causes all kinds of duress and unnecessary waste in product organizations — even today, and it's repeating every generation. It is quite possibly the largest productivity killer for all product teams in the industry. Particularly those teams in small businesses with a scarcity of experience who are operating under a lot of pressure.

You dont hear about this problem in the big tech companies because they have this baked in their DNA through past generations of the workforce over decades. But this is not the case for new startups or businesses where the leaders are generally new to tech.

I don’t see any evidence that they teach it at school yet, either.

Almost every product company is fighting this battle to varying degrees today. The outcome is “feature teams”. It’s not that no one knows about it, but leaning into with those who create that culture requires some deep experience and a ton of humility.

The worst part is that even though this fundamental concept generates so much pain and duress, wastes so much time and money, and burns so much trust in companies, it has people pointing the finger at people’s incompetence and commitment as the problem.

There are, in fact, very few good ways to communicate about it effectively so that non-tech people can understand it, internalize it, and act on it appropriately, well enough so that they change their own behavior and avoid triggering or creating negative outcomes about it in the first place.

“Why this ask such a big problem?”

It is so hard to communicate by product/tech people; it takes so much energy to demonstrate it effectively, and it takes a lot of genuine curiosity from business people to understand it well enough that, in fact, it just isn’t communicated, and the embarrassment and pain of it, is avoided or brushed over instead. It’s such a polarizer.

Worse, that does not make the problem go away. It just feeds the fire! And the stress builds. No one is happy. It is a lose-lose all around.

Of course, none of this works to alleviate the underlying intent, and very good, motivated, and ambitious people just end up getting too frustrated and quitting. Tech leaders need better tools to prevent this from continuing. It’s a waste of great capability.

What I am talking about here is the age-old question of:

How long will it [that] take?

• If there is one question that every product engineer/designer hates to hear, it is this one.
• If there is one question that an executive/manager can use very effectively to impose some urgency on something (that they desire) - it is this one.

But, the truth is that this question seems, at the outset, that it is completely benign. Innocent even. “What’s all the fuss about?”

“How could this simple request cause so much harm?”

What’s wrong with wanting to know how long a new “feature” will take to get to customers?

Seems completely reasonable, right? “We are simply preparing for what is ahead!”

• “What if I want to make future plans based on that knowledge?”
• “What if I want to set the expectation for someone else that this feature is coming within a certain timeframe?”
• “What if I want to know when I have to stop pursuing this task on my task list or end this email thread with an important customer that centers on this very thing?”

All seems very, very reasonable in a commercial setting. No?

Yes, in fact, it does. It seems very reasonable to ask these questions about most things in the world, particularly in commercial settings.

“After all, this is business and we have to make decisions and move forward, plan the future, and aim for success. Specifically, in a software product company, that success is dependent on these specific things happening in some known timeframe.”

So, what is the problem with this particular question when it comes to delivering something of high value to the business?

In the context of product teams, why is this question so hard to answer [reliably] for product teams when it comes to delivering new patches/features/capabilities to customers?

By the way, if you wonder why the word “reliably” is injected above in this context. It is because it is always there in this context.

We expect, and we demand, that the people who need to answer these questions give us reliable information. That is what we hire them to do. To be honest and open and truthful, and accountable. For them its a matter of personal integrity, and vocational integrity. Their word is their bond. For them, accuracy and precision is important, an unreliable answer is as good as useless to them.

Those people are merely highly conscientious and intelligent people trying to do the their jobs as best they can. Lying or leading people on, isn’t what we hired them to do, and certainly not what we want them doing. It is certainly very, very uncomfortable to them.

Let’s explore this concept in more depth.

Photo by Crissy Jarvis on Unsplash

Estimation in the normal world

Estimation, in the real world, is something everyone can relate to in their everyday lives.

• How long will it take you to cut the lawn?
• How long will it take you to prepare dinner?
• How long will it take you to drive to work?

In ordinary conditions, these kinds of things, this “class of estimation” is pretty reliable and accurate for the most part.

Here are some aspects of it you may not have thought about before.

• You’ve likely done it before in the past, so your estimate is anchored to the last time(s) you did it — you often derive an average from memory.
• There is usually one, perhaps a couple of things that can dramatically change the time it actually takes you. The causes of these things we call “variables” of uncertainty. These things are indicated by the well-known excuses that are communicated when someone holds you accountable to these kinds of estimates and, by the way, only when you run over your estimate significantly. Things like: “I ran out of petrol”. Or “I hit a stone and dulled the blade, so it took much longer to cut the lawn”. Or, “I had to run to the shops to get some missing ingredients, so, dinner will be another 20 mins away”. “There was an accident on the road, so I sat in traffic for another hour and a half on my way to work!”. All sound very plausible and relatable, so we accept them immediately.

So, for most of the things in life, we can estimate reliably and accurately because we have known past “track records,” and we generally are dealing with very limited “variability” that everyone around us can understand easily and can relate to.

When it comes to doing new things we haven't done before and making estimates on those, it is a little different. Things are much harder when we don’t have track records or can explain unrelatable causes of delays to people.

For example, if you are building an Ikea chest of drawers, you would have no idea how long to estimate to build it.

No track record! That is no problem. In these cases, you would simply seek an online average build time from either the manufacturer or others (online) who have a data point for you to use as an estimate.

Variability is an interesting one here. Let’s say that you don't have the specific tools needed to complete the job! Then, that estimate is suddenly not so certain. Certainly, your mileage may vary significantly from the average estimate you got from someone who has used all the best tools.

Contrary to popular belief, Ikea does not ship you ALL the tools you need for all their products; isn't that fun for a young adult trying to set up up their new apartment?

For example, go buy the Ikea STORKLINTA chest of drawers. If you don't have a screwdriver and hammer at home already (critical tools to make any progress in the first few building steps) you are in for a really sore experience, certainly blowing out the average estimation, by many hours. The key point here is that if you had an electric screwdriver over an ordinary hand screwdriver or no screwdriver, your estimates could change dramatically. That’s variability in action too.

So, let's bring this back to the software and why life in product development (specifically) isn't as straightforward as it is in what you experience in normal everyday life.

This is very important to understand because we as humans have some pretty strong cognitive biases and mental shortcuts that we often employ in everyday situations that often get us into trouble in the everyday business world. Especially when it comes to the pressures created in business, in certain complex and unfamiliar domains.

These mental shortcuts often have us make big assumptions and leap to conclusions that seem on the surface very reasonable and logical to us, but we forget that they are based on our biases forged in different conditions and different contexts.

Context matters more than most people think. We tend to project what we know, onto what we don’t know, as a way to simply and understand it better. But we are often wrong to do that. Its the Dunning Kruger effect, its Projection Bias, its confirmation Bias, its the Anchoring Bias, and its a bunch of other that we all do everyday.

Photo by Annie Spratt on Unsplash

Let’s put YOU on the spot for a second.

The question I am going to ask you at the end of this is, “How long will it take?” and I want you to think about answering that question now, in a few minutes, as if the Founder/CEO/executive/senior manager of a software company might ask you in a meeting about the work you have just announced that you have just planned to deliver.
…

Easy Test

Okay! Here goes - easy level:

• You need to host a dinner party on Saturday.
• It is now, first thing Saturday morning.
• You have 13 guests coming at 6 pm this evening.
• You‘ve decided it is going to be a roast chicken dinner with all the accoutrements, appetizers, and wine but no dessert.

…

Now, tell me, “How long will it take you” [to prepare the meal before your guests arrive]?

…

Easy Solution

Easy! right?

You would simply make a plan by following this kind of logical process:

• You list out the components of the meal you wish to offer your guests. What appetizers? What accoutrements? Which vegetables, gravy? cheese/crackers? chippies, you name it, etc, etc.
• You gather the recipes for each of the dishes. (we are conservatively assuming you don’t have this memorized already).
• You know how many guests are coming (x13), so you can easily figure out what groceries you need to buy. It's simply 13 times the ingredients in all the recipes you are using, minus what you already have in your pantry, fridge, freezer, and wine cellar!
• From the recipes, you can look up the total elapsed cooking time required. You simply get the times from the recipes of each dish, and you plan out a simple cooking sequence based on the capacity of the tools you have in the kitchen, i.e., what you can fit on your particular stove and in the oven (in your kitchen) at the same time, and what pots and pans you have. You might also account for some kind of cooking order, so you have all hot food coming out at the same time.
• You figure in some contingency time for unexpected upsets and distractions, feed the cat and dog, take a break to try the wine, and some time to get showered and changed before the dinner guests arrive.
• You figure in some extra time to prepare the place settings for 13 people and to vacuum and tidy up your house before they arrive.
• You sum it all up and work backward from 6 pm.
• 6 pm minus your estimated time is your start time.
• Job done! You’ve got your estimate on how long it will take you and now when to start.

…

Brilliant! Pretty straightforward. Now, notwithstanding a power cut, a shortage of groceries, a car accident, mass guest cancellation, a broken vacuum cleaner, or unpredictable things like that, you are probably feeling highly confident that you can deliver this dinner party to your 13 guests on time, within your plan, using your first estimate.

Perhaps higher than 90% confidence in your estimate? Pretty likely for most people. Life is easy!

Well, it still remains to be seen, right? You have a pretty hard deadline here to hold you accountable for your estimate. Guests will definitely start appearing before 6 pm, so if your estimate is wrong, you are going to feel very embarrassed inviting them into a steamy cacophony of a Gordan Ramsay’s Hells Kitchen in your own home!

The bottom line is that this exercise is formulaic, given you have all the certainty in the world and all the information you need, you have clear cause and effect, and you sum things up. Very little unknown. High certainty, high confidence. Low risk. These are all things a business loves to have. Simple, predictable formulas, no high risk, no stress!

Photo by Michał Parzuchowski on Unsplash

Estimation in Product Development

Alright, estimation in most processes in life is pretty easy for the most part. The difficulties really only come when you are doing things you've never done before or where you don’t have the information at hand to tell you how to do them. Or — you don’t have all the materials and tools to do them well. Or — you hit roadblocks along the way that set you backward. Or — you make mistakes that force you to do certain parts over and over again. Or — you have to experiment to learn how to do them properly. Or — select what the best solution could be, given the constraints you are working with.

Woah, those are quite a few more unknowns and risky things. Sounds scary!

Let's go to the next level.

This is the level that every product team on the planet deals with everyday of their working lives. I’ll explain how that works a little later, but first.

Let’s see how you do.

Photo by Soroush Karimi on Unsplash

Hero test

Okay! Here goes — hero level:

• You need to host a dinner party on Saturday.
• It is now, first thing Saturday morning.
• You have some guests coming after 6 pm this evening, but you don’t know how many guests are coming, nor do you know when they will arrive!
• Every guest must have a separate meal, and each meal portion could be different for each guest. No two guests can have the same meal!
• There is a predefined list of 50 meals to choose from, and you have to make them in the specific order on the list.
• You have no access to any recipes or knowledge of what is specifically in each meal! These are meals you’ve never made before! But, you are familiar with having seen and eaten them before in your life, and you have high confidence you can guess and figure it out.

…

Now, tell me, “How long will it take you” [to prepare all the meals before all your guests arrive]?

…

Hmmm! okay……

…

Hero solution

Well, you might think this challenge is unnecessarily unreasonable. But it is not really.

This is the challenge that every product team is faced with to a very large degree when building any new feature of every product every day! They’ve still got a kitchen, the utensils, and some ingredients lying around, and they are innovative! So, would they not give it a crack?

As an industry, we learned years ago how to deal with this challenge, and some (very few companies) have learned (the hard way) how to thrive in it and actually gain other benefits from working in the ways that they do. But it does take a dramatic mind shift to take advantage of it and move away from the driving certainty with accurate time estimations.

Let me show you how we in product development best tackle this seemingly impossible task to many.

• Unknown number of guests: We might know, by name, most of the features we want to deliver at any one time for any new release. But what we don’t know precisely is the number of tasks/activities/pushes/components within a cohesive product that it will take to add an individual complete feature to the system. Especially in an early stage or rapidly evolving system. We can roughly guestimate how many things there are to do before starting out (based on past knowledge and experience) to a very low resolution, but we cannot be absolutely sure in almost all cases. Nobody can. (one possible exception to the rule is that a single person built the entire system from scratch on their own; they have every piece of the system in their own head at once, and thus the system is small, simple, and nascent).
• A guest could arrive anytime, Nor can we predict the exact order of these tasks/activities/components to do first or last — they may not work at this time. We can make a guess on the next thing, knowing that we possibly won’t be right. What if we are wrong? It can have dramatic effects. We also know that we can’t know the precise order until we get started with the tasks/activities/components because they inform us as we go. We uncover knowledge as we proceed and build a picture in our heads. What is worse is that we often have to jump back and forth, inserting newly discovered pieces into the sequence as we go, because we have to make this change fit with what we have without breaking the existing thing. Then there are the things that we break by fitting this new piece in the mix. We can’t predict that either. We have to detect it and design a just-in-time fix to that as we go. None of this is predictable upfront — no matter what wishful thinking you have. No one can do that reliably upfront, either. We simply have to learn as we go. There is no other reasonable choice to proceed at all.
• Every guest gets a unique meal, differing portion size, no recipes: If we liken a meal to a “feature” then in product development, no two features are ever the same. All of them take different amounts of effort to bring to life. Each feature is unique in its own design, constraints, and implementation and varies in the actual work we need to do to fit it into the existing system. The biggest realization is that in product development, we also have no recipes to guide us! There are no known ingredients, no units of measure weights and measures, nor do we have a prescriptive cooking process with precise, reliable instructions to follow that guarantee any feature gets built correctly. What it takes to make the feature (ingredients) and process to make it; is unique and varied every time! That is a lot of unknowns to be juggling! All we really have is a past experience to draw on, but nothing more concrete and measurable to follow there, nor borrow from another person’s experience of doing it before us — no one has that experience. The best we can do is guesstimate what needs doing and how long it would take, based on past experiences of doing something like this before (in our own distant past). If we were to pretend that we knew with any level of accuracy on how to do it, or how long that takes, we do that knowing that we are cheating ourselves to the detriment of the quality of the thing we are doing. We can always cheat quality and take shortcuts. The question is whether that is worth it in the long run? But that does make the estimates any less accurate.
• There is a predefined list of 50 meals: this is fine. This is just like our plan of what features to build in any given release. Yes, we have this upfront to some degree of certainty. However, things change from iteration to iteration as we proceed, we discover new things, new constraints, and change designs and thus the outcomes, so that’s not that reliable either.

So, as you can now see, we enter this kind of process with all these variables: lack of information, lack of aids and assistance, and a ton of uncertainty.

Much like you would be doing by volunteering to host that dinner party!

The only certainty we could really have is that:

1. It is feasible — we CAN do it, eventually. We do HAVE the materials, knowledge, and experience to get it done, no matter what we face along the way. It takes courage and some confidence, as well as resilience and discipline.
2. It has to be incremental — we have to tackle it one step at a time, and it takes maximum focus and discipline in order to minimize adding too much future complexity to it, and keep the quality high. For us, this is building additional structures that decouple things and make things cohesive so that changing things later is easier/feasible. Remember, we are not building new, discrete things (like widgets). We are changing and adding new things to ONE whole thing. The system, the product.
3. We have optimism that we will get the outcome very close to what we started out to get. But that is not 100% certain either. Things go wrong along the way, too; sometimes, we encounter hurdles that are very hard to overcome right now, and we have to prioritize and compromise often to our detriment later.

To deal with all this uncertainty effectively, we intentionally and artificially put constraints on ourselves that have proven to effectively produce better results than just going off into the wild until it's done and only returning when it's all done and dusted — whenever that might be.

This is the outcome that every [responsible] product leader fears will happen if those doing this work are left unchecked. It's is highly risky, and its an all or nothing proposition. Sometimes, very rarely, it works out, and that is a fluke. Most of the time, it just doesn't work and fails. Requiring us to start again. Pure waste.

A process that more often works is to set ourselves smaller increments and goals, aim to create partially implemented features, actively learn about where the variability is, and adapt as we go to inform the next partial step that makes a slightly better feature. We come up for air frequently and assess what needs to be done to move on and when to stop.

If we hit some kind of time limit or constraint and can no longer do anymore, then so be it. At least we now have something partial that works, albeit it may not be everything we wanted. But we don’t have “nothing” to show for the time invested.

This is what iterative, incremental agility means in practice. Making the best of a very tricky situation of operating in very tight constraints.

Photo by Arshad Pooloo on Unsplash

Aint no widgets here

But you know what? There is one last kicker in the work that we do in product development that is at the heart of what we do, and that is hard to compare with cooking.

It is something that most people working with software prefer to overlook or ignore and not consider (when it suits them) to characterize the work that we do correctly.

By the way, takes decides of experience to come the this realisation on your own, so ironically, many engineers, who might work in software all day long, still haven’t become of aware of this either, as a major factor.

All the meals we have to cook in software (i.e., features, changes, patches, etc) aren’t delivered individually to dinner guests. In software, we ADD these features to an already existing collection of preciously delivered features. We call it a product, and, all those meals share the very same ingredients all at the same time! There is no good comparison with a dinner party here, but suffice it to say, in software, when we make a new feature, it has to fit in with all the other features that already exist, AND it can’t break any of the existing meals that already exist.

Perhaps this silly analogy might work since we are talking about cooking:

Let’s imagine that out of all the meals you’ve cooked before for your guests, some of the meals were Italian cooking. Now, let’s imagine that the moment that you cook a Greek dish and give it to a guest, all the previous Italian meals instantly go off and become inedible! Whether they were already eaten or not!

It's silly, but the analogy demands that if an Italian meal has already been consumed by a guest, then that person is instantly getting sick from food poisoning, and they are vehemently letting you know all about it!

In software, this is what we encounter when we try to fit a new feature into an existing product codebase, and without knowing it, we break some dependent feature somewhere else in the product in doing that. We ship this defect to customers, and some other part of the system breaks when customers try to use it, and we hear that back as a production support issue.

Sometimes, this is detectable (before we ship), sometimes not. Sometimes, it is easy to fix; sometimes, it is not. It is unpredictable in occurrence, unpredictable in detection, and unpredictable in the size of the remedy to fix it.

The good news is that we do have ways (better technical practices) and better processes to mitigate this happening, and again, doing those practices adds more time to add long-term quality, which is also hard to estimate up front, too — since it is proportional to the time to create the change in the first place.

Nothing anywhere in here product development is as predictable as knowing “how long it takes to cut the lawn.” So, rather than trying to be accurate, knowing that we can’t be, and lying about it, we don’t bother to waste our time trying to do it to any serious degree.

Photo by Dingzeyu Li on Unsplash

Conclusion

Most other creative pursuits in the world that most people experience in their homes and professional daily lives do not face this kind of complexity and uncertainty in their work. However, everyone making software products faces this problem and complexity every day, and they, too, are only human.

Experienced product development people everywhere know the difficulty and complexity of what we are dealing with here, and only the inexperienced members of product teams seem naively willing to ignore the realities of it. Making them vulnerable to sometimes setting expectations that can’t possibly be delivered reliably.

However, relentlessly, everywhere, many business people (who don’t understand the nature of this work) still believe they should insist on demanding the certainty of the quantity of time these things take as they can reliably do with other kinds of work in a software business. And worse, actually hold other people accountable for these quantities, happily ignorant about the real details of why that is the case.

But, the inconvenient truth is that this kind of certainty cannot possibly exist to any degree of meaningful accuracy in this kind of work.

That should now be a little clearer to those who wish they would be. Try hosting the second dinner party!

Ignoring this fact usually leads to all sorts of wastage and long-term damage to products, and trying to conjure the illusion of certainty distracts and hinders the very people trying to do this complex work.

This approach is simply not effective, and in most cases, pursuing it is just wasteful and destructive. We have better techniques than those these days. Learn to embrace them. If you are looking for what to do instead, we need to get into that in a follow-up article. Stay tuned.

Communicating or describing this problem of the challenge of accurate estimation to people without deep experience in software is extremely hard. It's so hard, in fact, that for most technical people, it's simply avoided because it's so hard to describe in terms that the listener could sufficiently relate to and then accept.

I am hoping that this article can be used as a tool to help both sides of businesses demonstrate and discuss this kind of complexity.

I certainly would have liked to have used this tool in the past to raise awareness and examine this difficult topic with past managers and leaders in their organizations.

Perhaps it can be effective for you, too?

p.s. It may have occurred to you, in that last exercise, that this scenario is not far from what every commercial restaurant deals with every day — An unknown number of guests, guests arriving throughout the day, many different meals, etc.

However, if you think about it, restaurants are able to manage the variability and unknowns more effectively by constraining a couple of the things: For example, they constrain the variance in the meals to a defined menu, which they know well how to prepare: they have the equipment, they stock up on ingredients, and do much prepare for. They also have an idea of how many guests could be coming in the door on any given day from analyzing past consumption behaviour. Yes, some meals stray outside the fixed menu with special requests, but they only deviate in very simple ways — adding or removing available components. For example, you don’t get to order a cheese beef burger and ask to replace the beef patty with oysters, and the bun with seaweed do you? They also constrain the meal service period to a certain number of hours in the day — opening hours. This removes other variability.

But perhaps the most profound difference is that they are not building a single thing up with every meal they deliver. The meals don’t add up to one giant meal, for example, and the next meal has no dependency on the previous meal made either. Meals are, in fact, discrete standard or custom widgets. Every day is a new day, and nothing carries forward.

So, despite the near similarities, a restaurant is not doing the same thing we are doing in product development at all. This is more akin to a just-in-time factory production line, than any kind of product development.

And those two things should never been conflated! ever!