And how to make them succeed
If you work in a product team, chances are you have had something like the following experience. Your leadership regularly hands down the itemized list of features they want you to build over the next year or quarter. These features range from customer wants and sales commitments, to things that the leadership and other stakeholders think the product needs to do. They ask you to estimate and prioritize each of the features and put them on the Product Roadmap. Your Product Managers go away and do some research and write detailed specs for each of the features. Product Design comes up with beautiful designs and mockups for how the features will look. The Product Delivery Team then tries to break it down into estimates, maybe using story points, maybe using t-shirt sizes to try and work out the relative effort. You are then asked to put a deadline on how long it will take. Of course when you hand over the estimates, you are accused of padding the schedule — it can’t possibly take that long. So with some compromise and fiddling with numbers, the Product Roadmap is finalized. It lists each of the features, maybe with screen mockups and the timelines in which you plan to deliver them. Sales gets excited and starts talking to prospects about the new features coming, and Product Marketing puts it in a pretty graphic and puts it on your website.
Two sprints in on your first feature and you find there is a whole bunch of stuff that you hadn’t anticipated. The UI that the product designer came up with is too complex to implement — but they won’t budge on simplifying it, citing “usability”. The feature relies on some APIs that are taking longer to build than you anticipate, and you have completely neglected to account for the impact on performance and the cost of new services you need to spin up. Some of the capabilities the product manager wants to add don’t make sense or require more thought, and some of your resources got pulled onto another project. You want to cut scope, but the Product Manager doesn’t want to give up on features that they spent a long time writing specs for. Sales says you can’t cut this particular capability, because it’s now part of a customer commitment. Everyone pushes the delivery teams to work harder. The deadline looms, you compromise on some design and some of the capabilities, and cut corners to get it ready to ship, but the delivery team knuckles down and gets it done. The feature is ready to ship on time. There is much rejoicing. Product Marketing starts to ramp up their Go-To-Market plans. The team barely has time to breathe a sigh of relief before the next feature on the roadmap starts looming.
As you are arms deep in implementing the next feature, you start getting bug reports on the feature you delivered last month. The shortcuts you took are coming home to roost. Product marketing is not getting traction, their outbound marketing efforts to sell your new feature are not converting. Your leadership is unhappy — what you delivered is not what they thought they were getting. Worse, you aren’t even getting engagement with the customers that claimed they desperately needed this feature. They either don’t have time to evaluate it, or they try it and it doesn’t quite work for their use case, or it’s too hard to use. You want the time to address some of these problems, but you’re under pressure to build out the next new feature. You have to choose between delaying your new feature, or abandoning the old one, or some compromise between these two. Of course no one is happy with those choices.
No one in this scenario is a bad actor. The leadership team truly believe that they have chosen the right things to build, and their vision of what those features look like was just not aligned with what Product ended up building (if only they listened!). The customers who asked for that feature, really believed that they would use it, but it turned out that the effort to move to the new feature just wasn’t worth changing their existing process, or that it was missing some critical component that they assumed would get built, or — as is often the case when building features for sales commitments — neither Sales nor the customer really understood what they were asking for when they asked to put the feature on the roadmap. The product manager believed in the feature too, they spent countless amounts of time thinking about how it would work, talking to customers about it, writing lots of specs and User Stories for the developers. Design committed to the vision — producing static designs that were clever and beautiful looking. Product delivery — the engineers — genuinely believed in their estimates and that they could get it done in the time, but couldn’t justify their padding because they didn’t know what they didn’t know.
So who is to blame? The problem is that top-down feature roadmaps assumes that you know up-front exactly what you need to build, rather than allowing you to discover and iterate in order to learn which ideas are best aligned with your customer’s and your company’s goals. I think this reveals some fundamental truths about working in software product companies.
You are wrong about half the time
Software development is a casino, where the odds of you being exactly right about any particular feature, aspect of a feature, market etc are probably not much better than chance. This is because there is a massive, ever shifting information gulf between you and your customers, not to mention the market and your competitors, that your intuition does a poor job of filling. Even the best meaning Product Manager with the most amazing research skills is not going to come up with the right answer simply by thinking about the problem. You have to actually have to test your hypotheses and learn what will work and what won’t.
Your customers don’t actually know what they want
Customers are experts in their pain, but they usually don’t do a good job of explaining their problems. Instead they tend to focus on what they think the product should do to solve them. Customers also find it very easy to say yes when you ask them the question “Will you use this?”, because it doesn’t cost them anything. However, when push comes to shove and you actually want them to use the thing that you’ve built, then often they don’t have time to evaluate whether that works for them, or to learn how to best use the new feature. Even when customers have the problem you are trying to solve, they may already be solving it in a way that is sticky for other reasons, and they simply have other priorities that you are not aware of.
When you commit to something you are also committing to the cost of not doing something
This is a major pain point with articulating a roadmap built around dates and features, particularly a public one, or where you are sharing features on a roadmap with your customers via Sales or your Customer Success team. You can often find yourself in the position where you have committed to building something, but because you are wrong at least half the time, you have committed to building the wrong thing, or there is a better choice available. Instead you find yourself stuck with the sub-optimal choice.
Estimating accurately is next to impossible
We call the process of software development “engineering”, so there is some sense that it’s similar to building a bridge, and that the steps are logical and should be amenable to project planning. But we’ve been building bridges for thousands of years, and the principles and structures are well understood. With software it’s likely the thing you are building has never been built before, and worse the thing that you are building will morph and change as you are building it. Also once a bridge is built it’s done save for occasional inspections and maintenance. With software you are going to be constantly modifying, refining and occasionally completely rebuilding. You are usually being asked to estimate on things that you aren’t going to end up building or will look so different in the final version, that much of that energy is wasted.
As an engineer in a product delivery team you have the least say over what it is you get to build and usually the most responsibility for when it doesn’t get shipped or doesn’t live up to expectations. You often feel like you are spoiling everyone’s fun when you try to be realistic, but it’s on your shoulders when the feature is late or underperforming. Engineers will be pushed in two different directions, either a desire to please by giving everyone the dates they want, or a desire to be realistic by putting sufficient buffer to account for the risk that something will go wrong.
This is not to say when you need to give firm commitments on delivery dates for a contracted feature it can’t be done, but if you start with a fixed scope and a fixed delivery date then that project, I guarantee that it will either be late or you are going to kill your team to deliver substandard product to meet a date that you should never have agreed to in the first place. Furthermore, you are going to spend some substantial future investment paying off the debt you introduced. Either the date or the scope has to be flexible — and probably both.
If you are going to commit to dates then you have to believe your Engineer’s padding and add some of your own. 30% is a good rule of thumb depending on how well defined the problem is an how much technical risk is involved. It also helps more if your teams have a singular focus. If they are working on multiple competing priorities and they have a significant maintenance or support burden then you are going to want to double everything.
Planning without learning leads to the fallacy of sunk cost
When you are sure of what it is you want to build, then planning seems like a good idea. Write out all the specs, do all the designs up front, put tickets in the ticket system for the product delivery team to work on. However, when you are wrong then this all becomes wasted effort. A problem I have frequently encountered is when the Product Manager and Designer have spent so much time thinking about an idea and getting excited about the possibilities that they find it very difficult to give up those ideas when they are wrong. This is the Sunk Cost Fallacy or to put it another way — clinging to a mistake, just because you spent a long time making it. If you are not learning and exploring while you’re planning there is a good chance you are wrong and simply wasting your time.
The Standard Objections
But… I know what many of you are thinking right now, something along the lines of: the business needs some certainty so it can plan, and customers need to know what’s coming or they will leave us for another competitor. If you are in a leadership position, or sales you are probably horrified at the idea that product teams will not necessarily build exactly the thing that you want, and they won’t be able to tell you exactly when it’s going to be done. These objections and more are the reason why top-down roadmaps are so popular with management, because they are a lie we can all tell ourselves that gives comfort about a possible future that denies the underlying reality.
The truth is that there are better ways that leverage the inherent uncertainty in the software product game. They allow us to reduce our risks by steering the product in the direction that best suits the business. They allow us to build products that customers not only embrace, but love. They prevent us from making expensive bets on bad choices, and measure our success not just in terms of the output we produce, but how it benefits the company's overall strategy.
Alternatives to Feature Roadmaps
Clear vision and strategy
If you can’t state in one or two sentences exactly what the core vision of your product is and the value it represents to customers, then it's hard to imagine that you really understand how to deliver a smaller piece of that in a way that is going to be coherent with the overall product direction. A clear vision looking 3–5 years ahead helps anchor and guide teams and acts as a filter through which the fit of new features can be evaluated. It also gives customers and idea of where you are going without you having to commit to the particular details of how you get there.
Of course 5 years might as well be a million years in a technology product company, and if you don’t have your game face on and are constantly delivering increments towards that blue-dot then your competitors will eat your lunch. For this reason a company needs a clear product strategy that outlines how they will get to the vision. This needs to be specific in the short term and will be necessarily vague in the longer term. If you go look at what major tech companies are talking about when they are talking roadmap to customers, it is typically not a bullet point list of what new features they are adding to their apps, but a public facing version of this product strategy. This again serves as a way for you to signal your intent to your customers and the market. Also by communicating our strategy clearly but keeping the specifics of how we solve them close we don’t empower our competitors to outbuild us.
What’s in a product strategy? It might leverage insights about where the market is moving, emphasize the areas of your product where you think you need focus, or where you might be weak against competitors. It might target a new platform, for example mobile where you can get uplift or new markets that can be explored. Importantly it should be open as to how those particular areas end up being embodied in product, and instead focus on what the outcome will be and how it will drive the product forward. Parts of this you will want to keep internal, parts of it you will want to share publicly.
Align with company goals and measurable objectives
A list of features delivered on the dates you planned does not tell you anything about whether those features will be successful in moving the needle for the company or the customers. If you are measuring output and not measuring outcomes then you are missing an opportunity to really succeed. Instead of a list of features you want your leadership to give you a plan for what goals they want to achieve and a way to measure them. A good framework for this is Objectives and Key Results (OKRs). Instead of telling you that they need this particular feature, they should explain what profit uplift they are looking for, and whether this should come from new or existing customers. They should explain that you need to outcompete this particular competitor or access this new market or acquire this strategic customer. They may think that equates to the same thing — because they believe that the features that have proposed is what will achieve those goals. But it’s the outcomes that are important and this gives product teams the necessary tools to analyze the most critical part of what a modern software product team should do.
Insights and Opportunities to be explored, not Features to be built
People mostly get into leadership positions because they are good at their jobs (we can all cite exceptions!). I find leadership skills tend to favor either people who are great at executing plans and managing people, and people who are great at inspiring and making entrepreneurial leaps of faith based on insights about your product, your market and your customers. These leaders, especially your CEO, Head of Product and Head of Marketing provide the spark and inspiration that drives the company vision. You want to listen to these people, because they got to where they are for a reason, and importantly they need to feel heard. Similarly your customers are the ones ultimately who are going to most benefit from the features you add. The trick is that usually these people don’t have the time, energy, resources or expertise to actually work out whether a particular insight and how it gets implemented is actually going to turn out to be a good for the business, the customers or the product. The tool that a product team can use is to evaluate these insights from your leaders and other stakeholders is called an opportunity assessment. The goal of this is to use product discovery (more below) to turn the spark into something that answers the questions (courtesy of the Silicon Valley Product Group):
- Exactly what problem will this solve? (value proposition)
- For whom do we solve that problem? (target market)
- How big is the opportunity? (market size)
- What alternatives are out there? (competitive landscape)
- Why are we best suited to pursue this? (our differentiator)
- Why now? (market window)
- How will we get this product to market? (go-to-market strategy)
- How will we measure success/make money from this product? (metrics/revenue strategy)
- What factors are critical to success? (solution requirements)
- Given the above, what’s the recommendation? (go or no-go)
Products are not built, they are discovered
Next and perhaps most critically is the approach you take to turn Insights and Opportunities into Product that meets the Company Goals, aligns with the Product Strategy and gets us closer to the Product vision.
In his book The Lean Startup, Eric Ries explains how companies can apply lean principles to technology startups. The key principle is the Build-Measure-Learn loop. We try something, run a small experiment and then measure the result and then learn from the experiment in order to inform the next experiment. An experiment may be as simple as running search ads to evaluate market need or adding a button for a new feature that just pops up message saying “We are thinking of adding this feature, tell us what you think?”. This process allows us to execute in short cycles where we iterate on the product opportunities, either by building prototypes or adding small features which we test with customers and get early feedback to determine whether they are the best fit. The idea is to use these experiments to achieve what is known as Validated Learning. These are specific measurable outcomes that allow us to determine that our ideas are panning out, that we are meeting what the customer is looking for. Validation is the most critical, but also the most difficult part of the whole process. In early stages this may be using marketing techniques to determine market need, or talking to customers or prospects and getting commitments based on mockups/early prototypes. Once your development is underway for B2C products you can leverage analytics and A/B testing to give you insights. For B2B where you may have fewer, higher value customers, you may have to use more intimate techniques such as user testing or early access prototypes to work out the kinks in your product. Importantly you want to learn what ideas you were wrong about early enough that you don’t waste a lot of resources that could be redirected into other opportunities and better achieve the company objectives.
Fix your time for discovery, not your scope
Managers are very big on estimates and project plans, and love a good Gantt chart. But as we have noted it’s hard if not impossible to estimate accurately, so these make people feel good, but they are really a fiction that ends up wasting a lot of time that could be better spent. There are a lot of techniques that do work well, relative estimation (story points or t-shirt sizes), but these tend to work better for smaller bursts when the problems have already been explored. If you don’t have a particular customer commitment then a good strategy is to pick a time window for exploring an opportunity, say 6 weeks and then use this time to iterate on your idea. At the end of this discovery process you will either have produced something that your customer’s love and you are ready for the next opportunity, you recognise you have more work to do and you are going to reinvest in another period of discovery and optimization, or your idea didn’t pan out and you have a good stack of learnings and perhaps new insights on what you can do differently (of course you should abandon early if this becomes obvious more quickly).
In addition, it’s usually much easier to estimate how much work you can get done in a period of time than it is to estimate how long one particular thing will take. This seems paradoxical, but it’s because you are usually comparing how many things of a certain size you can typically get done a particular period. This gives you a good sense of how many similar sized opportunities can be explored in a given time. What you might not be able to say with certainty is exactly what you will get done.
If you are forced to meet a date for a customer commitment, then you are going to need to either have done a good portion of the discovery work before you commit to that date, or you are going to have to negotiate with the customer to make it clear that the scope of your project might change.
But what do I tell my … when they ask me for the features on the roadmap?
These techniques are generally mature and have really revolutionized the way software products in particular are built. But no doubt not everyone has read the same things as you and you are likely to encounter some healthy skepticism. Here are some ways that you can approach those conversations with your various stakeholders.
Give me a clear vision for the product and a strategy for how you think we should get there. I want you to share your insights about product and customers but let’s agree on a core set of objectives and how we measure them. Some of those ideas will pan out, some won’t, but at the end of the month/quarter/year we’ll be accountable for the outcomes and how close we got to delivering the key objectives. Don’t worry I am going to touch base with you regularly to track how our progress is going and let you know what’s working and what isn’t.
Sales and Marketing
Sell your customers on the vision and share the strategy with them. You can use this to emphasize the strengths of the current product. We would love to understand what your customers are asking for, and what’s driving them to our product. Can we get involved in a conversation to understand what their problems are? While we have a lot of opportunities we are exploring, these are the ones in active development that we are ready to share with the customers, and we would love to get feedback from your prospects on how this will affect their decision to buy. We are confident enough in these opportunities that we can start exploring product marketing opportunities and test messaging with how customers and prospects will engage with this solution.
I understand you are curious about what’s coming next. Here is how our company envisions where our product is going, and specifically the areas we are looking to focus on in the shorter term. What are your priorities, not just for our product but what are your top pain points in your company in general? Do you already have solutions in place that would be hard to change? Would you pay us to solve them? How much? Would you like us to help you explore some of these problems so we can work out if it’s worthwhile for both of us.