Working Backwards Document - How We Ship Features: The Framework

Division
Product
Text
Our overview of how we use working backwards documents as our first and most important planning step for new projects.
💡
Our CBH Working Backwards Document is to be used for substantive Feature Specs (such as Matching Algorithm) and other Major Initiatives (for example, if we began an effort to get Joint Commission Accreditation today we would write a Working Backwards Document first).

With a nod to Amazon, who popularized this working backwards methodology, we’ve adapted it for use in our own corner of the economy. This is a living document, and I expect us to add to this document as we get more repetition writing these Working Backwards Documents and get more mature as a business.

Which projects necessitate a Working Backwards Document?

Writing a working backwards document is a mechanism to force higher quality thinking, but admittedly takes time, and thus is a tradeoff. Write a Working Backwards Doc if it checks at least 1 or 2 of the following:

  • Time: if the project will commit the company to more than two person-weeks of time (i.e. a feature that will take more engineering time than one engineer for two weeks) then write one.
  • Quality vs Speed: if the project is better launched one week later at a higher quality than if launched immediately, write one. (The obvious example of when not to write one is a bug in our mobile app, for example, in which case getting the fix out immediately is the priority). By the same token, if it’s better to launch immediately… are you launching it today? If not why not?
  • Second-order effects: if your project may impair other parts of the org, but for good reason (i.e. new credit underwriting policies that impact sales, account management, and billing) then write one. If your project may change the way our HCPs or HCFs behave in a way that’s costly to change back, write one.
  • Obvious to the reader: if your project is so patently obvious that if you emailed three people to do it without much description you would be confident they would deliver three identical solutions back to you, then you don’t need to write one. Otherwise, do.

The Working Backwards Methodology & Why We Use It

Working Backwards tests the customer value of your ideas and helps you crystallize what you want to do and why in customer-centric terms that anyone can understand. Customer-Centric is one of our Company Values. Note that some Working Backwards Documents will have internal-only customers, that’s fine, and identifying the true customer of your work is a key decision worth spending time on.

Three tools comprise the Working Backwards methodology and are each encompassed in this Working Backwards Document. Each tool is used to drive clarity into what we want to build and why:

  1. Future Customer Quotes As a customer-centric company, our product features and major initiatives are only successful if customers are delighted. So, we start at this end result: what will these customers say when this feature is out and they love it? We write these Future Customer Quotes before we start building anything to make sure that, as a customer-centric company, it's the right thing to work on.
  2. FAQs (from the perspective of each type of customer, as well as internal stakeholders) The list of Frequently Asked Questions is where we add meat to the idea described in the Future Customer Quotes. An FAQ includes questions that came up when writing the Future Customer Quotes - both stakeholder and customer questions. Put yourself in your customer's shoes and consider all the questions you would have and make sure these are included in your FAQ. Also, put yourself in the shoes of your stakeholder to address how it will get done and why it's important to do. The Future Customer Quotes and the FAQ are the business requirements.
  3. Visuals (for example, mock-ups if this is for a feature in our CBH App) Sketch out what the experience will be like for customers (or users if it's something you're building for internal users). How will customers discover it, and what will they want to be able to do with it? What will it actually look like? Visuals might be a drawing on a whiteboard, a workflow diagram, a wireframe, or several of these.

As a set, these three sections comprise the Working Backwards Document. To enforce clear concise thinking, the first two sections together should fit within a maximum of six pages (but can be shorter), and Visuals are often added as an appendix that can be as many pages as needed. This document is the currency we use to share, test, and bring our ideas to life for customers.

Section 1: Future Customer Quotes

As the first of the three sections of the Working Backwards Document, and the first of the three tools we use in Working Backwards, start here. Who is the customer?

  • If it’s an HCP, what are some quotes you’d hope to hear from them when they call in after this feature is out? What are some text messages you hope to get to Support because they love this feature?
  • If it’s an HCF, who inside that HCF (admins, scheduling coordinators, directors of nursing, etc) will care and what do you hope they’ll say?
  • If it’s internal stakeholders (such as NAMs, CSMs, SDRs, the Director of Sales, etc) who is it exactly, and what do you hope that person(s) say after you’re done?
  • If it’s the press (and the press is rarely our customer) what article do you hope they’ll write about your feature once it’s done and in production? Write that article yourself in this section as an anchor point for your design.
  • Note: oftentimes the customer is more than one type above at the same time - this is the magic of marketplace businesses such as ours.

A good Future Customer Quotes section answers the following very clearly:

  • Who is the customer?
  • What is the problem or opportunity? Again, from the eyes of the customer (internal and external)
  • Is the most important benefit clear? (Also, why would a customer care?)
  • Can the customer explain it to a friend easily and succinctly in a way that quickly conveys what it is and what the value is? i.e. what do you expect customers to say to their friends when they’re telling them about this cool new thing you built? Write that out in this section.

As part of the Working Backwards methodology, the Future Customer Quotes is a tool we use to start with the customer and work backwards. We leap into the future to imagine what we want a customer to say or feel when we launch. Writing Future Customer Quotes drives customer focus into our work before we write a line of code or commit resources. It helps us spend the right amount of time agreeing on who our customers are and what they care about.

Once you have written the Future Customer Quotes, you have an anchor that describes in a simple way what you're doing and why a customer would care - in their own words! Anyone can read your Future Customer Quotes and quickly get behind your idea. The FAQ section (below) accompanies Future Customer Quotes, and lists with answers to the questions that would be asked after reading the Future Customer Quotes or discussing the project.

Section 2: FAQs

FAQs are short for Working Backwards Frequently Asked Questions. We use FAQs to channel our customers and our stakeholders to define how something will work and how we will execute. Solid FAQs describe workflow, business rules, requirements, and scope. Solid FAQs uncover risk and help us decide if a project is the right investment – because big or small, every project is an investment.

Put yourself in your customer's shoes and consider all the questions you would have. Also, put yourself in the shoes of your stakeholder to address how it will get done and why it's important to do. An FAQ includes questions that came up when writing the Future Customer Quotes. It includes questions that people asked when you shared early iterations of this Working Backwards Document.

A well-written FAQ is important to the Working Backwards methodology because:

  1. Good FAQs help us vet an idea. While we may fall in love with the idea when writing the Future Customer Quotes, the FAQs help us get into the details of how it will work for customers and how we'd execute.
  2. Writing FAQs uncovers risk and helps us assess the feasibility and complexity of what we want to do.
  3. Good FAQs capture the decisions made for a project along the way. It's a living document.

Good FAQs help make team meetings more efficient. Ideally, all the key questions triggered by the Future Customer Quotes are already answered in the FAQ thus eliminating the need for any discussion. In most cases, the meeting can be focused on the few remaining unanswered questions.

Tips for Writing FAQs

  • Jumpstart the FAQs by sharing your Future Customer Quotes with other people who know very little about your space and capturing their questions.
  • Include your customer FAQs, not just internal stakeholder FAQs.
  • If you have more than one set of customers, include FAQs for all customer sets. At CBH this is almost always the case.
  • If you ask a yes-no question, the answer should be yes or no.
  • If you don't know the answer, that's okay; include the question and answer the question by saying, "We don't know yet." Bonus points for explaining how and when you'll figure out the answer.
  • Which questions do you want to answer the least or will be the toughest to answer? Include those, or you may get asked those questions by reviewers and have to answer them anyway, just with no preparation.
  • Imagine what your customers will be most frustrated about or surprised by and include that as an FAQ.

Default-mandatory Questions in every Working Backwards Document at CBH:

We have found with experience that these questions are almost always really useful to answer in the FAQ. If you skip one of these to make room for another, it’s okay, just have a good answer as to why.

  • What is the problem? Set up the problem & context so that we can use our network to get ideas/experiences/learnings from outside the building (including of course our customers).
  • What’s the metric that you most want to move - i.e. if you had a magic wand, and didn’t have to worry about how to move it, what would be the metric you want to move? (Maximum two, in situations where there are two countervailing forces such as the Number of New Customers Per Month and Customer Acquisition Cost)
  • Is this the best customer experience you can think of, unconstrained? If not why not? What constraints (internal, external) prevent you from your best customer experience idea?
  • Premortem: what’s the most likely cause of failure if this fails?
  • Any security, regulatory, or privacy implications?
  • If this feature requires some internal training, what’s an example of a training document (or even key snippets from it) so we can see how easy/hard it will be to train internal customer-facing people to represent/answer questions about this feature?
  • Rollout plan? What if it goes badly during rollout - what mechanisms have you put in place to tell (especially if by that time you’ve moved on to another project), and what will we do as a result?
  • What’s the plan for training internal teams and educating external customers? Including sample content? Almost all complex features that directly impact the customer experience will need sample Healthcare Professional Email and/or Text Message and/or samples of other methods of Notification & Education.

Pitfalls to Avoid in your FAQ Answers

  1. Where you make a recommendation or list the options, don’t actually say “this is what we will do”. Remember you’re not advising someone else writing the WBD; you’re writing the WBD.
  2. Use of adjectives: “more”… how much more? Specifically, what’s the number? Is it dollars? A billion? A penny? Remember an engineer will have to write code from your document and he or she can’t write “more” in the code — they can only write a specific number.
  3. Not actually answering the question: if the FAQ is a yes/no FAQ the first word of the answer should be, yes or no. If I stop reading your FAQ answer after the first question I should have gotten 80% of the answer.

Appendix of common FAQ questions to jump-start your FAQ section

These starting FAQs drive clarity into what you want to do and why. As the idea crystallizes, Future Customer Quotes will take a back seat and you’ll focus on the FAQs because you’re ready to get to the next level of detail. There is an art to asking good questions at the right time. Good FAQs are not every question that might be asked; instead, good Working Backwards FAQs help us determine if it’s the right project and the best plan.

  • Have we tried to solve this in the past? If so, what were those efforts and why didn’t they work? What if anything can we learn from them?
  • How would we fix this with only a spreadsheet and/or other existing tools?
  • How could we do it better with some Engineering?

Customer FAQs—Put yourself in the customer’s shoes

  1. Who is our customer?
  2. What is it and who should use it?
  3. What’s going to excite me about this?
  4. What might disappoint me?
  5. How will I discover or find this?
  6. Do I have to learn or unlearn something?

Stakeholder FAQS—Anticipate your stakeholder’s questions

  1. What is the problem we’re solving?
  2. Why is this customer problem or opportunity important right now?
  3. How will we know if this raises the bar on customer experience?
  4. How do we know if we’re succeeding?
  5. What’s provoking the most internal debate?
  6. How does this fit into the overall portfolio?
  7. Why are competitors not doing it? If they are, how have they succeeded or failed?
  8. What are the untested risky assumptions we have? Which are the ones whose risks can be lethal for success and which are the ones we can live with being untested right now in favor of speed?

FAQs for a new business

  1. Is this a big idea?
  2. Can we win?
  3. If we can win, is it worth it?

More FAQs: This list is not exhaustive, and there will be unique FAQs for your project. Use FAQs that make sense for your project.

More Customer FAQs

  1. How does it work? (i.e., features and functionality)
  2. How is this different from what we offer today?
  3. Can I use this on all of my devices?
  4. What do I need to know? (i.e., policy change)
  5. What do I need to do before I can use it? (i.e., sign-in)
  6. Does this cost money? If so, how much does it cost and how will I be billed?
  7. Are there things I need to manage or keep track of?
  8. Which features will I care about the most?
  9. Which features will I care the least about?
  10. I’m expecting to be able to do <insert a task>, why can’t I do that?
  11. What happens if I use it the wrong way?
  12. What if I change my mind and no longer want to use this?
  13. Why wouldn't I just use this/do this on <insert competitor>?
  14. Where can I learn more about how to use this?
  15. Who do I contact if I have a problem?
  16. Can I get a refund if I’m dissatisfied?
  17. What if I have suggestions about how to make it better?
  18. If there are changes to how it works, how will I find out?
  19. How will you use the information you’re asking me to provide? 20. Is this something I can share with my friends?

More Stakeholder FAQs

  1. Are there trustbusters?
  2. How does it work? (i.e., the technical details)
  3. Is this a hard-scaled or scrappy solution? Why?
  4. Do you love it?
  5. What would it take to move faster?
  6. What's the most pervasive constructive feedback from customers? And are we addressing it?
  7. Will this be available globally at launch? If not, what’s the plan for making this globally available?
  8. Are we following global standards? Do we know why we are making exceptions?
  9. What is the launch minimum? Why?
  10. Is the solution self-service? If not, why not?
  11. Are we overpromising? How can we set appropriate expectations and over-deliver?
  12. How are we going to drive adoption?
  13. What's the experience for the customer that is not the target customer?
  14. Where does the bad experience show up in our metrics?
  15. Have we made sure we're not treating our best customers worse than other customers?
  16. Is there potential for fraud or risk?
  17. What resources are needed to do it?
  18. If we had more resources, what would we do?
  19. Is there a closed-loop process that gauges customer satisfaction?
  20. How will we ensure this is still working correctly a year from now?
  21. What's the most complex thing about the plan? How can we simplify?
  22. What data are we monitoring and why is this data important?
  23. Who are we benchmarking against and why?
  24. What did you use to sanity-check your idea?
  25. What are our competitors doing in this space?
  26. What's the headline you don't want to see?
  27. Are there areas you are worried about?
  28. Have you had a usability study?
  29. How are you going to measure success?
  30. How are you going to test this?
  31. How many customers will use this product/feature?
  32. So far, How much effort have you put into this?
  33. What are your core metrics?
  34. What different things have you tried?
  35. What is your biggest risk?
  36. What is your plan for mobile?

Section 3: Visuals

Attach as many pages as necessary as an appendix to the first two sections of this working backwards document to visually illustrate (oftentimes with captions and call-outs) what your product will look like and/or how key algorithms work (state diagrams) so that stakeholders across the company can quickly understand what you have in mind.

For example, here’s a visual of the Working Backwards methodology:

Note that you don’t have to “check-in” with anyone between stages - unless you want help / to bounce something off of folks (like your manager) you can just go go go and run this process yourself. But as the diagram says, think of it as having these three steps, even if you’re doing them all “at once”.

The Working Backwards Document Review

We use the Working Backwards Document to greenlight initiatives.

The Purpose of the Review

The review performs three core jobs:

  1. Ensure there are no significant problems with approach
  2. Aggregate constructive feedback and/or ideas
  3. Make cross-functional stakeholders aware

Executing the Review

  1. If it’s not there already, ensure your initiative is on the Launch Calendar, the default status for your WBD should be “In Review”
  2. Send your WBD to your manager to ensure that the document is ready for a live group review – all WBD reviews are synchronous
  3. After you and your manager agree the WBD is ready for a live review, notify the #pd-reviews channel that you’re scheduling a live review for your initiative
  4. Throw an event on the calendar to schedule the review
    1. Who to invite?
      1. Select members of the Product team – ask your manager if you’re unclear
      2. Your engineering partners
      3. Bo if appropriate
      4. Any cross-functional stakeholders that you think need to attend

Outcomes of the Review

Your manager will work with you to distil the feedback from the live review and determine next steps (ie we’re not giving reviewers veto power, just the ability to provide thoughtful and sceptical feedback).

So, there are three possible outcomes for your WBD after your Review:

  1. In Review (yellow light)
  2. Ready for Execution (green light)
  3. Rework Needed (yellow light)
  4. Initiative Halted (red light)

You’ll be expected to update these statuses will be updated in the Launch Calendar.

A “Rework Needed” outcome might not necessitate another live review – please work with your manager to determine what’s needed as next steps.

How your Working Backwards Document will be reviewed

Stakeholders will come to Working Backwards Document Reviews to ask probing questions, many of which are already in this template! You’ll succeed based on 3 things:

  1. Did you ask yourself all these questions and even harder / more specific ones (and write down the answers) before you arrived at review?
  2. The quality and thoughtfulness of your answers.
  3. How many reviews it takes to get the document to a good state.

After the Working Backwards Document Review

After passing the review, there are additional artifacts you need to provide:

  • Project Plan
    • A SMART plan that is specific to your initiative – filled with specific action items, owners and dates that make up “executing” on your initiative
      • Here’s a good project plan: Dynamic Pay 1.0 Project Plan (though no names because Jaron was the owner for all items)
  • Dashboard
    • Every WBD should have a metric or set of metrics we’re looking to move
      • Build out the dashboard so stakeholders can follow along

The Spirit of The WBD

Just as important as the document itself is the spirit in which the document is written.

The FAQs are a powerful tool to sharpen your thinking, but you only maximize their impact if you engage with them in an intellectually honest way. While writing, have a debate with yourself so that you can pre-empt any concerns (in the form of better plans) and course-correct as early as possible.

Great working backwards documents evolve over several iterations – which means it takes courage to debate yourself while on version 5 or 6. You might even end up sunsetting a project yourself because you’ve come to realize that either the initiative is misguided or now isn’t the right time in the context of the broader strategy. That’s a totally acceptable outcome – and it’s likely healthy if every once in a while you end up killing your own project.

A lack of intellectual honesty in your writing is easy to spot and is grounds for preventing your initiative from proceeding.

Conclusion

By adopting the working backwards methodology as described in this document we will maintain our customer-centric thinking as we scale. By producing Working Backwards Documents we will get everyone on the same page. By reading and discussing each initiative using its Working Backwards Document we will decide which projects to work on, and know when the thinking has been sufficiently done to embark on those Investments. By doing all three we will drive more customer value more quickly and more often, by starting with the customer first and working backwards