Here’s a question I received recently: What kinds of indicators tell you if you are going in the right — or wrong — direction in your work?
I decided to answer it here, as a public Q&A, because it merits crowd-sourcing.
One way to measure success or failure is to take a systematic approach, as we do when we pursue Quality Improvement (QI). Dr. Mike Evans has a new video explaining the concept:
The key questions can be applied at an individual level (as in: I want to cut down on my smoking) or at an organizational level (as in: we want to improve rates of hand-washing in our hospital). Here they are:
Question #1: What are you going to improve and by how much?
Question #2: How will you know if a change is an improvement?
Question #3: What changes can you make that will lead to the improvement?
But what if you want early indicators that you are geared toward success? What if you can’t afford to wait for all the data to come in — or you have flexibility and can shift course if the weather vane swings toward failure?
The QI video explores another concept that is helpful here: Innovation Fatigue — that moment when you realize that your team is NOT as excited as you are about making a change.
Hunter Gatewood described in a 2008 talk how, when presented with an innovation, some people are like happy dogs in a pile of sticks and others are like cautious cats, watching and waiting to see what blows up in the dogs’ faces. A lesson I took was to watch for both types of people on my team. I could count on the early adopters (the happy dogs) to try new things, but I always knew we were on the right track when my late adopter (cautious cat) colleagues signed on.
So that’s another measure: know your team and use them as a barometer.
Another technique is to employ scenario planning, which I find to be really fun and useful. You first carefully assess the current field you are playing on, including all the possible factors that could change it — the knowns and the unknowns. You then try different combinations of possible outcomes — if X happens, then we can expect Y to grow — and write narratives to fit each one. (I found 3 resources for those who want to learn more: short, medium, and book-length.)
Now I’d like to hear from people about their personal “right track/wrong track” indicators. Wearing your failures on your sleeve may be fashionable in Silicon Valley, but it can be difficult for individuals — or other industries — to acknowledge and own it publicly.
When and where have you seen a well-told failure narrative? How might we make failure an authentic part of the public conversation about health and health care?
Please share your thoughts in the comments. And remember: if you don’t want to use your real name, it’s OK to post under a pseudonym or to email me directly (susannahRfox at gmail — note the R, please).
Hunter Gatewood says
Susannah, fun to see this mention of our first contact years ago. So many things to say, the topics introduced here cover most of my work!
To start at the top, with the question about which indicators are best: For quality improvement, you need indicators where you can get the data regularly, frequently (like daily, weekly, monthly) and where the data are meaningful for the work team and the patients or customers.
Quality improvement is largely process improvement, so the indicators are chosen to show us how a process performs. It’s different work and a different use of data than what we use in research work (medical, public health, geology, hey!). The indicators tell us if our process is producing the results we are responsible for delivering: Are all our patients with diabetes as healthy as they can be? (One indicator: % patients with blood sugar readings at healthy level) When someone gets out of the hospital, do our services help them not go back again right away? (One indicator: % patients discharged from hospital each month with care team contact within 3 days.)
Good indicators show you what’s happening and so start the drum beat for doing things differently to get a better result where our #s are not good.
Example: A good primary care performance indicator that impacts patient/customer happiness is “cycle time” which is the total minutes it takes for a patient to complete an office visit. Many providers make patients spend 90 minutes or more to complete a regular visit: sign in, sit, get into the exam room, sit, see the provider, sit, check out. When an office staff team realizes there are lots of ways to improve this situation with improved teamwork, you can imagine the innovation (to get to the other big topic raised here) that is unleashed. When people see the gap between current and ideal performance, the indicator drives the improvement work.
To get back to which indicators are good to use, there are “process measures” like cycle time or percentage of patients in a certain age range for whom we have a blood pressure reading recorded, “outcome measures” like percentage of patients (of those we have a record) whose blood pressure is in control, and “balancing measures” that are measures used to monitor our changes’ effects on the larger system. As an example of a balancing measure, we don’t want to add so much good self-management support to a primary care visit that it takes 4 hours for a patient to escape our help, so we can use cycle time to make sure the patients’ experience and basic daily clinic operations survive all our great ideas.
Leaders of change spend a lot of time creating the right mix of indicators, so that everyone cares about what’s going up and what’s going down, and what’s staying the !$%&*!! same despite all our efforts.
Hope this is interesting to you and your readers. Cheers!
Susannah Fox says
Wow, thank you!
Favorite line: “we don’t want to add so much good self-management support to a primary care visit that it takes 4 hours for a patient to escape our help” <-- yes, indeed, that would annoy me! And yet if I got to see all my (or my loved one's) clinicians and consultants at one time, I'd be grateful. That's what I have learned from Erin Moore whose call for care coordination is undeniable in its logic (see below for a link to her blog). One of my motivations for asking these questions is to spark a conversation about how we can talk about our setbacks in a way that's honest, as you do in your work, and not in the way we all would describe our "faults" in a job interview, i.e., "I work too hard and I'm too meticulous, those are my biggest faults." Unfortunately I think there is too much of the latter in health care, a glossing over of what is really going on. I'd love to start a list of readings or inspirations in the comments, too. For example, The Checklist Manifesto by Atul Gawande is a book worth people's time if you haven't already read it. I love how he brings in examples from aviation and skyscraper construction. What other industries can we learn from? Erin's post: Talk is cheap http://66roses.blogspot.com/2014/12/talk-is-cheap.html
Susannah Fox says
I intend to use this post as a magnet for resources and ideas — yours! please comment — and as a repository for things that I see or want to note here. Fair warning for those who subscribe to comments 🙂
The genesis of this post was a hallway conversation with Beth Toner at the Aligning Forces for Quality conference in Washington, DC, in November:
Idea: a health care conference where we talk only about our failures. See: FailCon http://nyti.ms/10IophR
And lots of people RT’d and replied — a few quotes (you can click the link above to see who said what, etc, I just want to jot down the notes):
– love it! It was help reduce the # of echo chambers conferences. :/
– we always say “fail fast,” but we never say …”then talk about it.”
– I’m sure MDs would be all in for this bc we have M&M (Morbidity & Mortality) conference regularly.
– Yup. ++impt for healthcare cuz we fear failure more
– You could get some academics easily: failed proposals and botched lectures
One person connected me with my colleague Colleen Young’s panel at Medicine 2.0:
Fail Better, Fail Faster, and Learn Together
What else can we add? What is your initial reaction to the idea of “failure” in health and health care? And again, where have you seen an authentic discussion of it?
Michael Hoad says
Susannah: How do we talk about failure in a world where there are public conversations but also private conversations? Where risk takers are often undermined out of sight … “y’know, patients don’t want telehealth … just ask them,” or “y’know, they took a bunch of medical students based on emotional intelligence, and now those poor kids are all basket cases …” Yes, I’ve heard both of those statements, sotto voce. Or, you hear the private conversations about physician suicide, Type 1 diabetes teens who are mortally scared of hospitals, folks who think their doctors recommend procedures because they don’t want to leave “money on the table.” So … my only minor suggestion comes from a thought I heard from a RWJF “Pursuit of Perfection” presentation a decade ago. Someone at a children’s hospital in Cleveland (I don’t remember which) said that after working with a patient advocacy board, they came up an initiative something like: “All conversations ABOUT a patient should be conducted WITH the patient.” I think we have to remember that private conversations can poison innovation. Share them. Break the definition of a “conspiracy” – whisper if you have to – but tell the people involved what you think. It’s only together that we “find the many ways” to overcome the fears, roadblocks or jealousies people mutter behind backs. All conversations about innovators should be conducted with the innovators.
Susannah Fox says
Incredible comment. Thank you so much!
Side thought: children’s hospitals — and pediatrics in general — is so often at the forefront of innovation in quality improvement. Why is that do you think?
Susannah Fox says
Your comment sent me in search of a citation for the Salzburg resolution: “Nothing about me without me.” Here is one:
Michael Hoad says
Love the PeoplePower reference.
I found the Pursuing Perfection report I’d heard 10 years ago … at Cincinnati Children’s. They call it “transparency” in this report. I remember the phrase “all conversations about the patient should be conducted with the patient,” which would be revolution for medical education (why should the med student report to the attending in another room?).
Michael Hoad says
Oh wow … Lucian Leape’s Institute just used Cincinnati Children’s Hospital and its RWJF Pursuing Perfection grant as the key case study in “Shining the Light,” using transparency as a way to better “see” safety at all levels:
Sherry Reynolds - @cascadia says
One well established method already used in healthcare patient safety circles are “Safe Table” exercises and it might be a forum that you could expand to areas beyond patient safety.
“Safe Tables offer the opportunity for hospitals to work together to address patient safety issues. These protected learning collaboratives provide a place for staff to share best practices, seek advice from peers, and learn from national experts in a confidential setting. Safe Table meetings are held both in person and over web conferences.”
Although the Washington State Hospital Association is the one I am the most familiar with (I was the first patient advocate to sit on the State WSHA Patient Safety Committee) http://www.wsha.org/0518.cfm (you have to hunt a little bit to find the links to webinairs and pdfs) there are some great resources at the Institute for Healthcare Improvement (where your video is from) as well.
Many of us are already trained in the PDSA improvement cycle and use Lean Process for healthcare frameworks. The Joint Commission Center for Transforming Healthcare has some great resources on “robust process improvement” as well that most of us in healthcare transformation utilize. http://www.centerfortransforminghealthcare.org/about/rpi.aspx
Another local (I live in Seattle) but nationally recognized resource is the Virginia Mason Lean Healthcare Institute – http://www.virginiamasoninstitute.org/ which has trainings and free online exercises as well
FYI – Many people know me for my passion for patient centered healthcare design but few realize that it is largely the outcome of system level changes (vs a purely grass_roots approach of asking individual patients to change their behavior) and requires process improvement tools and years of experience in root cause analysis and systems level change as well.
Susannah Fox says
Thank you! I can’t wait to dig into these links!
Susannah Fox says
On Dec 2, Anil Dash tweeted:
Do any startups ever do company updates that aren’t either “We’re killing it!” or warmed-over shutdown notices like http://ourincrediblejourney.tumblr.com ?
Click on the link to read all the replies:
Then he practiced what he preaches and posted an honest assessment of how his own startup, ThinkUp, is doing as a business:
That is the kind of authenticity that scares some, emboldens and inspires others.
Reading it, I thought of Paul Levy’s Running a Hospital blog, another honest, ongoing assessment of how things were going while he was CEO of Beth Israel Deaconess Medical Center.
What other examples do you see of an individual or an organization openly accounting for their failings?
Paul Levy says
Great topic, Susannah. In my simplistic view, unless an organization is willing to publicly acknowledge its failures, they will recur in some form or another–and the organization will lose a chance to learn and improve.
I operated on this basis while CEO of Beth Israel Deaconess Medical Center.
Sometimes failure was habitual, as in this case: http://runningahospital.blogspot.com/2007/04/i-want-to-be-proud-but-i-am-not.html
Sometimes failure was apparently isolated, but often the result of overlapping systemic flaws in the way work was organized: http://runningahospital.blogspot.com/2008/07/message-you-hope-never-to-send.html
But the discussion must be in the context of the overall purpose of the organization, and best employed when an audacious goal has been established for the place. http://runningahospital.blogspot.com/2008/01/aspirations-for-bidmc-and-bidneedham.html
As Peter Senge has noted, when you combine an audacious goal with publicly available progress towards that goal, creative tension is established that–like a big rubber band–pulls the organization up to the target.
Susannah Fox says
Paul, I am so honored by this comment — thank you!
— For those who, like me, fell in love at first sight with Paul’s final comment, quoting Peter Senge, I found these background documents:
e-Patient Dave says
2. My first question is how to get a video like this created.
3. SO much of this “how to achieve change” applies just as much to patients as to clinicians.
4. O,M,G. What a STUNNING summary of a large body of work – in less time than most TED Talks.
5. The origin of “nothing about me without me” seems to depend on context. In medicine per se, Healthcare in a land called PeoplePower seems to be recognized, via nurse Diane Rampling, at that Salzburg Seminar. This comment on e-patients.net four years ago(!) said the idea has been traced back to the disability rights movement in South Africa – basically, “Thanks, but don’t go deciding what rights I need without asking me what I think I need.” The book cited starts “I first heard the expression ‘Nothing About Us Without Us’ in South Africa in 1993.”
At the same time we discussed it on e-patients.net, I blogged specifically about the “PeoplePower” paper on my own site, with a number of my favorite quotes, e.g. –
“Efforts to improve care might take strikingly different shape if patients worked as full partners with health professionals to design and implement change”
“In [the mythical land of] PeoplePower, the Internet-based patient record exists as a single version that resides nowhere but is available everywhere. … Not only do patients have nearly complete access to their medical record, … they also write in it – elaborating, tracking …, correcting mistakes,… and at times suggesting both diagnoses and treatment plans.”
“On People Power hospital floors, families help not only their loved ones, but also volunteer to aid many patients in the hospital…”
“Primary care clinicians serve as ‘gateopeners’, in striking contrast to their former role as ‘gatekeepers.’”
6. O,M,G. Thank you for this video.
Susannah Fox says
Well, I’m thrilled. Really. To introduce you to Mike’s work is my honor. Gorge on all his videos over the holidays. You won’t regret it!
Mike Evans says
thanks dave and susannah! haven’t seen that many OMG’s since using my 17 yo daughters computer last week….
Dave- our whiteboard series (www.youtube.com/docmikeevans) takes advantage of a couple of emerging themes that might resonate…
– we storify evidence and imbed in the relationships of care. Medicine thinks that is largely clinician or org to patient but the biggest group is clearly peer to peer (loved one to loved one, friend to friend). A concept that susannah knows so well. We try and make our vids infectious (bacterial or viral!).
– It has been incredibly helpful having a team that is not based solely in healthcare.. we do engage clinicians and researchers.. but most of our material is driven by creatives (filmmakers, illustrators, designers, etc.. ) and patients. We sort of overlap design thinking with critical thinking. We try and have our questions coming from patients- not just going to where we have perfect answers.
– our biggest feedback is “thanks for not being pushy”. we just lay it out and let people overlay their own values and experiences on our work. Our stuff doesnt work for everybody, but it does respect the user.
– Being creative requires an appetite for tweaking and failing well. This is not common in medicine where we often see so much focus on what could go wrong. So to use your example of EHR/PHR… We focus on hurdles such as privacy, potential patient misunderstanding, legal, remuneration– all important for sure… but this focus on silos undermines experimentation with what is so important – an informed & incredible user experience. As we said in the video- thinking a bit less about what’s the matter and a bit more about what matters.
thanks for watching… happy to discuss more.