
I’m looking into public perceptions of patient safety as a possible research topic and have run up against a question I can’t answer. Can you help?
Popular and academic citations of the 1999 Institute of Medicine report, “To Err is Human: Building a Safer Health System,” nearly always mention one attention-grabbing analogy:
Perhaps its most famous contribution was the extrapolation of the Harvard Medical Practice Study data and the Utah and Colorado Medical Practice Study data, which led to the famous estimate of 44,000 to 98,000 deaths per year from medical errors (the equivalent of a jumbo jet a day). [emphasis added]
– AHRQ (see also: Quality and Safety in Health Care, O’Reilly Radar, and Wachter’s World to pick three more from thousands of citations)
Yet the original report doesn’t actually close the loop and use that “jumbo jet a day” analogy, nor does the press release. Here is the excerpt from “To Err is Human” containing the damning estimates:
Extrapolation of the results of the Colorado and Utah study to the over
33.6 million admissions to hospitals in the United States in 1997, implies
that at least 44,000 Americans die in hospitals each year as a result of preventable
medical errors. Based on the results of the New York study, the
number of deaths due to medical error may be as high as 98,000. By way of
comparison, the lower estimate is greater than the number of deaths attributable
to the 8th-leading cause of death.
And here are the two instances of the word “jet” in the original report:
Although the risk of dying as a result of a medical error far surpasses the
risk of dying in an airline accident, a good deal more public attention has
been focused on improving safety in the airline industry than in the health
care industry. The likelihood of dying per domestic jet flight is estimated to
be one in eight million. Statistically, an average passenger would have to
fly around the clock for more than 438 years before being involved in a fatal
crash. This compares very favorably with a death risk per domestic flight of
one in two million during the decade 1967–1976. Some believe that public
concern about airline safety, in response to the impact of news stories, has
played an important role in the dramatic improvement in safety in the airline
industry.
[…]
The risk of dying in a domestic jet flight between 1967 and 1976 was 1
in 2 million. By the 1990s, the risk had declined to 1 in 8 million. Between
1970 (when the Occupational Health and Safety Administration was created)
and 1996, the workplace death rate was cut in half. Health care has
much to learn from other industries about improving safety.
Yes, this is a small point, but I’m a geek. I like to get things right. I don’t cite something if I can’t verify it myself. So, before I spend another hour trying to track it down, I thought I’d share the question with the health geek tribe: Who created the “jumbo jet a day” analogy?
Featured image: Flight by Pam Morris on Flickr.
I don’t know who first said it in medicine, but I first heard (and used) it in the 1980s applied to tobacco deaths. My friend Thom Hartmann ran a company called The Newsletter Factory, back when desktop publishing was new; he taught day-long seminars teaching people how to be newsletter editors. One thing was “Take unimaginably big numbers and break them down to something people can relate to.” That was one example.
(As a side note, I was one of their part-time instructors, and that was where I learned how to be totally responsible for an audience’s experience: it was a solo gig – get up there and talk for a full day, nothin’ but you, responsible for their being ecstatic, no matter who they are and why they came there. Talk about learning to dance with the music…)
Nice!! That’s very helpful. There’s nothing new under the sun, as my grandmother would say.
But I’m still curious about the IOM meme-creation story. High hopes for a health geek historian to come forward.
This just in from @BrandonAylward via Twitter:
The odds now of dying in an airline crash are 1 in 45 million
http://www.dailymail.co.uk/news/article-2278382/Why-blasting-space-shuttle-safer-walking.html
Hi Susannah, I believe it was first mentioned in a paper by Leape. http://jama.jamanetwork.com.proxy.lib.umich.edu/article.aspx?articleid=384554
He quotes WE Deming, but perhaps this was the first time that the analogy was mentioned? I will send you the pdf by email. Thanks! Joyce
Thanks, Joyce, for this comment and for emailing the PDF.
Here’s the salient quote:
“Also in 1991, the Harvard Medical Practice Study reported the results of a population-based study of iatrogenic injury in patients hospitalized in New York State in 1984. Nearly 4% of patients suffered an injury that prolonged their hospital stay or resulted in measurable disability. For New York State, this equaled 98 609 patients in 1984. Nearly 14% of these injuries were fatal. If these rates are typical of the United States, then 180000 people die each year partly as a result of iatrogenic injury, the equivalent of three
jumbo-jet crashes every 2 days.”
I’ve heard from two other colleagues via email who echo Dave’s memory that this “jumbo jet crash” analogy may have many origins, but this is a nice citation to have.
This is fun. I did an archive search. Leape’s comment was picked up — accurately — in the month’s following by everyone from the Washington Post to the Boston Globe. By March of 1995, Newsweek had slightly altered it, dropping the “three-every-two” formulation to just “a jumbo-jet crash every day.”
The analogy seems to have disappeared until “To Err,” at which point it began cropping up as the daily crash stat, generally without crediting Leape, Harvard or JAMA. By 2006, you can see it mis-attributed to IOM every once in a while. The last time I can find where it was correctly cited *and* properly attributed was in 2005. (By Bob Langreth, then at Forbes.)
So geeky, so awesome. Thanks, Brian!
Dear Joyce: Can you provide the DOI or PM ID for this article or the formal reference. your link as provided is not accessible to us normal living folk
JAMA. 1994;272(23):1851-1857. doi:10.1001/jama.1994.03520230061039
Does that help?
This is incredibly helpful in some research that I am doing for an educational project at the American College of Cardiology. I was pulling my hair out, trying to find the reference in “To Err is Human,” but it’s not there at all. Excellent work, and thanks.
This is very useful for me, too. Thank you!!!
I heard a new one last weekend. But first, to answer your original question I googled 44,000 deaths “jumbo jet” and found a medical malpractice site that links to this 1999 article in Public Health Reports, an interview with revered safety figure Lucian Leape, in which interviewer Mark Yessian tells Leape, “…you’ve indicated… they’re about equivalent to the death toll from three jumbo jet crashes every two days.”
So I googled lucian leape jumbo, and presto I think: Leape’s 1994(!) article in JAMA, Error in Medicine. (It’s at the bottom of the first column.)
Last weekend I attended a conference in a different field, HRO (High Reliability Organizing). Wikipedia defines a high reliability organization as “an organization that has succeeded in avoiding catastrophes in an environment where normal accidents can be expected due to risk factors and complexity.”
Conference organizer Daved (yes Daved) van Stralen used a new plane-crash calculation for the latest American deaths figure of 440,000/year: it’s as if the entire Southwest Airlines flight of 737s crashed six times a year.
Numbers like that are just beyond comprehension. I mean, that means 73,300 deaths every two months.
___________
For citehounds:
Wikipedia says Southwest’s fleet is now bigger – the numbers work out to over 100,000 total capacity, so the whole fleet would “only” need to crash every 2.7 months.
The 440,000 deaths number comes from the September 2013 article in Journal of Patient Safety A New, Evidence-based Estimate of Patient Harms Associated with Hospital Care.
Those links are not working for me. The paper is L. Leape (1994) ‘Error in Medicine’ JAMA, 272(23).
The intriguing thing from reading this quote is that I was under the firm impression that the numbers in ‘To Err is Human, 48000 – 98000, were for the whole US. But Leape, referring to the NY study, where death rates were higher than in the Colorado and Utah study, says that the death rate there could be extrapolated to 180,000 for the whole US.
It’s important to be clear that these numbers are for deaths “associated” with iatrogenesis, not caused by it.
I was recently advised from a good source that the current best research on preventable death is Hogan, H., et al. (2012). “Preventable deaths due to problems in care in English acute hospitals: a retrospective case record review study.” BMJ Quality & Safety 21(9): 737-745. That paper found that 5.2% of all deaths in hospital were preventable. Moreover, “Most preventable deaths (60%) occurred in elderly, frail patients with multiple comorbidities judged to have had less than 1 year of life left to live.”
Peter, can you be more specific about “those links are not working”? If you’re talking about the ones in my comment preceding yours, I just checked and they’re all working for me, at least. But maybe you mean another.
The first link Joyce provided to jamanetwork needs a sign in, the second one, the doi, I have just found how to use, although it only give a first page preview.
Your link to the JPS article worked fine Dave
One other thing. In the Hogan paper above the preventable death rate was extrapolated to 11,859 for England. If I covert that to the US (using a US population of 319 mi and England population of 53 million) I get 71,000 annual preventable deaths approximately in the US. This is an inadmissible conversion, but it is intriguing next to findings of the 210,000 – 400,000 deaths “associated with preventable harm” found in the John James (2013) paper referenced above.
Check this: De Vries et al. (2008) ‘The subjectivity of objectivity’ in Brave New World of Health. They have picked up on the same problem I raise above. The 180,000 “body count” “bounces” down to 98,000 for no explicable reason. The authors guess this is for rhetorical reasons. 180,000 is so large that it serves as an “indictment of everyone involved in healthcare.” 98,000 easily rounds to 100,000, “big enough to demand action but not so large as to invite push back.”
Thank you for adding to the thread (and for your patience — I was on a cross-country flight and then in a meeting so am just approving these comments). Intriguing additions.
Thank you. I think now there is some poor scholarship in the de Vries et al. article I’ve mentioned. The 180,000 deaths Leape refers to are “partly as a result of iatrogenic injury” whereas the 98,000 quoted by the IOM from the same study are deaths associated with preventable harm. Big difference. The 210,000 to 400,000 or more deaths claimed by James (2013) looks very suspect to me. I think the Hogan et al. study, which is English, but which can be extrapolated to 71,000 annual preventable deaths in the US is probably the more accurate study: 1) it’s published in a much better journal (BMJ Quality Safety is absolute first quality scholarship in my experience) and 2) I’ve looked a little bit at James’s method and the studies he’s drawn from and it looks suspect to me. Big numbers may be captivating but they need big proof and I don’t think he’s given it. And nor have I seen this paper before. If it was good I would likely have heard about it by now, especially because these numbers are so attention grabbing.
You people really have too much time on your hands, Who gives a rat’s arse if Lucian Leape said this or not–and who the hell knows how many people there are on an average jumbo jet anyway.
You should be spending your time on more interesting and relevant issues, such as does AC/DC with Bon Scott or Brian Johnson singing improve the efficacy of psychological testing in university studies
Oh, yes there’s a link!
http://tzellofouska.blogspot.com/2011/11/brian-johnson-vs-bon-scott-scientific.html
In case people don’t know Matthew, he delights in playing the clown, all the while thinking, analyzing, collecting, correcting, and curating.
Michael Hoad answers the question beautifully in the next comment, which was posted before I approved this one. Another answer is in my original post: I admit it. I am a geek. Geeks care about details. The long tail of the internet means that we geeks can find each other and talk about whatever we want, on our own time. In this case, over the course of 2-1/2 years!! Hooray for geeks!!!
Great thread. You’re right, the metaphor and the number affects how we as patients respond to the concept of medical error. Indictment of everyone involved, or part of living and dying? Error is a part of life; systematic error is a failure. My father-in-law suffered “death by weekend” as an uncoordinated scattering of clinicians failed to catch an encroaching c. diff. infection – even though we could see him shutting down. But he was there for pneumonia complicated by malignancies and the absence of most of a lung after cancer surgery (cigarettes). Should we be angry that their systematic weekend lack of watchfulness led to a “premature” death? Or is that just part of how we die?
The jumbo jet analogy first appeared in Lucian Leape’s 1994 “Error in medicine” article in JAMA, the article that began the patient safety movement.
http://hospitalmedicine.ucsf.edu/improve/literature/error_in_medicine_leape_jama.pdf
I’ve been using this analogy since the original IOM report. I typically don’t use jumbo jets, but typically say 100,000 people per year is like 27 jets crashing each month, each with 300 people on board. If this was happening, the FAA would ground all flights.
Thanks, Steven! I’m grateful to know that other people share my curiosity about the airplane analogy — and that this post is still useful, 4 years on.
I should probably add that the analogy was used by a friend (Thom Hartmann) in the 1980s in teaching newsletter editors to make huge numbers human-scaled. In that case it was tobacco deaths.
Amusing how commenters years later repeat things that were said years earlier … including ME repeating myself about newsletters. At least I didn’t contradict myself.
Oh, btw, at the Health 2.0 conference’s Town Hall Policy Forum on Sunday, someone admiringly noted Don Berwick as the source of “Nothing about us without us.” Back in 2010 the e-patient blog discussed its origin in the disability movement back in 1993 – spoken by activists, not academic observers 🙂 … the blog discussion cites precedents of Our Bodies, Ourselves and “Power to the people.”
Love how this thread now appears in the chapter notes of Dr Victor Montori’s new book – Why We Revolt – The Patient Revolution.
https://patientrevolution.org/why-we-revolt-chapter-notes
That might be a first for me! Thanks, Sherry, for pointing it out and thanks, Victor, for citing the post (and indeed, the thread). It’s the conversation that counts, as in so many cases.
I have been thinking about this quote & this number in light of the >500,000 deaths from COVID. It is a whole new perspective. How many jumbo jets a day are we up to now?
Just did the math. with COVID19, it is roughly 4 “jumbo jets” a day (assuming 366 passenger seats).
Thank you, Nichole! It’s an arresting image for a shocking number. Grateful to know this post continues to be useful for people.
Carrying Covid to Dave’s comments years back…
Sweet has 766 jets which hold a total of 115,000 people as of feb 2021
https://en.m.wikipedia.org/wiki/Southwest_Airlines_fleet
530k covid deaths in US at 1 yr
530k us deaths per year / 115k people per fleet = 2.6 months of Covid for 115k people to die.
Covid deaths in US In 1 year is like all of Southwest’s Jets crashing every two and a half months for a year and killing everyone on board…
And, Southwest has 766 jets in its fleet operating as of February 2021.
Oh wow. That’s 1) better math & 2) horribly depressing & hard to comprehend.
Even more horribly depressing and hard to comprehend is how many people in this country got convinced to ignore science and party like there was nothing to be careful about.
On the other hand, the general public is held to a lower standard than health professionals.
What they seem to have in common is the response “I’d rather not think about that” when confronted with unpleasant realities.
I too have gone down this rabbit hole in developing a podcast on medical injury for ‘Pomegranate Health’ racp.edu.au/podcast.
Danielle’s Ofri’s 2020 book “When we do harm” opens with a discussion of the analogy, correctly citing Lucian Leape as “a jumbo jet and a half crashing every single day. In fact she opens her book with “medical error is the third leading cause of death in the US” claim from a 2016 BMJ paper. https://www.bmj.com/content/353/bmj.i2139.full
She very quickly picks both numbers apart and indicates, as mentioned above, 1) the raw data were collected at just a few institutions and generalising across the whole population/ health service may not be appropriate. 2) not all deaths following medical error will have been caused by the medical error.
Ofri goes on: “The IOM report could not answer that question, and the media could not be bothered with such parsings. The headline was just too juicy to pass up. It is likely that we do not have a jumbo jet worth of patients crashing on US soil every day as a result of our misdeeds. The number is probably smaller. However, it’s not zero.”
I admire your attention to detail, Susannah. On so many occasions have I found “accepted truths” repeated over and over again, until they lead back to a single source that either makes no such claim, or does based on speculation not fact. This is as true in the academic literature as it is in the media.
Thanks, MC! I found Pomegranate Health on Spotify and will listen on my next long walk.
Let’s keep adding to this thread when we find new evidence and examples. As I said above: Geeks like to get things right. Happy to dive into rabbit holes with all of you!
I know I’m late to this discussion but I wanted to share that I had the privilege of interviewing Lucien Leape when I was Manager of Media Relations for The Joint Commission from 1991-1994. It was a brief for a newsletter that I believe was called simply, Quality Resources. I distinctly remember him using the jumbo jet analogy in our interview. I feel he said this to me before the JAMA article came out.
Thanks, Stephen! The discussion is never over online. My apologies for not approving your comment until today. I’ve been neglecting my blog while I work on my book.