“Can you really trust what you read online?” is a frequent (and valid) question about online health information, particularly peer to peer advice. People who are part of healthy peer communities sometimes scoff at the question since they see so much good coming from the opportunity to connect and share information. But how might we convince skeptics?
One way is to collect evidence. For example, one of my favorite studies, published in 2006, analyzed 4,600 postings on an online breast cancer forum. Researchers found that 10 posts contained incorrect or false information, seven of which were corrected within hours by the community. Just 3 out of 4,600 posts were potentially misleading and allowed to stand uncorrected. Pretty darn good.
See: Esquivel A, Meric-Bernstam F, and Bernstam EV. Accuracy and self correction of information received from an internet breast cancer list: content analysis (British Medical Journal. 2006 Apr 22;332(7547):939-42.)
In a more recent study, researchers reviewed two years’ worth of messages posted to an online patient community focused on implanted cardiac defibrillator (ICD) treatment. They found that the majority (about 75%) of the advice shared was benign or helpful, but 24.4% of the material shared was inappropriate for most patients. That’s sobering.
This quote captures the hopeful angle on the data collected:
In cases where members sought general information about ICDs and their functionality, the advice provided on this message board provides a succinct, accessible, and well-organized resource of basic information of interest to ICD patients and candidates. In this sense, anonymously submitted information appearing on this internet message board acts as a resource that might help avoid gaps in fundamental understanding among patients observed previously and may provide a reliable reference to which providers can refer patients.
Peer to peer advice can be an essential, even extraordinary, resource for people who are newly diagnosed, unsure about how to handle their next step along a path, etc. How might we boost the positive aspects of peer communities while guarding against the potentially negative ones?
See: Knoepke CE, Slack DH, Ingle MP, Matlock DD, Marzec LN. Quality of Medical Advice Provided Between Members of a Web-Based Message Board for Patients With Implantable Defibrillators: Mixed-Methods Study (JMIR Cardio 2018;2(2):e11358)
NewsGuard is an independent service that employs journalists as roving fact checkers. One of their studies found that popular “natural health” sites are rife with misinformation and unproven claims. The connection to peer health advice? Social media helps to feed the beast. NewsGuard says that some of the misleading “sites accounted for more than 49 million engagements (shares, likes, comments, etc.) on social media in the past 90 days.”
Note: I have not been able to find a link to a formal NewsGuard study or details about how they defined the pool of websites included in their evaluation. If anyone has a link, please drop it in the comments below.
To add to the pile of evidence about the validity of online health information, let’s look at a 2013 study which assessed the quality and readability of mental health information returned by Google and Bing. The researchers found that when they searched for 11 common mental health terms, both search engines were likely to return pages from WebMD, Wikipedia, and the Mayo Clinic among the first 20 results. And the information contained in the search results was mostly (67.5%) good quality. The biggest potential challenge for people seeking mental health information online is that it is likely to be written at an 11th grade reading level, far above the recommended level.
I was particularly interested to see that the researchers used a 16-question assessment tool co-developed by the UK’s National Health Service and a group of patient experts. The tool, called the DISCERN Instrument, is freely available online along with a handbook to guide people who would like to use it to assess the quality of written health information.
One of the key questions asked by DISCERN is whether an article or page describes the risks of each treatment. The researchers describe an intriguing finding:
An examination of specific items on the DISCERN demonstrates that some items are not often discussed on the mental health Web sites we examined: risks of treatment, uncertainty about treatment choices, and the risks and benefits of not getting treatment. This could reflect a protreatment bias on mental health Web sites, which in turn may be a result of Web site authors attempting to counter stigmatization and undertreatment of mental disorders.
In my fieldwork and interviews with people who participate in mental health-related communities, I’ve heard stories about how refreshing — even revelatory — it is to hear from fellow patients about treatment side effects that their clinicians either do not mention or downplay as unimportant. Again: Peer to peer advice plays a vital role as a supplement to what someone can learn from Dr. Google, Dr. Bing, and their own clinicians.
See: Grohol, John & Slimowicz, Joseph & Granda, Rebecca. (2013). The Quality of Mental Health Information Commonly Searched For on the Internet. Cyberpsychology, behavior and social networking. 17. 10.1089/cyber.2013.0258.
When I asked the lead author of that study, John Grohol, founder and CEO of PsychCentral.com, about his experience and opinion about online health information quality, he shared two additional studies:
- Quality of Weight Loss Advice on Internet Forums (2007).
- User generated content and credibility evaluation of online health information: A meta analytic study (2017)
There are other dangers besides poor quality information. See, for example:
- “They all supported me but I felt like I suddenly didn’t belong anymore“: an exploration of perceived disadvantages to online support seeking. (2010).
- Internet peer support for individuals with psychiatric disabilities: A randomized controlled trial (2011).
- Are online support groups always beneficial? A qualitative exploration of the empowering and disempowering processes of participation within HIV/AIDS-related online support groups (2014).
- The Pros and Cons of Getting Engaged in an Online Social Community Embedded Within Digital Cognitive Behavioral Therapy for Insomnia: Survey Among Users. (2016).
- First do no harm: Cross-sectional and longitudinal evidence for the impact of individual suicidality on the use of online health forums and support groups. (2016).
John also pointed out that most studies focus on established communities, not information shared, for example, in YouTube videos or on Facebook. Social media platforms are more difficult to track and study but they dominate the public conversation.
If you know of other studies looking at these topics, please add them in the comments. I’d like to build a robust literature review. I promise to share it.
Image: “Trust?” by Jan Mennens on Flickr.
Paul Wicks says
OK this is now murdering thread etiquette but I meant to say something like this in the 3rd para:
…but the contributor ratio was less than 1%, the number of people that could access it was tiny, and of course they were very biased to be highly educated. Fast forward to the 1990’s and the great AOL/ADSL explosion greatly expands and broadens the number of web users and the availability of open source-ish platforms like PhP and VBulletin means that anyone can set up their own online community. These form close-knit links and micro-cultures around their 1% active contributorship but get lots more readers from more diverse sources but are prone to splitting and technical issues due the instability of their platforms and their essentially voluntary nature. In the mid-2000’s we get the social media explosion and new funding for places like PatientsLikeMe, DailyStrength, CureTogether, as well as non-profits finally waking up to the benefits of having an online support forum, which means you can now have…
10k+ registered users in even quite rare diseases, so for a lot of things you can search and more easily find the key-lock or convene a micro-squad to problem solve with you.
Susannah Fox says
Paul, thank you. There is a lot to unpack in your comment.
I’ll start with our shared observation that the early online communities felt wondrous, like the discovery of fire (as my friend Vic Sussman once said) because people had that never-before experience of finding other people who shared the same interests (including health conditions), no matter how rare or obscure.
And yes, as the online world expanded and became more corporate in some areas, more static in others, the path diverged and became many paths. (For our visual notes, I’m imagining something that looks like capillaries extending out from an artery.)
But that first, essential aha moment is still happening every day. That’s the peer to peer magic I can’t get over. Finding the “just in time someone like me” who can share what it really feels like to give birth/recover from brain surgery/start that drug/care for a loved one with dementia/shave your own head/set up a baby-safe hotel room/your particular challenge here… And now, as you wrote, there are many, many more ways for people to connect and be connected.
How are all the various connection mechanisms helping people to sort the good from the bad, the true from the not-true, the safe from the dangerous advice? That’s the question I’m working on this week and I’d love to learn from people about what they see in the wild AND in the literature.
Jan Oldenburg says
Susannah,
Thanks for a thoughtful post on a topic that deserves our time and attention. I have been stewing about this issue for some time, feeling as if the way social media can amplify bad/exploitive information about health threatens to overwhelm the good that peer-to-peer support produces. I also fear a broader impact on the whole patient empowerment/participatory health movement, as awareness about the spread of anti-vaccination information, for example, makes it easier for doctors to dismiss people who are actively seeking to improve their health with peer support and other forms of online activity.
We are facing some of the same issues across all aspects of science and other news, as it is harder and harder to distinguish what information and sources are credible.
I suspect that people may be leerier about challenging posts from others that convey invalid health information because they may fear being overwhelmed or attacked by those with different views. Still, sites with strong moderation whose moderators help sort fact from fiction and who can be trusted to manage disputes can clearly make a difference.
I have more questions than answers and am interested in other suggestions you hear as this feels like an urgent problem for those of us who believe in the power of peer support.
Andrea says
Thank you this is an EXCELLENT set of resources. Will be reading further.
On topic of misinformation …. I just came back from DEFCON where Eric Perakslis shared a presentation on targeted medical misinformation campaigns from state actors, and how this is affecting measles outbreaks all over the world.
Check out this paper: https://jamanetwork.com/journals/jama/article-abstract/2736435
Susannah Fox says
Thanks, Andrea! That’s a great article — and it led me to this one:
Counteracting Health Misinformation: A Role for Medical Journals, by Paul W. Armstrong, MD, and C. David Naylor, MD, DPhil.
Check out this comment from Richard Bedlack, MD PhD | Professor of Neurology, Duke University:
“I wanted to tell you about an example of physicians and scientists working together on the problem you describe.
There is a program started in 2009 called ALS Untangled (www.alsuntangled.org). It includes a virtual bulletin board where patients submit alternative and off-label treatment (AOT) ideas for amyotrophic lateral sclerosis (ALS), and vote for the ones they are most interested in. Patients have now submitted more than 400 AOTs; several have received more than 1,000 votes. A team of 120 clinicians and scientists from across 10 different countries invented and crowd-sourced standard operating procedures including a “Table of Evidence” that facilitate objective review of AOTs. Using this infrastructure, we have now published reviews of 49 different AOTs. Through an agreement with the journal Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration, these are made free open access so patients never have to pay to read them. ALSUntangled reviews comprise 8 of the top 10 most downloaded articles in the history of this journal. Some have more than 30,000 downloads, and collectively they have more than 100,000 downloads. We recently started short podcast summaries of our reviews and these are already getting hundreds of listens each month.
Through a collaboration between ALS clinicians, scientists and a medical journal, ALSUntangled is helping patients to better understand AOTs, and make more informed decisions about them.”
Now, I’m going to be charitable and say that he meant to include the patients in his description of “scientists” since they are the ones doing a lot of the work here, engaging in their own n=1 experiments and reporting the results.
It is in working in partnership that patients, caregivers, clinicians, and researchers can bat down misinformation and build up evidence of what works (<-- my editorial comment)
Paul Wicks says
I can confirm that ALS Untangled does also include patients like Eric Valor, who contribute equally =)
Paul Wicks says
(Here’s how my comment should have read, feel free to delete the first one)
Great post as always – isn’t it interesting how we’re still asking the same research questions 20 years on?
I’ve been looking to try out a theory about online communities with you and this readership.
In the beginning (1980’s on The Well, Usenet, list-servs) it was helpful just to hear you weren’t alone and perhaps get some tips, but the contributor ratio was less than 1%, the number of people that could access it was tiny, and of course they were very biased to be highly educated. Fast forward to the 1990’s and the great AOL/ADSL explosion greatly expands and broadens the number of web users and the availability of open source-ish platforms like PhP and VBulletin means that anyone can set up their own online community. These form close-knit links and micro-cultures around their 1% active contributorship but get lots more readers from more diverse sources but are prone to splitting and technical issues due the instability of their platforms and their essentially voluntary nature. In the mid-2000’s we get the social media explosion and new funding for places like PatientsLikeMe, DailyStrength, CureTogether, as well as non-profits finally waking up to the benefits of having an online support forum, which means you can now have 10k+ registered users in even quite rare diseases, so for a lot of things you can search and more easily find the key-lock or convene a micro-squad to problem solve with you.
But now as people can address the low hanging fruit with Wikipedia and other static sources, the nature of the locks changes. The specificity and requirements of the answers run deeper than you can find through Brownian motion in forums, plus a lot of attention is being sucked in to Facebook where the features are severely curtailed. Fast forward to today and into the mix comes “digital therapeutics” that don’t aim to just shotgun people at each other with the hope that a meaningful collision will occur, but rather they’ve been designed with the “patient journey” in mind and guide users through education and tools down the common paths that almost all patients will face about diagnosis, understanding, treatment decisions, managing side effects, etc. with options between static content and social features along the way, supplemented increasingly with plug-and-play tracking and increasingly EHRs. Most of the content is expert-generated and refers back to national or organizational guidelines rather than free-styling it.
In *that* system you no longer need say 100k patients to be all having random chats about drugs and the weather, you might not need any at all as you’re being presented with 100 keys that patients like you often face. They might not be as fully satisfying but they also don’t rely so much on engagement, community size, and to top it all off they look a lot more like a programmatic wellness management tool that can attract reimbursement, as opposed to the online patient community choices of a.) ads, b.) sell data and data services, c.) VC underpants gnomes, d.) referrals to pharma and its agencies for its needs.
Would be awesome to animate this somehow. Anyway, interested in thoughts!
Susannah Fox says
Paul, thank you. There is a lot to unpack in your comment.
I’ll start with our shared observation that the early online communities felt wondrous, like the discovery of fire (as my friend Vic Sussman once said) because people had that never-before experience of finding other people who shared the same interests (including health conditions), no matter how rare or obscure.
And yes, as the online world expanded and became more corporate in some areas, more static in others, the path diverged and became many paths. (For our visual notes, I’m imagining something that looks like capillaries extending out from an artery.)
But that first, essential aha moment is still happening every day. That’s the peer to peer magic I can’t get over. Finding the “just in time someone like me” who can share what it really feels like to give birth/recover from brain surgery/start that drug/care for a loved one with dementia/shave your own head/set up a baby-safe hotel room/your particular challenge here… And now, as you wrote, there are many, many more ways for people to connect and be connected.
How are all the various connection mechanisms helping people to sort the good from the bad, the true from the not-true, the safe from the dangerous advice? That’s the question I’m working on this week and I’d love to learn from people about what they see in the wild AND in the literature.
Deb Maskens says
Hello Susannah and all,
This is a fascinating and really timely topic. Thank you. As a patient advocate in cancer (and specifically renal cancer), I have been involved in various online forums over many years and see the value — perhaps additional value in rarer cancer/rare disease cases where information is scarce and peers are not easily found offline.
That said, I’m concerned about what I see as Anecdote vs Evidence (the power of the single narrative — e.g., President Jimmy Carter’s “cure”.) The Evidence is often harder to grapple with than a single patient story that might well not be representative. But everyone remembers that patient story and wants to be that patient.
Another dimension that I’m not sure has been published on (anyone?) is whether participation in an online forum does potential harm to psychological wellbeing. (For example, on some of our Forums, patients & caregivers are logging on several times a day (sometimes in the middle of the night) because they don’t want to miss something (FOMO). I wonder whether there have been any studies that would help patients appreciate when the online connection is doing them psychological harm. In metastatic cancer forums, we see a lot of disease progression, a lot of deaths, and a lot of anger (not to mention unproven treatments, dietary recommendations, etc.)
Hope this will spur on some more conversation!
DebM
Susannah Fox says
Thanks, Deb. This line of yours will stick with me:
“But everyone remembers that patient story and wants to be that patient.”
I’ve lived that with people I care for and indeed it made it hard to talk about the realities of the path ahead.
Regarding your question about potential harm to psychological well-being:
One reason I wrote this post is to smoke out more studies related to this topic, to list what I’ve got and hopefully have people say, “Wait! You forgot this one” or “Hey, why aren’t there more studies about XYZ?” and then like a posse pursuing truth we’d find them.
Check out the studies that John Grohol shared with me — the list at the bottom of the post — to see if they ask the questions you’d like to see asked of your communities. For example, the HIV community was found to be both empowering and disempowering of their members:
“Six empowering outcomes were identified: increased optimism, emotional well-being, social well-being, being better informed, improved disease management, and feeling confident in the relationship with physicians. Potentially disempowering processes were also identified which included: being unable to connect physically, inappropriate behaviour online, declining real life relationships, and information overload and misinformation.”
e-Patient Dave says
Apologies for not getting here yesterday – I was returning from a fascinating vacation trip to college friends and then to a childhood home, on the road back. What a mind-bender – got to stand in the living room, dining room, kitchen of my grade school years. (In a way it evokes thoughts like Paul’s tour of the early days, above.)
While I agree about all the concerns and studies of it, my angle on all this is different. To me the real question (the *only* real question) is “How can I get the best advice on what I should do??” The importance and difficulty increase with the severity of the need and – importantly – with the uncertainty of the answer.
And that’s where it gets ucky, because when people question the reliability of internet information, they never want to place it side by side with evaluating the reliability of the peer reviewed literature. I believe this is a gigantic methodological error, ignoring the real issue of being scientific in assessing the evidence.
You (Susannah) gave me the mind-blowing epiphany on this when you said years ago, in the umpteenth argument about internet anti-vax problems (paraphrasing): “Wait a minute: the smoking gun that started all this was that the peer review process at a top journal failed, so a fraudulent study got published and wasn’t discovered for years!”
Again: the reason we NEED science, vs rumors or superstition is because the REAL question is, how can I / we decide what’s the best thing to do for this particular problem?? Continued in the next comment.
e-Patient Dave says
Interested readers can find our blog posts about the weaknesses in the peer reviewed literature by searching these phrases on e-patients.net:
– “Ben Goldacre”, and his AllTrials.net about active suppression of unfavorable trial results – flat-out scientific corruption
– Seroquel – a particular example
– “Research issues” (whole category of blog posts)
– “Reproducibility Initiative”
– “John Ioannidis” (Stanford researcher who pokes holes in weak science)
I’ll link to just one, which includes some severe eye-popping examples of how establishment science isn’t as certain as people think:
“The Decline Effect: Is there something wrong with the scientific method?”
a href=”https://participatorymedicine.org/epatients/2011/01/the-decline-effect.html
Yet consider this:
– the BC community you cite quickly corrected 70% of the mistakes found
– but very few scientific papers are ever retracted, even after being busted by other scientists like Ioannidis
An important structural driver of this issue is that many people in the scientific professions have a stake NOT in the accuracy of their findings, but in whether their finding gets published, which is precisely the opposite of the stake held by patients.
Never ever forget that the editor-in-chief of the New England Journal said a few years ago that they NEEDED to let authors have conflict of interest, or they wouldn’t have enough papers to fill the journal.
One more comment coming up. (Well, you asked…)
e-Patient Dave says
Comments by three editors of major journals … I cite these because some people consider all attacks on journals to be anti-establishment, but if it’s from the TOP of the establishment it makes no sense.
Marcia Angell, 2009: “It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine.”
Richard Smith, former BMJ editor: “After 30 years of practicing peer review and 15 years of studying it experimentally, I’m unconvinced of its value.”
Richard Horton, Lancet: “A lot of what is published is incorrect. … something has gone fundamentally wrong with one of our greatest human creations. The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.”
So honestly, really, truly – let’s shift the focus. The question that has intellectual integrity is not “can we trust the internet,” it’s “How can we best decide what to do in this case?”, and except in the simplest situations (e.g. a broken arm, an ear infection), the question must be pursued in a context of uncertainty.
I’m not against checking one’s source – I’m just saying that pointing a finger at the internet and NOT at the establishment is a mistake.
I could go on and on (there’s tons more to say) but that’s what it comes down to.
e-Patient Dave says
And now I want to apologize for going off target, not answering what you were seeking. I get ranty sometimes, and the question touched a persistent sore point of mine, but my response wasn’t helpful to your question.
Susannah Fox says
No apology necessary. You make great points! The challenge I’m struggling with is this: The established journals and gatekeepers ARE the ones that many people look to for guidance. So if we want clinicians to give out “information prescriptions” like Danny Sands did for you, or we want big health care entities to invest in peer to peer communities, how do we convince them that the information being shared is worthy?
Susannah Fox says
Since this blog is my outboard memory, I’m capturing the tweets of people who haven’t made the jump to post here:
Bob Gann: “How can we boost the positive aspects of peer communities, while guarding against potentially negative ones”?
@SusannahFox
considers the evidence. BTW interesting to see researchers using DISCERN health information quality tool, which I helped develop https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1756830/
(I replied to Bob: Thank you for helping to create such a useful tool and – crucially – posting it online for free. I’d love your perspective on where we stand these days with online health info, particularly in peer communities if you have time to post a comment.)
Bob Gann replied: There’s a useful @PiFonline overview of health info quality tools but I’m not sure how well they work for peer communities. Maybe we just need to trust the (usually good) sense of community moderation https://www.pifonline.org.uk/pif-resources/topics-index/evaluating/quality-assurance/tools-for-assuring-quality-and-useful-resources/
Laurent Rotival: Excellent article, thank you @SusannahFox, which secured, trusted, person-focused 3rd parties will step up and deliver the other half?!
@RareCandace: I think it depends on the condition. I hear from cancer patients that their doctors tell them to stay off the internet, because what they read will scare them. Alternatively, Rare Disease patients rely on online patient communities for info where doctors fall short.
(I replied to Candace: Yes, I have seen that pattern, too. Where you lie along the “known <--> unknown” spectrum plays a role in your decision-making about using #PeerHealthAdvice. Problem is: knowledge is dynamic. What’s “known” when it comes to YOUR case vs. general evidence?)
Candace replied: That’s partially why I started my blog back in 2014. I couldn’t find any patient stories on Rituxan for ITP. Reliance on peer health advice has many factors, but I would be willing to bet that the more detailed it is, the more patients rely on it.
Naomi Kaufman Price: Depends on the potential level of harm from the other half. Eg: “Have you tried aromatherapy/crystals/homeopathy” for cancer/autism/nut allergy.
ALS Advocacy: I think there’s a bit of a checks-and-balances system at work.
We always hear if a doc corrects some bad advice someone got from a peer.
We seldom hear if a peer corrects some bad advice someone got from a doc. That happens w ALS all the time.
Doug Rachac: I have so many questions about the study parameters. What does “inappropriate for most patients” mean? Also, they deemed a response to obtaining a tattoo inaccurate when the response perfectly mirrors what one device manufacturer recommends. I participate in many ICD patient communities. Some are better than others (some drive me crazy). But the good ones, I feel, far exceed this study’s findings in accuracy. The info shared is backed by evidence. There are many very informed patients out there.
Doug continued: Oh my, I could go on and on… The good communities have knowledgeable patients and HCPs, and the correct answers to questions come quickly. Unfortunately they are surrounded by incorrect answers as well and even evidence won’t sway some believers. Case in point: can ICD/pacer PTs go through airport security? 15 years ago it was considered dangerous, but today it’s safe. Still, you’ll find 5 people saying “my doc/nurse said no” and 1 with links to industry correctly saying yes. Who should PTs believe? Drs don’t have hours to spend answering PTs questions, so PTs find communities to learn more because what we fear is the unknown, and knowledge alleviates fear/anxiety. The hard part is weeding out the junk and finding the truth.
Gretchen_R: Great article! Why are patients using #PeerHealthAdvice ? We don’t have a choice. With CIEDs, remote monitoring prevents us from interacting w/our Drs. RM is now standard of care. EPs track 000s of patients and have abdicated pt education in favor of watching computer screens.
Matthew Holt: Sounds like it’s about the same ratio as the rest of medical care (54% of doctors follow guidelines according to the Rand study from way back)
Amy Price says
What if we were to use the DISCERN TOOL to evaluate these topics or 10 most discussed health conditions on Facebook/Twitter etc and run head to head comparisons, in this way we could objectively share the accuracy of each platform for health information and raise awareness about how and where influence becomes infiltration and is no longer fit for purpose. This deals with information and not opinions, personalities, intellectual property, business interests or hierarchies and each entity being measured could choose to improve information and value and grow their DISCERN score.
Susannah Fox says
Love this idea! How might people organize to get it done?
Samantha Luu says
Susannah, your post and community comments spurred some meaningful reflection from our team and had the following thoughts (also shared on our Tuesday Tips e-mail) —
In seeking to understand challenges faced by in-person peer support programs, we asked investigators funded by Peers for Progress if they had any instances of misinformation on the part of the peer supporter and how that might be addressed. Of five programs accounting for nearly 250 peer supporters, four reported no instances of misinformation and one reported that where a peer supporter was unsure, the study nurse sat in on peer support groups to assist (Peers for Progress, 2015). We know that in selecting for peer supporters, we want to look for traits like resourcefulness and self-awareness while avoiding those who may be too bossy or overly confident. This is important because we want peer supporters who are naturally resourceful while understanding where their boundaries lie in offering support or personal anecdotes (E. B. Fisher et al., 2018; E. Fisher, Hacker, McDonough, Nielsen, & Tang, 2015). We have found that underscoring all of this is the critical component of accessible backup (E. Fisher et al., 2015). As in the case above, a nurse, program coordinator, or other clinician should be available to assist with questions regarding accurate information and to provide credible resources to the peer supporters which they could share with their peer clients; the backup’s responsibilities can also include crisis management and ongoing supervision
Online forums used by peers often exist without moderators, supervision, or peer supporter training (disclaimer: there are also a number of moderated forums). While your original blog sources, and ensuing commentary, does find a variety of credibility to the health information posts in these forums both positive and negative, it seems that generally, information posted in online forums can be beneficial. We found it heartening that (Brady, Segar, & Sanders, 2016) found that forum participants, specifically those for those with chronic conditions, largely framed their posts as experiential, deferred major decision-making to discussions with healthcare professionals, and practiced self-policing within their communities. Further, they found that those seeking information in online forums identified and trusted posts from those with (longer) lived experience and whose personal experiences were more like theirs – those with a higher level of peerness to themselves. On some forums, healthcare professionals partake in posting or monitor the information being circulated on a forum. There is no way to ensure that online peer health advice is 100% accurate, but we can certainly do more to promote good health information and reduce the spread of information that can cause harm.
Susannah Fox says
Thank you! I’m grateful for these resources and insights.
Everyone: You should subscribe to Peers for Progress’s e-newsletter — it is one of my must-reads each week.
Here’s the bibliography for this week’s, which includes the citations above:
Bibliography
Appleton, J., Fowler, C., & Brown, N. (2014). Friend or foe? An exploratory study of Australian parents’ use of asynchronous discussion boards in childhood obesity. Collegian (Royal College of Nursing, Australia), 21(2), 151–158. doi:10.1016/j.colegn.2014.02.005
Brady, E., Segar, J., & Sanders, C. (2016). You get to know the people and whether they”re talking sense or not’: Negotiating trust on health-related forums. Social Science & Medicine, 162, 151–157. doi:10.1016/j.socscimed.2016.06.029
Fisher, E. B., Bhushan, N. L., Coufal, M. M., Kowitt, S. D., Parada, H., Sokol, R. L., … Graham, J. T. (2018). Peer Support in Prevention, Chronic Disease Management, and Well-Being. In E. B. Fisher, L. D. Cameron, A. J. Christensen, U. Ehlert, Y. Guo, B. Oldenburg, & F. J. Snoek (eds.), Principles and concepts of behavioral medicine: A global handbook (pp. 643–677). New York, NY: Springer New York. doi:10.1007/978-0-387-93826-4_22
Fisher, E., Hacker, T., McDonough, M., Nielsen, M., & Tang, P. Y. (2015). Peer Support in the Patient-Centered Medical Home and Primary Care. Presented at the Peer Support in the Patient-Centered Medical Home and Primary Care, American Academy of Family Physicians Foundation, National Council of La Raza, Patient-Centered Primary Care Collaborative, and Peers for Progress.
Peers for Progress. (2015, March 25). Better Supervision of Peer Supporters. Retrieved August 18, 2019, from http://archive.benchmarkemail.com/Peers-for-Progress/newsletter/March-25-2015-Newsletter-Copy
Schon, J., Ristic, I., & Manning, J. (2015). When the Internet Becomes the Doctor: Seeking Health Information Online.
Zhao, Y., & Zhang, J. (2017). Consumer health information seeking in social media: a literature review. Health information and libraries journal, 34(4), 268–283. doi:10.1111/hir.12192
Dave deBronkart says
Wow, I had no idea there was a body of work on peer communities! It makes perfect sense, but this just opened the gates to a whole new kingdom of knowledge. Thank you!
I just LOVE discovering that what we look at “locally” (within healthcare) is just an instance of something more universal. It provides such credibility in arguments, and allows anticipating further parallels.
Susannah Fox says
Update on the NewsGuard article cited above: Matt Skibinski, GM of NewsGuard Technologies answered my inquiry about whether there is a longer report or methodology I can look at.
He wrote:
“To answer your question–we actually did not produce a longer whitepaper or report for this analysis, as it really came about just by analyzing at our database of website ratings data in the course of normal business. In other words, this was not a separate research effort beyond what we normally do–so there’s no broader study to link to–though we have since heard some interested in using this data as part of a more formal research effort.
As for how we determined the pool of websites: It was the same way we determine our pool of websites to rate in general. We work from an ordered list of news and information websites (using data from NewsWhip supplemented by Alexa traffic data and some other sources such as Trendolizer) based on how much engagement they get on social media platforms. We then rate each site based on our nine criteria, issuing it a credibility rating. Our database covers the nearly 3,000 news and information websites that account for 96% of engagements on news/info in any 90-day period. (Of course, the websites that account for the last 4% of engagements are quite numerous due to a long tail effect–but the ones we’ve rated are the most influential and important ones.)
It’s important to note that list we’re working from includes both mainstream and credible news and information sites and sites that contain news and information content but are unreliable (such as naturalnews.com and its affiliated sites). The credible sites get GREEN ratings from NewsGuard, and the ones that are not credible get RED ratings.
What our analysis showed was actually quite simple. We looked at how many of the sites with RED ratings were flagged because of health misinformation. That came out to just over 11% of all sites.
We can’t say with certainty that the 11% number wouldn’t be different if we could rate every news and information website on the internet, but we think the sites we’ve rated thus far constitute a substantial sample size (and they cover sites that get the most engagement, so this data set is the most important).
We also realize that our analysis probably undercounts the amount of medical misinformation online, since we’re only really looking at websites that publish medical misinformation regularly and egregiously enough to get a RED rating. If a generally credible site occasionally publishes poorly researched health content (but not egregious enough to cause a red NewsGuard rating based on our criteria), this analysis doesn’t account for that. So, we’d love to see some broader studies of this done in the future.”
Thanks, Matt!
Kyle D. Lloyd, BS, CPS says
Who will you include on your jury?
Susannah Fox says
Thanks for the comment, but I’m not sure I understand. Can you say more?
Kyle D Lloyd says
Slander – look it up.
Susannah Fox says
Now I’m really confused. If you have time to share what you have concerns about, I’d like to know. From what I can see on your LinkedIn profile, you are an expert on peer counseling and mental health. Thank you for your service to veterans and to the mental health professional community.
Kyle D Lloyd says
Towards the closure of this article the statement is made that peer websites are not monitored, etc., that the advice isn’t 100% fact-checked, and that there is possibility that harm is spreading. I’ve been practicing over 12 years in this field and have yet to see a peer working that spread or precipitated harm of any kind.
Dave deBronkart says
I’m on the fence about NewsGuard. We’ve had a robust discussion on the Society for Participatory Medicine’s “Connect” member forum, which (as you know) has some pretty savvy observers, and the savviest of them think it’s either questionable or downright sketchy.
Ken Masters, a continuing ed professor, said “I teach my students Kapoun’s five basic criteria of Accuracy, Authority, Objectivity, Currency, Coverage. Again, not infallable, but a good start. (Kapoun, J. 1998. “Teaching undergrads WEB evaluation: A guide for library instruction.” C&RL News (July/August 1998): 522-523.)” He also points out that when you install the extension you have to grant it the rights to “Access your data for all websites,” so he said no thanks.
SPM co-founder “Doc John” Grohol, founder of the venerable PsychCentral peer community, says he’s not sure mere (my word not his) journalists are the best judges of whether a health info website is trusty, and points out that it doesn’t seem to have had any independent validation or peer review.
He also notes that their criteria seem biased toward traditional news organizations – and as any savvy e-patient knows, the Health News Review service has long documented how sieve-like news organizations can be, letting really weak stories get into the mainstream.
So I keep coming back to “no easy answers” and “no sure authority.”
Amy Price says
Kyle, I think it is wonderful you have spent 12 years and have “yet to see peer working that spread or precipitated harm of any kind.” An expert on peer counselling and mental health would recognise the value of building trust and creating psychological safety even for those whose voices are not the same as theirs such as on this feed. The comment “Slander, look it up” does not make for a sense of openness or communication. Please, can you reframe this? I HAVE seen significant harms from peer working, even deaths. I have also seen incredible recovery and discovery that would not be possible without peer working. Can we agree that this is an area to nurture, protect, and equip so that those who are vulnerable are safe and that using words and approaches that build mental health in others helps us all?
Susannah Fox says
(Note: this is a continuation of a thread. Because of the comment limitations, I can’t reply to Kyle’s last response.)
Kyle, I think you and I are on the same side. Maybe your critique is not directed at me but at another commenter? I’ve read and re-read the original post and can’t see where you’d draw a conclusion that I’m not in favor of peer-to-peer communities.
I, too, believe that peer health advice is an essential tool in helping people navigate toward well-being. And I’m very happy to hear that in your experience – 12 years! – you have seen no evidence of harm. That’s fantastic testimony.
What I’m hoping to build — or access, thanks to colleagues sharing their own literature reviews — is a pile of evidence that we can show to anyone who doubts the helpfulness of peer health advice. Some people are convinced by storytelling. Others are convinced by peer-reviewed studies.
Again, thanks for taking the time to share your experience and your expert views.
Samantha Luu says
(Note: This is a thread response to both Kyle Lloyd and Susannah’s last responses posted Aug 22 at 9:41am and 10:18am, respectively)
Mr. Lloyd, I believe your comment is referring to a reply I originally posted on Aug. 20 at 1:05pm on behalf of our research group, Peer for Progress.
First, I’d like to echo what Susannah has stated in that we too feel that peer health advice, including that in online forums, is key to improving mental and physical health via peer support. In fact, we believe that sharing peer health advice is a key function of peer support (E. B. Fisher et al., 2018).
Second, it is fantastic that in your career as a peer supporter you have not had any evidence of harm perpetrated by peers. We too have found very little evidence of harm or misinformation in peer support programs that we have funded and evaluated.
The statement that we made and that you highlighted about not being able to moderate *all* peer health advice is in the specific context of online forums which have a very different organization and protocol than in-person/phone peer support. Whereas the latter is structured in a way that often involves training/certification and ongoing supervision of some sort, online forums can proliferate without either. Thus, it is important for consumers of those forums to understand that information shared there is experiential and not be taken in lieu of medical advice; however, that is not to diminish the power of those forums for peers to connect. In fact, in that same paragraph, we cited an article (Brady, Segar, & Sanders, 2016) which did find that 1) some consumers do understand the limitations of online forums, and 2) those who post in online forums understand the limitations of their experiences, and 3) other peers in the forum will “self-police” their community for misleading posts.
I hope that helps to clarify our response to Susannah’s original post. In Susannah’s process of compiling literature to prove that peer support is helpful, it is important to also consider literature and views which may be more skeptical of peer support so that in our advocacy of it, we understand the arguments against. Our original response (Aug. 20, 1:05pm) was in no way to detract from peer health advice, but to add that in the medium of online forums, we need to be cognizant of the limitations and challenges to delivering high quality peer support.
Susannah, we are excited to follow and be a part of this continuing conversation.
Mr. Lloyd, thank you for all your work as a peer supporter and for sharing your perspective.
___________________________________________________
Sources:
Brady, E., Segar, J., & Sanders, C. (2016). You get to know the people and whether they’re talking sense or not’: Negotiating trust on health-related forums. Social Science & Medicine, 162, 151–157. doi:10.1016/j.socscimed.2016.06.029
Fisher, E. B., Bhushan, N. L., Coufal, M. M., Kowitt, S. D., Parada, H., Sokol, R. L., … Graham, J. T. (2018). Peer Support in Prevention, Chronic Disease Management, and Well-Being. In E. B. Fisher, L. D. Cameron, A. J. Christensen, U. Ehlert, Y. Guo, B. Oldenburg, & F. J. Snoek (eds.), Principles and concepts of behavioral medicine: A global handbook (pp. 643–677). New York, NY: Springer New York. doi:10.1007/978-0-387-93826-4_22
Anna De Simoni says
Dear Susannah and all,
This is timely and fascinating topic and a valuable collection of evidence, thank you.
About other studies looking at these topics, we looked at peer health advice in an online stroke community of 2,348 UK users over 7 years (2004-2011) in 2 studies https://bmjopen.bmj.com/content/6/4/e010501 and https://academic.oup.com/fampra/article/34/5/612/3093265. In agreement with the paper you cited (Equivel, BMJ, 2006), whenever inappropriate medical information or health behaviour were brought up (e.g. stopping a medication due to side effects), they were promptly and appropriately counteracted by forum users, e.g. advised to seek healthcare advice.
We also found that up to 95% of the users’ intents were met by the replies received.
The community was moderated, therefore the removal of inappropriate posts by moderators needs to be taken into account.
One different angle to look at peer health advice is its role in the shared decision making process with clinicians, a process that Elwyn et al called ‘deliberation’. It allows patients to become aware of choices available, understand their options and have the time and support to consider what matters most to them, described in https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3445676/. This process may require more than one clinical contact and include discussions with others. Rapley ((https://onlinelibrary.wiley.com/doi/full/10.1111/j.1467-9566.2007.01064.x) has referred to this need to talk to others, at different times and places, as a ‘distributed deliberation’ process. Recognizing this need, and allowing time for it, is a cornerstone for effective shared decision making. The results mentioned in our paper above (https://academic.oup.com/fampra/article/34/5/612/3093265) suggest that peer health advice through an online stroke community facilitated the distributed deliberation process of participants and their subsequent shared decision making with clinicians. Although in many ways not the best example, this could be compared with the number and variety of peer reviews on an item one may be interested in. If the number of reviews is high, it would most likely include a number of negative/incorrect ones. After consulting with the expert (in this example, healthcare professionals) before committing to a choice, one might feel more empowered taking the decision through reading as many peer reviews as available on the item, including the ‘bad ones’, which hopefully can be be told apart (at least this was the case in our study above).
Finally, a thought on peer health advice in established communities (where most evidence it) versus information shared, for example, in YouTube videos or on other social media and the role of superusers. In established communities, users and superusers are more likely to share a common set of norms and values, which may not be the case in less established ones. [Superusers defined as the top 1% of users characterised by the largest number of posts written in the community]. As superusers contribute with a high proportion of posts, their contribution to the overall quality of peer health advice is significant. In our study of the Asthma UK and British Lung Foundation online communities, superusers wrote 32% and 49% of all posts https://www.jmir.org/2018/7/e238/. A common set of shared norms and values is essential in self-organising social groups to prevent fragmentation. In the study above we found that superusers played a fundamental structural role in the formation of a connected network where most users were mutually reachable each other. This connected structure is a means for widespread diffusion of peer health advice. We showed that superusers were responsible for holding the network together. Removal of superusers caused the community to collapse. Without superusers there would be no effective spread of peer health advice within the community. Superusers tended to write to users with low posting activity, making other users talking and connecting to each other.
Susannah Fox says
Thank you! Fantastic additions to the collection.
Susannah Fox says
This blog is my outboard memory so I’m adding some links to studies shared on Twitter this weekend:
Quality of Medical Advice for Chronic Pain on Social Platforms (PDF) by Kaitlin Light Costello, PhD
Impact of patient-provider communication on online health information behaviors in chronic illness (PDF) by Kaitlin Light Costello, PhD
Grateful to the community for continuing to add to this conversation!
Jean says
“A common set of shared norms and values is essential in self-organising social groups to prevent fragmentation”. — Anna De Simoni
Here’s the Functional Forum, “… innovating at the cross-section of Functional Medicine and community “. Awesome interview. I have not attended. https://drhyman.com/blog/2019/11/13/podcast-ep80/
Functional Medicine– treating root causes of chronic diseases– far more than any other approach I’ve seen so far. I healed from several major chronic conditions (one life-threatening) using a similar approach. I’m not in the health care industry, btw.
Susannah Fox says
Since this blog is my outboard memory, I’m adding a resource that I found today: the Patient Information Forum (PIF)
https://pifonline.org.uk
Here’s why:
[quote]
The PIF TICK is the UK-wide Quality Mark for Health Information
Having the PIF TICK on leaflets, websites, apps or videos shows an organisation’s health information has been through a professional and robust production process. To be awarded the PIF TICK an organisation must show its health information production process meets 10 criteria.
In summary the 10 criteria for trustworthy health information are:
Information is created using a consistent and documented process.
Staff are trained and supported to produce high-quality information.
Information meets an identified consumer need.
Information is based on reliable, up-to-date evidence.
Patients are involved in the development of health information.
Information is written in plain English.
Print and digital information is easy to use and navigate.
Users can give feedback on information.
Information is promoted to make sure it reaches those who need it.
The impact of information is measured.
NHS organisations, health charities and companies creating and publishing health information can apply for the PIF TICK.
The PIF TICK is awarded after a thorough application and assessment process. Each of the 10 criteria has a number of requirements. Organisations applying for the scheme must meet the acceptable standard in each of the core criteria to be awarded the PIF TICK.
The PIF TICK criteria was developed in consultation with the Patient Information Forum’s 1,000 strong membership representing 300 cross sector organisations.
Consumer research found 80% of the public would look for a quality mark on health information. Evidence, plain English and trained staff were the top three criteria for quality health information identified by the public, although all criteria were supported.
We have created a website to provide guidance and support to members of the public who are looking for trusted health information and to highlight organisations accredited with the PIF TICK. You can find the website here: http://www.piftick.org.uk
[end of quote]
I’m not aware of an equivalent “tick” or “seal” for health information in the U.S. If there is one, please share it here.
Dave deBronkart says
Holy cow! How could we all have not known about this??
I’m going to ping some UK colleagues from the BMJ patient advisory group.