I’m going to teach a 90-minute class on participatory research at Stanford Medicine X in September, so I’m going to start blogging resources I plan to incorporate (or that simply inspire me). As always, I’d welcome suggestions, comments, and questions.
First up, Kate Crawford’s Strata 2013 talk about “big data”:
Dave Clifford says
Susannah,
Part of this is really just about sensor bias. I.E., your underlying data set is dependent on the quality and placement of the sensor that you use to collect it. Humans are just another sort of sensor. This is the problem (now) with a lot of quantitative social sciences research because, simply, of the number of people that won’t answer a phone call and spend 20 minutes responding to structured questions (as of course you know). What we’ve constructed instead, as Kate Crawford points out, is a weird sort of panopticon where we can perceive certain sorts of extrinsic behaviors (Where I tell you I am and maybe why it’s important to me, what I’m doing, how I’m feeling) through social media – some intrinsic behaviors (How I move, for example – E.G., Ginger.Io) but it’s difficult to elicit things from people simply because they don’t have time.
I have a variety of other thoughts that are spinning out in tangents on this including:
— How are humans “sensors” in other ways – We make cohesive frameworks for data all the time, for example, and sit on top of massive information streams and pull the interesting bits out, compile them, and pass them on (Joan Didion’s “We Tell Ourselves Stories in Order to Live)
— If humans don’t answer questions they don’t want to and instead expect people to assess explicit and implicit behaviors passively, what does quantitative research look like and how do we create an accurate sample
— What’s the “valid” methodology there?
— How do we compensate people appropriately?
— What happens when the academics of the baby boom die, because academia is a useful sort of “sensor” and it’s (nearly) impossible to join its ranks anymore (I.E., bright people in my generation focused on solving problems rather than thinking about solving problems, and I think we’re going to have a “pondering” gap soon because of it)
— Why do people use multiple free text questions in online surveys and who allows people with methodological training that shoddy to be Ph.D. candidate (that’s a separate rant)
Susannah Fox says
Thank you! I had a similar reaction, watching the talk, I think (“um, duh, Twitter is a poor mirror for society”) but I liked her framing and conclusions. And I’m looking for questions like the ones you pose, not because I think I can answer all of them, but because I want to at least keep them in mind, to stay humble in the face of this work of trying to measure and explain what’s going on out there.
Melissa Hogan says
Great stuff – now you’ve got our heads spinning…. Would love to take part in that master class.
Susannah Fox says
Thanks, Melissa! It’s going to be a participatory curriculum so you’re already taking part. I’m hoping to create a process similar to the concept of a Jewish wedding as a year-long simcha or celebration. From the moment of engagement, the wedding begins. Every party, every dinner, every toast in the couple’s honor is part of the wedding. The preparation becomes the event. I speak from experience when I say that this approach took the pressure off the Big Day and made us feel like we were part of a community, not stars of a Broadway show, one night only. So: welcome!
Susannah Fox says
Also: I created a Storify to capture related tweets, documents, etc:
http://storify.com/SusannahFox/participatory-research
Melissa Hogan says
I watched this again to scratch a mental itch I was having. In thinking about bringing big data together with small data, one thought that struck me is that is what you’ve been doing with data I’ve seen about rare disease, because you’ve come into the community and are observing, participating, and sharing. And for sure, that experience would color (I would say for the better) your analysis and reporting of big data regarding the rare disease community.
However, what then struck me is that often researchers, physicians, and others in the role of analyzing or helping a specific community seek to remain “objective” without becoming emotionally or subjectively involved, in their view, to view and report the data without bias. But when the data inherently contains bias, is it possible to use subjective experience to remove that bias, or does it create more?
I would love your thoughts on this, as I originally imagined it in the context of your experience. I also picture it in the context of Case’s clinical trial and possibly related pharma and biotech research where there are often two distinct camps – those who feel that meeting patients and their families is a positive for researchers and those who fear doing so could compromise their research.
Susannah Fox says
Melissa, I wanted to let you know that I’ve been thinking about this comment since the moment you posted it. Still thinking… may call in some other expert views from the points of view of clinical, social science, patients, caregivers. Thank you for digging deep!
Susannah Fox says
If I had a magic wand, we’d be sitting fireside having tea, discussing your question with my (online) friend Julie (@SolidFooting) who wrote this post about her experience meeting the researchers who are working on her daughter’s rare recessive genetic disease:
A Little Pot of Yeast
Then I’d bring in the Heywood brothers, who founded ALS-TDI and PatientsLikeMe in order to BE the researchers who are working toward a cure for their loved one’s disease. And since I’d have a magic wand, I’d make sure Stephen was there, through some time-machine or spirit-world connection.
Then I’d bring in Francis Collins, director of the NIH, who serenaded the Moebius Syndrome conference last year, singing about how people living with rare disease inspire him.
But what about the other side? Who should we invite to tell us about how they do not want to meet the people who have the disease they are researching? Who is doing the research to show that emotions do create a bias in scientific discovery?
Melissa Hogan says
Very good question. Where I’ve heard the issue come up is with some academic researchers and some basic and translational scientists inside and outside pharma. I am fully of the opinion that meeting patients *does* help as there are some unique insights into the effects of the disease as well as trends involving the disease since many in a rare disease community know and interact with many more with their disease than most doctors will see in a lifetime. In addition to scientific or disease insight, meeting patients offers a connection from their research to real world effect, i.e., purpose, which I would venture to say we’d all like to have in our work. I spoke once at a pharma conference and several scientists said I was the first patient they’d met and it made them see more tangible purpose in their work. Now on the flip side, I’ve heard the argument that if scientists get emotionally involved, they might cut corners or overlook mistakes in an effort to speed along science. I think reading the book “The Cure” gives some merit to that argument, but doesn’t tell the whole story.
Lisa Jarvis says
Much food for thought, Melissa! Everyone I’ve talked to on the pharma/biotech side thinks patient input is invaluable. The emotional context is a given; stepping outside the lab and seeing the real effects of your research is hugely motivating. But I’ve also been told those meetings provide so much time-saving information about the practical needs of running a clinical trial (for example, on a very basic level–can people swing a three-day visit vs. a four-day visit to a trial center?) as well as what the end product needs to look like (for ex: if IV, how often is practical vs prohibitive). That said, this level of contact with patients is new for many companies. Although the Genzymes of the world have long incorporated patient perspectives into their process (The Cure shows the start of that, as well as the pitfalls, but I think that case is the exception rather than the rule in terms of clouding judgment), but that’s a cultural shift for the big pharmas that have gotten into the rare disease space. In terms of discovery, would be good to circle in some academic researchers being funded by patient orgs and get their perspective–the few I’ve spoken to seem seriously committed and understand the time pressure. I don’t get the sense its causing them to cut corners, but could instead be a way to drive efficiency in drug discovery (not to mention collaboration between academics that previously worked independently).
Jamie says
Susanna,
I want very much to come to your fireside tea party. Stephen is always there as we collaboratively create meaning from his absence. I am pretty sure he would want vodka not tea though!
Thanks for the lift on a Friday afternoon.
-Jamie
Julie says
I’m in for fireside tea party as well. Thinking on all the comments above and wondering if we worry too much about research bias and how emotion or personal affinity to a patient may affect things.
We are human and the relationships we form, in so many different ways, are invaluable. Do we always have to control for them? Can they not be part of the bigger picture and simply accepted? (Oversimplification?)
As a parent of a former ‘undiagnosed’ child – now a ‘rare disease parent’, I count on the personal relationships. I want them to know my daughter, to understand she is more than her disease. I respect the attitudes of the physicians and researchers who don’t want to get too close, but I have a stronger affinity for those that have a more personal approach. It makes me feel like they care more, and in invested. I want that person in the fight for my child.
Erin Gilmer says
In what context are you using this video? And how are you defining “participatory research”?
My initial reaction to that term is the participation of patients in medical research. I recently wrote 2 blog posts about patients in the research process – one on Sharing Clinical Trial Data (http://healthasahumanright.wordpress.com/2013/03/30/sharing-clinical-data/) and one on my experience as a PCORI patient merit reviewer (wp.me/p1sgji-dI). Which I think relate to the description of your class (http://medicinex.stanford.edu/2013/04/04/a-master-class-with-susannah-fox/).
Would love to keep in touch as you develop your ideas and curriculum.
Susannah Fox says
So far I’m using the video as a way to start a conversation — and it’s working! Thank you so much for pointing to your blog posts. I read them with my jaw hanging open.
I’m hoping that by explaining how I practice participatory research in the setting of the Pew Research Center, and gathering examples of other types of participatory research, we can create (as a class) a framework and a shared understanding of how these models can be used in many contexts. Essentially, I’ve gained more insight from listening/doing fieldwork than I could have hoped to do if I stuck with the traditional phone survey methodology. I want to not only share that with my social science colleagues, but with everyone, in case it kindles something in their own work.
This is just the beginning of the journey. I’d love to have you as a member of the wagon train, heading West toward the promise of something new.
Erin Gilmer says
Happily aboard!
I laud your efforts and glad to contribute. Thank you for listening to this epatient, patient advocate, lawyer, IBM writer, Nebular Health Tech leader’s voice. When we bring together voices from around the world, we learn so much. And ultimately, we get to change our world together.
AfternoonNapper says
“But what about the other side? Who should we invite to tell us about how they do not want to meet the people who have the disease they are researching? Who is doing the research to show that emotions do create a bias in scientific discovery?”
Allow me to pretend:
I am a genetic researcher. I work in a lab. I stare through a microscope at de-identified samples. The samples contain no identifying information because the research sample is small, and I, given enough characteristics and curiosity could most likely track down the patients to whom my samples belong. Consequently, there is another worker who collects the samples. I never meet the patients. I only know data. Data is a funny mirror. It reflects not just what we see, but all that we do not see. I see my data clearly; however, should I see the humans whose samples generate the data, my perspective would be skewed. In them I would see the clause in the research agreement that states they will receive no information about their own condition as a result of their participation in my study. In them I would see what I now know but they do not. In them I would see all the many times before that research has failed to provide an answer. In them I would see my fear of being yet another research to fail to provide an answer. So I choose not to see them at all.
——-
And thus, I do not pretend:
I am a patient. I live with a chronic, incurable illness. I stare at my computer screen most days, reaching out across the Internet to connect with others who are like me. Finding people who are like me makes me feel so much less alone. I feel as though my army, however small, is still mighty. We share resources, lessons we’ve learned in caring for ourselves and being cared for by others. We trade journal articles and quote research studies like sports scouts quote statistics. We know all the players by name, by interest, by institution. We join wait lists. We travel great distances. We define acceptable risk differently. Data reflects not just what we see, but all that we do not see—and yet there are great gaps in what we as patients know and what the data itself scientifically proves. We are the humans whose samples generate the data, and our perspective is skewed. We know that our own struggles may very well not provide answers for us but only for the patients who come after us. We know that research funding ceases, departments are cut, and phase two trials ended. But we also know hope. In those who research our conditions and cures, we see potential. We see reason to sign the clause in the research agreement that states we will receive no information about our own condition as a result of our participation in their study. We see a team member who may not understand what it means to live with a disease but who is our chance of living without it. We want to meet the researchers who research us. We want to say thank you for doing what we are not able to do, but allow us to do what we are able, which is give ourselves to you so that you may continue to give hope to us.
Susannah Fox says
Beautiful – opened up avenues in my mind. Thank you!
Erin Gilmer says
Agreed. This is exactly how I feel as a patient. Thank you.
Liza Bernstein says
Hi Susannah —
Erin Gilmer suggested I alert you to the post I wrote about what’s happening over on the #BCSM Community website. I called it “Collaborative Medicine in Action.” It hints of future aspects and possibilities of collaborative design of clinical trials. http://itsthebunk.blogspot.com/2013/04/collaborative-medicine-in-action.html
Looking forward to seeing you at MedX!
Best,
Liza
ps: I love your idea of the year-long Simcha vs the one-night Broadway show. More participatory/collaborative medicine in action!
Susannah Fox says
Thanks, Liza! I added your post to my Storify so it’s captured there, too. I can’t wait to see how the researchers react to what I’m sure are yeasty, yummy comments in that one open text box of their survey.
One of the most important early lessons that Tom Ferguson taught me: be sure to have a no-character-limit text box attached to every single question on an online survey. And always include very open questions at the end along the lines of “If there is anything about you that you think might help us to understand your experience, please let us know…” and “Is there anything else you’d like to add or areas we should have asked about?…” One person wrote, “I always wear black.” It actually explained a lot about their answers!
Liza Bernstein says
Thanks Susannah! That is so interesting — the person who wrote “I always wear black”! I would have loved to see what her answers had been.
IMO, science, business and other fields deeply influenced by Positivism are so enamored of data, metrics, quantifiables, that too often, the CONTEXT, which can only be illuminated by a qualitative approach, is lost. And so often, that context can help to clarify what is actually going on.
In other news, the #BCSM community is at it again. I posted about it here: http://itsthebunk.blogspot.com/2013/04/more-collaborative-medicine-in-action.html
While not technically a formal research study/clinical trial, Dr Deanna Attai’s (@DrAttai) idea to crowdsource from (with? within?) a community of interest is another example of what can be achieved through collaboration. Inspiring!
Susannah Fox says
In an effort to bring more people into the conversation, I tweeted:
Have any researchers publicly stated that they do not want to meet the people who have the disease they are working on?
A few replies:
@nickdawson: assuming a “no” doesn’t translate into actively seeking out the subjects of research either.
@susannahfox: I assume the same. Wondering if anyone has tested for bias (or noticed a difference).
@afternoonnapper: Fascinating question. Flip it too. Pts who would like to meet the researchers.
(See her comment above for a beautiful elaboration.)
@annbartlett: How could a researcher not meet pt? At the human trial level, I would think they have to meet the pts.
@annbartlett: Also, pt enrolling in clincial trial need face to face to lessen fear and participate.
@afternoonnapper: Disagree re: need for f2f for trial participation. Need to make a difference. Need to improve future.
@savingcase: I met a researcher on Case’s clinical trial drug once, only for a few minutes. Would have loved hours.
@mhoskins2179: I’ve heard some scientists compare to appeals court cases: Not about person (patient), but broader legal issue (science) 4 ppl
@mhoskins2179: Maybe not those researchers confined specifically to a lab, though. Might have no benefit. (?)
@aureliacotta: I’ve heard plenty of researchers say it. (Overheard) common with pathologists who refuse to meet patients.
@aureliacotta: hey can’t admit it publicly. Many feel awkward, uncomfortable. Not highly socially skilled. Could be fixed IMO
@diabeticfury: Is there any one so wise as to learn by the experience of others? #Voltaire
There’s quite a bit to discuss here if anyone wants to elaborate, ask a different question, or otherwise react.
Mike Hoskins says
Thanks, Susan. Interesting topics.
Yes, I think so much of the discussion depends on the type of data and research we’re focusing on. Scientific research in labs might be one branch, where patient interaction isn’t necessary because they’re really trying to remove the personal bias that comes with an individual. That’s where I have heard some scientists refer to it like appellate court cases – “it’s about the bigger legal issue (or science), not the particular person’s case (or the individual patient).” But in some ways, the scientists can benefit from seeing the patient and be motivated, even if it doesn’t impact the science it speaks to their human emotions. As a patient, I’d totally want to meet the researchers to see what they’re doing – that may not apply to everyone, though. Of course, in other types of data research, like with Big Data and analyzing social issues or even social media use, the personal aspect is a key to understanding what we’re even looking at when it comes to data. Of course, I think there’s also a whole lot of nuance and intersection/overlap in any area, so it’s not completely clear cut examples of A, B or C for each topic we’re looking at.
Katherine Ellington says
The idea that data points hold the power of stories and connection lays out a framework the makes mixed methods research important especially among minority and vulnerable populations. For example, “portraiture seeks to unveil the universal truths and resonant stories that lie in the specifics and complexity of everyday life” and/or ethnography are approach to adding more social context to social media data. I was also in New York City during Hurricane Sandy on the Upper Westside of Manhattan and communicating with my family and friends in Queens, Brooklyn and New Jersey who were without power using FB and Twitter my timeline notes were helpful (they could see message when ephemeral opportunities to plug-in) as I sat in one place thinking about neighborhoods and people I know well. For example, in the neighborhood where I grew up in Queens people assume that if there’s a power outage the electric company knows about the situation that consumers don’t have to call-in or make contact. During Sandy I reminded people that calling, emailing (or using the Con Ed app) to report outages was essential for tracking and resolution. This message spread among community leaders too so the help and support could reach those who needed it.
As early as possible in research efforts data has to connect stories to avoid the trap of bias and (missed) signals. Mixed method and other qualitative research can remedy these challenges.
Susannah Fox says
Mike, Katherine,
Powerful comments — thank you so much!