One of the “top tweets” of this week’s Health Datapalooza was one I dashed off on my way to the event:
1 in 4 U.S. internet users track their own health data online – @pewinternet pewrsr.ch/khtiMB #healthdata
— SusannahFox (@SusannahFox) June 5, 2012
It was great to see how this research tidbit was treated like news — and if it’s new to you, then yes, I think it is “news.” Unfortunately, some people didn’t click through to read the report section and misinterpreted the findings. Others did click through and felt burned that it was based on a 2010 survey. I have to share the best of those tweets:
#baddata @Rock_Health @susannahfox this data is 2 yr old and didn’t mention frequency. A carney guessed my weight once #stilluseascale
— Peter Verrillo (@PeterVOrtho) June 7, 2012
Touché!
When my Pew Internet colleagues and I formulated the questionnaire for the 2010 health survey, we included two exploratory questions about self-tracking. They are not perfect, but I think they were a fine first measure. For everyone’s dissection, please see items E and F:
ASK ALL INTERNET USERS (Q6a=1 or Q6b=1):
Q24 Apart from looking for information online, there are many different activities related to health and medical issues a person might do on the internet. I’m going to read a list of online health-related activities you may or may not have done. Just tell me if you happen to do each one, or not. (First,) have you… [INSERT ITEM; ROTATE]? (Next,) have you…[INSERT ITEM]?
a. Signed up to receive email updates or alerts about health or medical issues
b. Read someone else’s commentary or experience about health or medical issues on an online news group, website or blog
c. Watched an online video about health or medical issues
d. Gone online to find others who might have health concerns similar to yours
ALWAYS ASK e-f TOGETHER, IN ORDER
e. Tracked your weight, diet or exercise routine online
f. Tracked any other health indicators or symptoms online
CATEGORIES
1 Yes
2 No
8 (DO NOT READ) Don’t know
9 (DO NOT READ) Refused
Again, we had no idea how many people would answer “yes” since this was a first-time measurement. It turned out that 15% of internet users have tracked their weight, diet, or exercise routine online and 17% of internet users have tracked any other health indicators or symptoms online. In all, 27% of adult internet users say yes to either question. That’s where I got the “1 in 4 internet users…” referenced in the tweet.
Now I need your help:
- If you’re a data geek, download the September 2010 data set, dig into the full questionnaire, crosstabs, SPSS, etc., and let me know if you see interesting stuff I missed.
- If you think these questions need improvement, tell me why in the comments below. How would you expand them? What follow-ups would you ask? Dream big: what open-ended questions would you include (the most time-consuming type of question)? Do you want to know about specific tracking activities? Frequency? Impact on the respondent’s overall health?
I personally believe that if a survey question isn’t relevant and useful to you, the health geek tribe, then we need to re-examine it. Some of the best ideas for our health surveys have come from this community (for examples, please see: Crowdsourcing a Survey 2008 and Crowdsourcing a Survey 2010). Thanks in advance for the input.
Thanks for taking my tweet the right way, criticism with a smile.
Some thoughts.
I would add currently and on a regular basis to the questions, the. Let them define regular basis with a daily, weekly, monthly, quarterly.
I would also ask them if they keep it on paper, in their head, on a spreadsheet or an app.
I’m not a survey pro, but love dissection of data. Launchrock does this well for ideas and upcoming websites, why not use the same approach for surveys. I would love to be able to characterize the N. N of 3001, but 33% came through the @rock_health tweet. Well they are likely in startups making a health app, separate that data. If their aunt Mary sees the Facebook post and fills out the survey, is she more likely to use her niece’s app than the general population?
As health entrepreneurs, you need to solve problems for the other 99%.
Hooray! I hoped that you’d be the first to comment since it was your tweet that put me over the edge to write this post.
I like the idea of adding frequency measures to the question and asking how, exactly, they track. In the report I refer to Carol Torgan’s work in this area, hoping that readers will forgive me for not thinking of that before we fielded the survey.
The 2010 question was asked only of internet users but I’m thinking that to get a more accurate measure, we should ask an initial question of all adults and then work from there — asking previously-identified internet users some online Qs, etc.
I’ll look into Launchrock, but I want to quickly add a link to some info about the Pew Research Center, where I work, and our methodology:
http://www.people-press.org/methodology/frequently-asked-questions/
The numerical data in our reports is based on what is currently the gold standard in public opinion research: the random-digit dial telephone survey. We include mobile phones in the sample (pollsters are not bound by the Do Not Call rules) and we offer both English & Spanish language interviews.
Public opinion researchers are facing significant challenges with phone survey response rates, to put it mildly. The signal is growing faint. Here’s a paper that anyone who ever cites survey data should read, by Scott Keeter, who I refer to as the Obi Wan Kenobi of polling:
http://pewresearch.org/pubs/2270/polling-survey-research-cell-phone-only-households-random-samples
The August survey will be fielded in the traditional way because it is still the most accurate measure of the U.S. population, those online and those offline. The internet impacts both groups, so we’ll continue to survey both.
Again, thanks! And let’s keep the ideas coming — I’d love to hear from more people.
I agree with the above comments. My concern: a person that googles, “what is my ideal weight,” and another that googles, “how do I train for a marathon,” would both say “yes” they “tracked” their weight and health training program on line. Addressing weekly, quarterly may address this, but I recommend attacking this from another angle.
As a physician who treats hundreds of pts with severe illnesses, I see a very large fraction of patients and family members using the internet for heath searches and you are exposing the details of this practice. Very, very rarely does anyone really track their health data over time.
The critical difference in my view between heath searching and health tracking is the *storage* of longitudinal data, so I would hit at this. e.g. do you store any of your health information on your computer or smart phone? Do you store any of your health information stored with a web-based service? Then, how often do you update this information?
When I meet a patient tracking their information over time, invariably that person is an engineer. I would bet money that very few store their heath data anywhere and update it rarely if ever. This is an exciting area for growth a la Eric Topol.
Best of luck.
FWIW, I had no problem with the 1 in 4 claim (and I did click through to read the survey results). It qualified as “news” to me because I would not have thought the number quite that high. I’m sure it could be sliced and diced much more finely – including in the ways Peter suggests above, particularly recency and frequency – but to me it’s a rough and ready indication that a large swath of ordinary Americans engage in some sort of health tracking. The fact that this was 2010, to my mind, only strengthens the general finding.
For those forming companies in and around the area of health self-tracking and quantification, and their would-be investors, I think this represents still-relevant data.
Also, Susannah, your willingness to engage criticism by providing more data, and offering to collect still more, sets a great example for others in the field. Thanks for this post.
Peter, good points particularly around potential bias in the sample population. We run into this in diabetes and chronic pain quite a bit; for example, are the poor/working poor appropriately represented in these studies? Socioeconomic factors greatly influence engagement in one’s health and separately use of technology, particularly internet technology. I would be interested in seeing Pew look at this. It has serious implications for how we address these populations and move them to self-engagent in their conditions.
We do include all socioeconomic groups in our surveys – that’s a strength of RDD phone surveys – but only “internet users” get the full battery of questions about online health tools (including the series I highlight above).
I put internet users in scare quotes to acknowledge that our definition is shifting as technology adoption & use shifts. There are people, for example, who answer “no” to our long-standing internet use questions but who answer “yes” when we ask if they access the internet on their cell phones. We’re always working to improve our surveys.
Since you may not be acquainted with Pew Internet’s research on health, here’s a tip sheet we put together — just basic findings and highlights:
http://www.pewinternet.org/Commentary/2011/November/Pew-Internet-Health.aspx
You might be particularly interested in this section of our “Health Topics” report:
Internet access drives information access
http://www.pewinternet.org/Reports/2011/HealthTopics/Part-1/Internet-access.aspx
Age, education, chronic disease status, and disability status each have a significant, independent effect on someone’s likelihood to go online.
Great opportunity to add crowdsourcing to the research methodology.
Thanks, Katherine!
Peer review by community colleagues is as important to me as peer review by Pew Research colleagues.
Pew Internet has practiced open research from the very beginning, in 2000, seeking input from experts in the field during survey design and then publishing all of our reports and data sets online, for free.
In 2008 we expanded the opportunity for peer input by posting some survey questions on e-patients.net. Some of the best ideas for expanding or sharpening questions have come from that community. For example, Amy Romano (@midwifeamy) convinced us to include pregnancy & childbirth as a health topic. Others advocated for asking about end-of-life choices.
Pew Internet researchers maintain editorial control over questionnaires, data analysis, and reports, but we welcome input from everyone and anyone interested in the social impact of internet.
Great! I think that the Pew data is helpful. I’m often weigh what the research indicates and my experience. For example, in my African American family both of my parents have smartphones, but getting them to use the apps and the functionality is challenge. Just recently, I got mom to use the Walgreens pharmacy feature that allows you to refill prescriptions by scanning the barcode on the bottle from the phone. It’s easier/faster than calling in, it took a minute for her to adopt the change. Mom’s foray with internet life was online banking, the idea of not writing out monthly bills/check motivated her start more than a decade ago. The next push can was when we got WiFi in the house and laptop. She’s now become active on FB because her church has a community. She’s also creating one for the senior center where she works. Glad to connect and share.
For me, survey design always starts with the overarching goal of the survey and then what factors are important to tease out in order to realize that goal. In this case I am making the assumption (right or wrong) that by understanding self-tracking at the population level we are developing a better understanding of how people are taking health and healthcare “into their own hands” so to speak. This is an very important aspect to understand in the current and future landscape of personal and participatory health.
For future surveys I think it would be interesting to be a more specific on the self-tracking front. For example, how would a someone who uses a Fitbit or a Withings scale answer these questions? I’ve been using both of those devices for over a year, but I’m not sure that I would answer in the affirmative to question E above. Yes, the data lives “online” and I have access to it there, but that is not where the tracking actually occurs, it occurs at the device level. With the explosion of devices and applications in the self-tracking space it makes sense to add more specificity. Maybe something like this would help:
Have you ever used a device such as a pedometer or digital scale to keep track of you exercise, weight or other health data. Was that data available online?
Have you ever used a mobile phone or tablet application to track health data or information such as food diaries?
Have you ever used a website to enter and track health data?
I realize that in surveys such as these brevity and clarity are of the upmost importance, but it might be interesting to build a question set that helps to differentiate how people are interacting with their health information through digital means.
Nail on head – thanks so much! My colleagues & I will work on a battery of questions that captures these ideas.
Great comments above.
In addition, may be worthwhile probing not just on the tools used to track health and frequency of, but also on how people interact with / act upon the data they (or others) track. Do they view it regularly? Use it to make health decisions? Share it with their doctor? Extending this further, curious what mode or expression of tracking encourages greatest likelihood of action.
Hi Susannah,
I always appreciate your perspective and your data. This topic seems as if it could go beyond the self-tracking question to begin to create a taxonomy or hierarchy to measure levels of patient engagement. So I’d start by adding to question 24, including things like:
* Have you ever sent an email to your doctor and gotten a reply?
* Have you ever gone online to look at your health data? Please explain:
** Laboratory results
** Condition list
** Immunizations
** Allergies
** Medication list
* If yes, have you take some action as a result of reviewing your data:
** Read an article
** Enrolled in a class
** Created a personal plan
* Do you have a chronic disease? If yes, do you regularly track information about that disease, such as blood pressure, weight, pulse, blood sugar, peak flows
* Do you regularly track information about your health such as:
** What you eat?
** How much you exercise?
** Moods
And then I think there could be a whole set of questions about what kinds of self-tracking tools they use. I heard a statistic last week that said that the average life span of a downloaded application is 2 weeks. So there clearly need to be some questions to measure the length and strength of the commitment as well.
Getting people to be activated and engaged in their health enough to actually make positive behavior changes is sort of the holy grail in health care, and by heaping us measure and assess these behaviors you would be doing us a huge service.
Susannah,
Thanks so much for giving us the opportunity to re-examine your data and methodology. I agree with Peter that it would be great to know how often the trackers track. I know you focus on the use of the internet, but Peter also raises a key point about tracking on paper vs. ”online.” I’ve given presentations on self-tracking to numerous audiences, most recently to health and fitness professionals (presentation / info here: http://bit.ly/HNhCrp). When I asked this audience to raise their hands if they tracked, everyone raised their hand. When I asked how many tracked at least one metric by old-fashioned paper and pencil, more than half of the hands remained raised. I see this response repeatedly.
Perhaps we should view it as an ecosystem of trackers/tracking, that includes paper/pencil, and gadgets as Ernesto discusses (there will be lots of overlap). In addition to these ‘quantified’ trackers who typically use numbers and graphs, we might also consider the ‘qualified’ trackers. As you know, I think YouTube represents a huge self-tracking community. A key issue is what percentage of the tracking ecosystem is captured in your surveys? For now, I’m just thrilled that we have some baseline metrics.
I think we also have to remember that self-tracking is just a tool for people to capture information that offers a narrative of part of their life. A bigger question is whether it ultimately leads to improved health. P.S. for you gadget fiends out there, I have a Pinterest board on wearable tracking devices: http://pinterest.com/caroltpin/wearable-tech/
Susannah!
thank you for opening your data- we had so much fun with it. Our in-house medical anthropologist and all around data geek, Melissa Maravic, took a spin through with the Eliza lens on. We were particularly curious about people we would call super trackers – we had the hypothesis that they are a unique bunch. Below are some things that caught our attention.
Super-trackers, those who endorsed tracking both weight, diet or exercise routine and any health indicators or symptoms online make up 6% of those surveyed and are significantly different in other ways than those who reported not tracking either or just one. These super-trackers are more likely to be/have (all differences reported p<=0.05 unless specified):
• Female (8% of respondents, compared to 4% of males)
• Younger (mean age of 40.72 compared to 46.2 mean age of non-super-trackers)
• Report being caretakers
• Faced a personal health crisis in past 12 months (p=0.09)
• A loved one with a chronic medical condition
• Someone close to them face a serious medical emergency or crisis
• Endorsed engaging in many other online health behaviors (q22a-29f)
• Endorsed getting information regarding a health issue from both online and a doctor or other health professional
• Endorsed getting information, care and support for a health issue from both online friend and family/friends
• Endorsed getting information from others who have the same health condition both online and offline
• Endorsed looking for online health information for both themselves and someone else’s health or medical condition
Of super-trackers who endorsed having a personal or family doctor, or other health care professional such as a nurse that they usually rely on if they need medical care, they were more likely to say:
• their health professional was “not at all helpful” or “somewhat helpful” – and less likely to say they were “very helpful” in general, and in providing the medical and health information they need.
• “other sources, such as fellow patients, friends and family” are more helpful than doctors when you need practical advice for coping with day-to-day health situations, a quick remedy for an everyday health issue, or a recommendation for a hospital or other medical facility (p=0.08).
Thank you!! This is a great example of why Pew Internet believes that open research = better research. We have posted our data sets online since the beginning of our project, in 2000, because we know that other people will see things we don’t see and bring forward insights we didn’t have time (or brains) to come up with.
If we were to go after Super-Trackers in a more serious way, what should we ask?
Do you think it’d be possible (or worthwhile) to try and gauge people’s comfort with sharing / connecting / tracking *if* they got sick? Like something that could probe different comfort levels in different circumstances. Maybe even a conjoint analysis if you’re feeling fancy.
For instance, see “Sharing Medical Data” section in http://www.jmir.org/2010/2/e19/ and also the blog about sharing diagnoses here http://blog.patientslikeme.com/2011/02/22/patient-choices-the-shape-of-sharing/
What these tell me is that:
a.) People who might not have considered sharing personal details when they were relatively well might change their mind when they’re sick
b.) That this differs by what they have (e.g. stigmatising invisible condition like HIV vs. difficult to hide, need help from others like kidney transplant)
c.) That this differs by who they’re sharing with (see for instance the difference between conditions in the blog post)
d.) If you do share somewhere, you get more comfortable with the idea of sharing
e.) Their level of comfort may vary by how anonymous they can be, what sort of questions are asked, and their level of control over sharing it. So on the one extreme you might not want Facebook telling your network you’ve just checked in at the STD clinic on Foursquare, but you might be happy to tell other patients suffering from herpes in an anonymous online community that you went for your followup appointment
On to a different tack, we all know there’s substantive drop-off in tracking using apps over time, so it might be useful to try and qualify an item with things like “for more than a month” or “more than 10 times” to help identify subtypes who stick around – are they just quantified selfers? Is this a trend that’s growing?
Finally do you think something about their willingness to make their anonymous health data public might be relevant? See for instance efforts in the UK to make “every willing patient a research patient” – would be interested in US comfort levels with that! http://www.bbc.co.uk/news/uk-16026827
Best wishes
Paul
Thanks, Paul! I was only kidding a tiny bit when I gleefully tweeted that I’d tricked you (and everyone who has commented so far) into giving me free consulting advice for our next health survey. Truly, I am grateful.
Your first question — can we ask people to imagine a future circumstance — falls outside the bounds of what Pew Internet usually focuses on: actual behaviors, hopefully recent (which is why we often ask respondents to think about the “last time” they did an activity). But some of the questions we ask do help to illuminate how people make different choices when they are in different circumstances.
For example, my colleagues and I have been working on a new report (to be released within the next couple weeks) about family caregivers, people who are helping to take care of a loved one who is experiencing significant health issues (physical and/or mental). The data leads us to believe there is a “caregiver effect” — holding all other variables constant, someone who is caring for a loved one is more likely than other people to take part in certain activities. Since many of us will be caregivers in our lifetimes, this data can serve as a window into a possible future circumstance. What will we choose to do? How will we react to the situation when we are faced with decisions for a loved one?
Other questions we include which might be a window into possible futures: have you experienced a medical crisis in the past year? Are you living with a chronic condition? Are you living with a disability? (Note we use the U.S. federal series – 6 questions in all – but we don’t survey using TTY or other assistive tech.)
So maybe by asking a basic question about health data access and/or sharing, and then categorizing the population using these sub-groups, we can create that time machine we all wish for.
I like your idea of adding a subtype of “stick-with-its” vs. “tried-it-dropped-its” when it comes to tracking. It speaks to the analysis that Stephanie & the Eliza team did regarding the Super Trackers.
What other subtypes should we consider adding?
Psst, between you & me & the internet, if you subscribed to comments on this post, I wanted to let you know that I included a sneak preview of the findings of the 2012 survey questions about self-tracking in a speech I delivered at Stanford Medicine X. The full report won’t be out until later this fall, but the video of my remarks is up:
http://medicinex.stanford.edu/videostalks/
Thank you all, again, for sharing your ideas with me!
Hi all,
The report inspired & shaped by this post is up:
Tracking for Health
http://pewinternet.org/Reports/2013/Tracking-for-Health.aspx
Please let me know what you think!