I had the great honor of being part of the first Medicine X conference at Stanford University last weekend. I presented a sneak preview of new survey results collected by the Pew Internet Project and the California HealthCare Foundation. Overall, the conference was magical, as I wrote in a previous post.
In this post I would like to explain why I interrupted the HealthTap presentation.
Earlier this year, a reporter contacted me to ask about some Pew Internet data that HealthTap was using on the front page of its site and in slide decks. Pew Internet welcomes accurate citation with attribution. That is a common use of our research, whether it is a for-profit, a non-profit, or a government agency making a case for its work.
Unfortunately, HealthTap’s interpretation of a data point was not accurate (more about why in a minute). I wrote to Ron Gutman, founder and CEO of HealthTap, and he responded pretty aggressively that he stood by his own interpretation of the data. I was taken aback. I have never had someone tell me I wrongly interpreted my own data and I wasn’t sure what to do next.
I corrected the reporter’s impression of the data and decided that was the approach I would take if I were asked about it in the future – fact checking on demand.
Here are the facts about our findings: Pew Internet has historically asked a pair of questions meant to measure the broad impact of the internet on health in the U.S. We have asked the questions of all adults, not just internet users. That respondent audience is important, so keep it in mind:
Have you or has anyone you know been HELPED by following medical advice or health information found on the internet? [If they answered yes, the respondent was asked: Would you say the information provided major help, moderate help or minor help?]
Have you or has anyone you know been HARMED by following medical advice or health information found on the internet? [Again, if yes, the respondent was asked a follow-up question: Would you say the information caused serious harm, moderate harm or minor harm?]
Some respondents get the “help” question first in the sequencing of the questions, others get the “harm” question first, to minimize bias.
People who answer “no” to the “help” question represent a population who, at that time, do not know anyone who was helped by information found online. They may have done a health search and found relevant information, but may not feel strongly enough about it to say “yes.” Or they may have looked online for medical advice and not found anything useful. We don’t know. That’s the beauty and the curse of a poll. All we know is the number of people who say the internet has been a help. It is not a question about the internet’s “helpfulness.”
The same goes for the “harm” question. The respondent is asked to recall if they or anyone they know was harmed, not to rate the “harmfulness” of online information.
Our findings have fluctuated over the years, so let’s just look at the data cited by HealthTap, from our report, Chronic Disease and the Internet, which is based on a 2008 national phone survey:
Looking back at this infographic, I have to admit I don’t like it. I can see how HealthTap could interpret it the way they did, emphasizing the “no help” finding.
However, they take it a step farther and imply that people were rating the “helpfulness” of online health information:
HealthTap’s interpretation distorts the meaning of the question. The people who answered “no” were not saying that the internet was unhelpful. They were in all likelihood saying they do not happen to know anyone who has been helped.
Also, one-quarter of adults in the U.S. didn’t even have internet access in 2008, when this survey was fielded, and 20% of internet users had never looked online for health information. There is simply no way to slice the data to fit HealthTap’s conclusion.
Again, looking back, I don’t like our question. It is too vague. Our infographic does not display the data well. I do not fault anyone for making a mistake and misinterpreting the finding. What is troubling is HealthTap’s refusal to consider my explanation.
Fast forward to Stanford Medicine X.
I have seen Ron talk about HealthTap on multiple occasions and I know the pattern of their pitch pretty well by now. On Saturday, when Geoff Rutledge of HealthTap took the stage, I was interested to learn more about their mobile platform and growing network of doctors. But then the familiar patter began. He was warming up to the same pitch I’d advised Ron to stop making. And there on the screen was the same incorrect table being described as “a recent study from Pew Internet.” I couldn’t let it pass, so I raised my hand and said in a loud voice, “That’s my study and that’s actually not how I would interpret the data.” Geoff paused, said we could discuss it later, and went on to his next point.
I immediately felt bad about interrupting Geoff. It is not my style to be disruptive and, frankly, rude. I took to Twitter to explain myself in the back channel:
- I apologize for the interruption but I have asked HealthTap to stop using that data point. #medx
- Here is the data that HealthTap cites and I disagree with their characterization http://www.pewinternet.org/Reports/2011/Social-Life-of-Health-Info/Part-1/Section-7.aspx
- Note that I think HealthTap is a fascinating model – no argument there #medx
- I feel strongly that we have a responsibility to our survey respondents to be cautious in our interpretation of their answers #medx
Denise Silber, the moderator for the panel, invited me to the microphone before their discussion began so I could explain myself. I apologized, reiterated the points I made in my tweets, and read the questions out loud. I know I went on too long because Denise had to give me the time-out sign. I was obviously wearing my data geek heart on my sleeve that day.
When the session ended, Geoff was very gracious. I have promised to send him a care package of Pew Internet data to replace that slide and I’ll post it publicly, too.
I had a few thoughts and questions to share:
- Pew Internet is going to delete the “help/harm” data points from all future slide decks. These were questions we formulated in the 1.0 era and did not include in the 2012 Pew Internet/CHCF survey – the true kiss of death in research.
- What style of conference is the right one for the health/tech field? The TED-style “sage on stage” who does not take questions? Or the scientific-meeting style of engaged debate? Or is there a place for both?
- Do different rules apply to start-ups? Is it OK to fudge a little bit to make a good point, as one might do in a pitch? Personally, I do not think people are entitled to their own facts. There’s too much at stake.
Two people came to me privately to say that they had gone back over their slides for their presentations on Sunday. They wanted to be sure that every slide, every data point, was fact checked and correct. I still regret the discomfort I caused and felt, but if it sparks a mania for facts in the health/tech arena, it was worth it.
As always, I’d like to hear what other people think. Please share questions and comments (and thanks for reading to the end of this very long post).
To my mind, one of the most important things about dialog in the modern age is the following trope:
“You get to have your own opinion but you don’t get to have your own facts.”
Very, very frequently in Health 2.0/e-Patient movements, etc., there is a tendency to recite highly positive top line statistics from studies that support a speaker’s case without examination of the what/why/wherefore of the studies bias. I can think, similarly to your point Susannah, of the number of times that a clinician or policy advocate at a roundtable will discuss the staggering adoption of “health apps” as though the category included SOLELY disease management tools and leave out the “[and fitness]” portion of the equation that covers many of the most popular applications in that domain.
Again, with the caveat that this is my OPINION: The powerful thing about where healthcare is now is that we can move from being a discipline that relies on heuristic levels of judgement (assessments backed by clinical history on an individual by individual basis and informed by the practice of care over a lifetime) and to a data-driven business but we can’t do this if the pioneers obfuscate/lie/or aggrandize about the value or meaning of the data we have on hand. The more often someone cites the raw number of pentabytes of data (without asking what the data IS) or the huge numbers of people using their EHRS (without asking if they comply with meaningful use) or the proliferation of mobile applications for health (see above) or the number of people who use The Internet for X or Y (and ignore that people with chronic illnesses who are participating in a phone poll may be on the other side of a Digital Divide (subparentheses, although that number is shrinking)) the harder time we will have convincing the traditional system that data means something.
So let’s be honest with each other to the best that we can. I think it’s totally appropriate and commendable to speak truth to power and to pull back the curtain and show people the Wizard.
You absolutely did the correct thing and acted professionally and appropriately. I consistently find that data are misinterpreted or reframed to fit a different agenda or a different spin. But to argue with the person who created the data in the first place seems a bit misconstrued to me not to mention downright questionable. I have other comments but I would prefer to share them with your privately. Meanwhile, I am happy that you stuck by your guns on this one.
I’m so glad you wrote this to clarify the way HealthTap is massaging your data to fit their messaging.
And I think it’s wise to consider the source: it’s in the best interests of Health Tap (and any business) to make it look like their product/service/people are far superior to what’s already out there. In other words, to attempt to emphasize their interpretation of your data: that 59% of people diagnosed with a chronic condition find NO HELP in health information found online – hence the clear “need” for consumers to use the Health Tap site (and attract patients to their stable of signed-on physicians).
The bad news: Ron Gutman’s odd response to your objection to the way your results were being used by HealthTap. The good news: you had immediate feedback (those two presenters who came to you afterwards!) that your objection was being heard by thoughtful audience members.
Thanks to you, there well may be hope for fact-checking after all!
regards,
C.
Thanks, all. I hope that this discussion expands and includes many voices. Since it’s just me moderating this blog and I am also on deadline for a report, I may not immediately approve comments from new folks. Don’t worry — I’m checking in even as I fact check our new mobile health data!
Don’t apologize. The best comment above: “You get to have your own opinion but you don’t get to have your own facts.”
I don’t know what you are apologizing for, on the personal side — you were/are professional. You emailed the dude prior. It’s pretty erm,ahh,uhh to interpret the data the way they did given that people cite the pew data the way like a hit song that the dj plays over and over and over again (did they think they would NOT get called out?), GIVEN that’s there’s a whole community of people who say otherwise.
I am a bit peeved (to put it mildly) at his refusal to hear your explanation. I mean why would he want to hear the explanation of a data scientist, who authored the damn study? *eyeroll* Ugh. Making money off of people’s misinformation is not a good look, furthermore it is a played out song that Michael Jackson and Paul McCartney did in the early 80s. Say, Say, Say, what you want…
To me the more interesting question is – When data is being used as part of a marketing campaign (we all know how it is used in political one) is it okay to use selective interpretations in order to sell your services and how does that impact the “trust” that people have in the source and online health information in general.
When you ask a drug company for data vs the FDA does the source matter? Or their agenda? Are you under an obligation to do a meta analysis and highlight the flaws or just select what will sell your position or your product? I think it is really challenging to set aside personal or corporate goals when presenting data and we need the pure voice of researchers like you to help keep the rest of us honest.
Now at HealthTap he is doing the same thing.. Getting doctors to provide ” Free up-to-date answers from the best doctors Ask any health question and get quick answers from thousands of trusted, U.S.-licensed physicians” at the same time claiming “HealthTap does not provide medical advice, diagnosis or treatment” (to avoid being shut down for practicing medicine online with no relationship to the patients). Is it data, information or advice?
HealthTap is very open about their agenda.. To make money by giving providers a forum to solicit new business by providing “information” to people. It uses the implicit power of a physician to drive business to the doctors who answer questions on their site (without providing any medical advice). It also allows others who might not ever go to that provider to learn. It isn’t however providing any sort of evidenced based treatment recommendations though. Ron Gutman, its founder also founded a company that bought up bloggers content, repackaged it and resold it, so using other peoples data is his standard business model.
So I can appreciate how when we move from data points, to information, and leap knowledge to marketing it can be upsetting for the researcher but it is hardly unique to HealthTap. Drug companies do it all the time and in fact we know from research that most providers also fail to provide “data” about options to patients and instead usually selectively interpret it for the patient based on their own goals.
Hi Susannah,
Myrna Morales has it spot on (“I don’t know what you are apologizing for”).
Glad you got rid of that question though, I think it was bound to be interpreted eg. it’s highly unlikely that you “do not happen to know anyone who has been helped by” health information found on the internet – especially if you happen to know a Doctor 😉
Interesting that you’re currently fact checking your new mobile health data as I found your previous Mobile Health report in 2010 was very misleading because it ignored the fact that the most used (and revenue/profit generating) mobile apps are native. I’ve explained the importance of this point in more detail in this post:
http://mhealthinsight.com/2010/10/20/susannah-fox-pew-internet-publishes-mobile-health-2010-report/
Of course if you would like any eyes to help you in the fact checking I’d be only too happy to oblige FOC as the work you produce at Pew is of great value to the healthcare industry and in the mHealth market I’ve noticed that it’s proving a great resource to help focus the efforts of the many entrepreneurs who are coming from other industry sectors and are keen to try and add value.
Thanks for that offer. I will read your post & contact you directly.
One of the tweets I loved on Saturday was from Peter Gubbe aka @EnFuegoInc:
“I see, it was a poorly designed table with typically silly questions used in many surveys. Thank You for the Clarification!”
I laughed out loud. Why? Because I love critique. I love being told when I’m wrong, so I can get it right next time. I’m supported by my colleagues (both at Pew Research and in the larger community) so I can therefore take chances, try stuff out, shine a light on a few things, sharpen my analysis, etc.
You might enjoy this post, which was sparked by another “eat your own dog food” tweet:
https://susannahfox.com/2012/06/07/unpacking-self-tracking/
And here’s a post about how I approach research in general:
https://susannahfox.com/2012/07/11/participatory-research/
Finally, you might be interested to know that Pew Internet is constantly iterating our survey questions, doing our best to measure the social impact of the internet in the U.S. even as everything shifts under our feet. For example, here’s a post about how we define “internet user”:
http://www.pewinternet.org/Reports/2012/Counting-internet-users.aspx
Again, thanks!
Susannah – Thank you, as always, for creating conversation about these important issues. The key point here is that ‘truth’ is an emergent construct. Even though we all have our favorite ideas, and can find facts to back them, it is open engagement around the data that creates knowledge. Making this process transparent, as you did, is a service to all who care about understanding.
Well said, Michael. One of the many things I appreciated about MedX was the opportunity to delve into the data itself and get away from what many conferences offer, which is unabashed hyperbole. The conference reflected and reinforced open engagement around the data and was — quite refreshingly — centered around learning and discovery rather than pitching and selling. One more reason I and many others intend to return next year.
Cosign Miss Myrna.
Susanna, this is only the tip of the Ron Gutman et al. iceberg. I’m glad you stood up to them, and so did I in the past. For more information about their history of deceptive practices, please see my post here: http://getbetterhealth.com/how-the-health-blogosphere-was-scammed/2009.01.28 and here: http://getbetterhealth.com/scammers-alive-and-well-in-the-health-blogosphere/2009.03.28 Health Tap is just Wellsphere version 2.
Dear Susannah,
As someone who has voraciously consumed all the data and insights you produce for over 10 years (yes, I have slides dating back that far with your name on them), I feel like I have a little stake in supporting its integrity.
Here’s what I’ve learned from being a student of this work:
1. Don’t try and “redo” the survey in your perspective when you are using this data. It’s tempting – “what if they asked this, which is what I really want to tell my audience, can I create that answer from this data?” It’s not a startup issue, it’s an everyone issue – we are all working to create a future for the people we serve and everyone who’s listening to us says, “where’s the data.” Instead, just study it, read it, figure out what it is saying without changing the question around. If you give it time, you’ll find that it supports your work, or supports you in working on something else :).
2. You’ve always been around for me when I’ve asked you to look at a slide or two that I created to make sure it conveys the intent of the research. It’s slightly terrifying to present anyone’s research to them when they are in the audience (I’ve done that too), so it always bears a little confirmation up front to make sure distortion is not happening.
I am also separately (and always worried) that the hype cycle trend of the day tends to pull your data off the shelf and amplify just certain points, and I know you are too, so when it results in this kind of a reaction, I have a feeling it takes a lot for you to get there.
Helping me use data well, with respect, compassion, and maybe some correction, is what a community colleague does, that’s why I when I wanted to single you out in a blog post, it’s the term I came up for you. It fits.
Best,
Ted
Dear Susannah,
I was in the audience at #MedX that day and want to commend you for your commitment to transparency and ethical behavior. You are an absolute professional.
Most inspiring.
Best,
Liza
Thank you, Susannah. Both for courage to defend your work (despite the discomfort) & for the skill with which you back up what you know in real time (standing up and raising your hand, immediately tweeting the data source, then blogging) for all of us to learn. Everything moves really fast these days.
I use the Pew Internet & Amer Life Project data in nearly every talk I give, both in the academic conference world and when speaking with hospitals, CME conferences, etc. What I love about using the data you work so hard to get right is that it reflects what we (Americans) are doing, how we explore. My hope is that your data *is* a reflection of our population. If I didn’t believe that, I wouldn’t use the numbers. I expect you (and the Pew) to ensure we get it right, too. If we don’t, your data will lose its value as will we–the voices that speak to it.
I attend traditional medical conferences, ethics conferences, and ridiculously appealing conferences like MedX. All of them approach discussion differently. At ethics conferences, your hand-raising wouldn’t be unexpected, for example. Ethicists are born and bred to argue to uncover the truth. But at a typical CME conference, your interruption could be interpreted as awkward. In my opinion, the ethics conference model may be the new normal. Social media is changing how we listen and how we transmit information & fortunately provides those usually unaccustomed to speaking out, a very nice platform. This is changing how things happen. Your actions at MedX may just be a glimpse into the future of the conference world.
The reason?
We are best at making decisions when we know the facts. Any smoke screens and opacity and we lose the sincere opportunity to choose.
When you stand up at my talk to correct me— I’ll gulp and my stomach will drop. But I will be thankful for the help.
Until then, I’m triple-checking the facts of my slides…
Following the presidential debates on Wed, the Boston Globe published “Fact Check” to delve into the accuracy of statements made in the debate. The analysis of a seemingly simple statement was quite complex. Is this what we need for conferences? (And in the classroom, where I cite Pew data all the time?)
I was present to watch Susannah stand up last week and have enjoyed the discussion she sparked here. I definitely see a trend in business in general, and Silicon Valley in particular, towards bending the truth as far as possible if it serves commercial goals. I’m all for vigorously advocating for a company’s product, and being creative in presenting the truth with the best face, but I believe that rock solid integrity may become a business requirement for companies engaged in our burgeoning social media / mHealth / health IT industry. I think the reason that there is so little integrity at large is that commercial efforts focus on short term returns — a quick acquisition, this year’s bonus, flipping your options, etc.
However, our industry faces a real problem of the commons. While companies provide legal KYA disclaimers that we aren’t providing medical advice, the reality is we’re all trying to help improve behavior, decision making, and health outcomes — otherwise we’re just wasting everyone’s time and money. If we are not honest about the value of what we do, then we will become guilty of scorching the earth for everyone else when people discover they were cheated or misled.
While this particular case was relatively mild and trying to amp up the insufficiency of the status quo vs. claiming false outcomes, blatant dishonesty and strong-arming has a fine tradition in business. Early in my career a Fortune 500 company announced our product feature set as being available in their product line nearly two years ahead of us when we could prove it was impossible for them to do it at all (they never did get there, but I had to argue with customers about it for 3 years). They timed this to kill our venture capital fundraising effort after a customer leaked our sales materials to them as part of a systematic strategy to crush competition.
My first board meeting was the morning after the press release – which is a fun story for another day. A few years later I was shocked when a senior VP at a fortune 500 chided me us for being “too honest” in forecasting the release date and feature set for our product — they had internally added 1 year to the date and discounted the performance targets we gave them because this competitor systematically lied about those things. We had to pay a heavy tax for years before we earned the trust of the market, turned those tactics against our competitor, and made integrity a fundamental business asset.
The point of this is that even one bad actor can make it hard for the rest of us to be trusted because once the well is poisoned, no one trusts the water. Having an impact on health and healthcare is hard — orders of magnitude harder than stumbling across a hit game or the latest social media fad. So reality is somewhat boring and there is huge pressure to inflate everything we do just to be exciting and to be heard above the din of all the other ‘marketeers’. The problem is that in the limit, this leads to selling snake oil — a tonic that makes you feel good and may invoke a placebo response, but doesn’t do what is advertised.
If we don’t self-police, and stand up for integrity in reporting the facts when they describe or impact the health of real people, then either we will all end up losing the trust of the people we want to help, or we will force regulators to step in and make us jump through expensive hoops to prove we aren’t selling snake oil. This might improve quality, but will definitely reduce the pace and diversity of innovation.
So, thank you Susannah for taking this socially difficult and courageous stance. I think we need more respectfully critical colleagues around to help keep us honest. Peer review after all is central to the scientific enterprise and we have far, far too little of that intellectual discipline in general across industry, journalism, and government these days.
S, I feel your pain, one introvert to another. Thank you for your courage to speak the truth, and not ignore the man behind the curtain (“I am the great and powerful Oz”). Thanks for all you do at Pew, and for sharing it with the rest of us! RF
Thanks, Russ! Myers-Briggs finds me to be an ENTJ (E is for extrovert for anyone who isn’t familiar with it) but I do find that I need long stretches of alone time. In fact, I left the building after this particular panel and the attention it drew to me so I could be alone and think about it.
And the thanks for Pew Internet should go right up the chain of command to my boss (Lee Rainie), his boss (Andy Kohut), and our funders (the Pew Charitable Trusts, and for the health portfolio, the California HealthCare Foundation). It was they who also stood behind me when I raised my hand, along with the thousands of respondents who trust us to represent them. When you see me, you should see them, too.
I need to trust the data being disseminated at cons. And I don’t ever want to wonder, “is this guy representing the data honestly?” Pew’s reputation has always been above reproach, and this man’s unwillingness to change the slide to accurately represent the Pew data is not only inappropriate, but overtly diingenuous. In other words, he is overtly and purposefully ignoring the truth and therefore actively lying. Pew is too good an organization to allow this without interrupting, or actively attempting to correct the mininformation. I say stand up and yell when you catch a lie. You. Go data geek.
Yes, you did alright to interrupt the speaker who “touched” data to support his argument. I do not see why one would not intervene, even if the “touched” data are not his. Wish I had your guts and interrupt a speaker last year who distorted data to support his view of the steep healthcare expenditure increase….
I also applaude the moderator, Denise Silber for giving you the floor to present your view, before the discussion, so that the audience remained focused on the pres and your point. After all, it was a good occasion to check the integrity of Health Tap statements…
Thanks, everyone! I can’t tell you how lovely it is to a) get this level of support for my work and b) see the discussion heading off in some very important & intriguing directions. This is why I blog & tweet & participate — to push myself & others to do the best we can, because we are working on stuff that matters.
I want to pick up on the thread about conference styles, particularly Wendy Sue’s comment.
When I attended TEDMED in DC last spring, I got into a discussion with some med students about how frustrating it was to sit in the audience and not be able to question the speaker. There was no fact checking, no opportunity to say, “Hey, wait a minute, can I see your data for that point?” There was some back-channel discussion on Twitter, but that scrolls away quickly unless someone captures it in a Storify or other form. One reason I felt confident about speaking up is that Medicine X fostered a supportive, academic-style atmosphere. I’d love to hear other perspectives on this.
I also want to pause for a moment to appreciate another theme threaded throughout the comments: the importance of integrity.
For the record, this is certainly not the first time Ron Gutman has played a little loose with the actual data. In his well-viewed TED talk about the “power of the smile,” Gutman misinterprets and mis-represents, in my opinion, the research he claims he’s summarizing. And that’s so frustrating because as a scientist, I want people to stick to what the actual data show — not what we think it shows, or what we want it to show.
My blog from a year ago about the issue is here:
http://psychcentral.com/blog/archives/2012/08/19/ron-gutman-smiling-while-confusing-correlation-with-causation/
And as I note in footnote #4, HealthTap actually deleted my short comment to their blog highlighting Gutman’s talk. In the comment, I simply pointed out that causation does not equal correlation. And one study does not constitute a robust finding in psychological research.
As Dr. Val reminds us, Gutman was also the brains behind Wellsphere. Why anyone would be interested in HealthTap after what happened at Wellsphere is beyond me. As my writer friend Thomas Derr says, “Enthusiasm is not a business model.”
Dr. John: In answer to “Why anyone would be interested in HealthTap after what happened at Wellsphere is beyond me” – apparently, lots of people with money are, including Google’s Eric Schmidt and angel investor Esther Dyson (herself a featured speaker at MedX). More at:
http://techcrunch.com/2012/10/05/esther-dyson-homebrew-health
As you write in that post, John, Ron is likeable and enjoys smiling. After talking with him about his personal observations of how smiling helps his running, I tried it and found the same effect — if I smile, I seem to run better.
Now, it doesn’t take much to make me smile. And I am quite sure I look like a lunatic as I run down Mass Ave, smiling at everyone I meet (which, at the hour I run, is mostly workers on their way to the Japanese, British, and Korean embassies, either smiling tentatively back or looking away).
I like how smiling makes me feel (who doesn’t?) and it would be nice if smiling more lengthened my life. But I’m not going to count on it. I am going to count on that run, though, which I enjoy even more thanks to Ron.
Thanks for the anecdotal data point. Just to be clear, there’s no research that suggests ‘forcing’ yourself to smile actually is the same as smiling naturally. So if you’re not doing it naturally, the research data suggests it won’t help much.
Susannah and John,
About smiling: you’ve brought to mind a blog, When happiness is a hazard: http://www.caretoknow.org/story/when-happiness-hazard.
As a short-sighted (talking eyes, not outlook) I regularly smile at what turn out to be complete strangers – with um, interesting results – so I take the lack of data to heart.
And on the subject at hand, I can only add to what’s already been clearly said. Who’s the more credible source Susannah and PEW or smiling Ron. ( I haven’t the data to judge based on like-ability)
Thanks Susannah, for starting, moderating and being open to everything and anything.
Me, I’m smiling as I write this.
Kathy
Hey, I am a “smile scientist” and I have to tell you, we certainly have our data misinterpreted, quoted in pieces, and generally mangled more than many other researchers. Who doesn’t want to talk about smiling? It’s frustrating.. real data without neat interpretations are always more interesting, IMO
Kathy, your story reminded me of a friend who was planning a trip to Russia. He was specifically warned (by Russian ex-pat friends now living in Seattle) NOT under any circumstances to smile at passing strangers on the streets of Moscow – for fear of being taken for being either “feeble-minded” or “American”!
this is so interesting to me. I have never attended a conference at which interruptions were not permitted..it might even drive me to tweet! I didn’t even realize how revolutionary your standing up for the real data was until I read all the comments here.. With a good moderator in place, the best part of conference presentations is always the speaker vs. the scientist cited, when the sparks can fly, respectfully of course. It would have been fun to see you in action.
Thanks, Karen! Apologies for the delay in approving your comments — went to NYC for another conference and didn’t check until now. And yes, the back channel of Twitter is a pretty good substitute for having a voice on stage or in the Q&A (but yes, I’d always rather have a mic, not just my phone, when my own research is being discussed).