Tweetomics: an omics seminar summarized via TwitterJuly 19, 2012
[my tweets from a seminar I went to today, presented in reverse chronological order]
The middle-author problem: if you get little credit for your role in a study, where’s the incentive to check & vouch for the whole project?
Larry Kessler: As part of their training, grad students can re-run publicly available code and see whether they get the same results.
One Duke prof falsified a line of his CV, saying he got a Rhodes Scholarship … to Australia. Not really Rhodes Scholar material.
Kessler: Also, letters of concern to journals should not just be forwarded back to the authors of the article under suspicion.
Witten: How can 2 reviewers fairly evaluate a multidisciplinary 30-person paper? No way to do that, plus it isn’t their job.
Question from audience: role of journal editors and peer review in Duke case?
Kessler: Duke scientists wanted fame, promotions, grants … but weren’t aiming for royalties on diagnostic test. Wasn’t about $$ per se.
@smfullerton asks about conflict of interest, the elephant in the room. Duke scientists had vested interests in clinical trials?
Funders support discovery of exciting new approaches but are reluctant to support validation, so validation naturally gets neglected.
FDA needs to develop a guide on how to bring an omics test to the clinic. It’s been “putzing around” for a decade, Kessler says.
Institutions don’t always support multi-discipline work to ensure proper credit and accountability. What about middle authors?
There is a “bright line” between test discovery/validation on one side and evaluation for clinical utility/use on the other.
Kessler says we think we know how to do discovery and test validation, but lots of mistakes get made. Don’t validate on already-used data!
IOM report explains how to “correctly” translate omics diagnostics from bench to beside, and lists responsibilities of various parties.
Institute of Medicine report was released in March. Larry Kessler (who served on the IoM committee with Witten) will tell us about it now.
Aftermath of Duke debacle: dozens of papers retracted, careers of 162 coauthors jeopardized; public faith undermined.
The Duke problem was not just an academic one. The Duke papers, with their mistakes, were being used to guide clinical trials!
Keith Baggerly & Kevin Coombes of MD Anderson reported mistakes in publicly available Duke data (Annals of Applied Statistics 2009).
Concerns about Duke work were initially brushed off as “squabbles among statisticians.”
2006: Duke people started publishing high-profile papers on using omics to predict cancer outcomes. Others had trouble replicating results.
Challenges of omics data: (6) multidisciplinary analyses require cooperation and trust.
Challenges of omics data: (5) it’s hard to apply intuition to models with thousands of genes. Hard to tell what the data *should* look like.
Challenges of omics data: (4) expense of experiments and limited samples mean that results aren’t always validated.
Challenges of omics data: (3) complicated experiments and analysis increase the likelihood of errors.
Challenges of omics data: (2) Batch effects may cause results in one lab not to generalize to other labs.
Challenges of omics data: (1) many variables (e.g., genes) vs. number of observations (e.g., patients) leads to high risk of overfitting.
Recent @nytimes series “Genetic Gamble” highlights promise and difficulty of using genomics to guide cancer treatment.
Daniela Witten’s definition of omics: characterization of global sets of biological molecules (DNA, RNA, protein, metabolites, etc.).
Today’s topic: recommendations of Institute of Medicine panel on responsible use of omics data in clinical research.
Website for UW Biomedical Research Integrity program: http://depts.washington.edu/uwbri/
BRI seminar will feature Larry Kessler and Daniela Witten, with moderation by @smfullerton.
Now attending the UW Biomedical Research Integrity seminar “Responsible Research in the Era of Omics: Past, Present, & Future.”