Career Development Course for Post-docs February 2, 2009 Critical Reading and Evaluating Scientific & Medical Literature Fred A. Pereira, Ph.D. Huffington.

Career Development Course for Post-docs

February 2, 2009

Critical Reading and Evaluating Scientific & Medical Literature

Fred A. Pereira, Ph.D.

Huffington Center on AgingBobby R. Alford Department of Otolaryngology–Head and Neck Surgery,

Department of Molecular and Cellular Biology,Co-Director, Graduate Program in Cell and Molecular Biology

Bioengineering Department (Rice University)

Communication

• Critical Reading and Evaluating the Scientific and Medical Literature

• Oral Presentations Dr. Gayle Slaughter

• Posters Dr. Gayle Slaughter

• Scientific Writing I-Critical writing and manuscript preparation Dr. Susan Marriott

• Scientific Writing II-Reviewers, revisions, and responding to critiques Dr. Susan Marriott

The Scientific & Medical Literature

Reading/Strategies• Sources• The parts & Reader

Expectations• Strategies

– Skimming– Scanning– Concept mapping– Cornell notes

Evaluating• Pitfalls / Jargon• Questions • Significance• Scientific Arguments

– Valid/Persuasive

– Statistics

– Other Biases

• Scientific Method • Philosophy• Ethics

The Scientific MethodAsk a Question

Do Background Research

Construct Hypotheses

Test with Experiments

Analyze Results / Draw Conclusions

Hypothesis is True Hypothesis is Falseor Partially True

Report /Publish Results

Rethink Hypothesis

Philosophy of Scientific Inquiry

• A quest for knowledge and truths

• The scientific literature is a discussion– Policed by peer review

– Textbooks, Lectures, Seminars have truths (at the time of writing!)

– Reviews, Mini-reviews and Perspectives report truths (at the time of writing!) – Classical articles

Ethics: Definition of Research Misconduct

Research misconduct means fabrication, falsification, or plagiarism in proposing, performing, or reviewing research, or in reporting research results.

(a) Fabrication is making up data or results and recording or reporting them.

(b) Falsification is manipulating research materials, equipment, or processes, or changing or omitting data or results such that the research is not accurately represented in the research record.

(c) Plagiarism is the appropriation of another person's ideas, processes, results, or words without giving appropriate credit.

(d) Research misconduct does not include honest error or differences of opinion.

http://ori.hhs.gov/

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Houston Chronicle October 30, 2008

• TSU professor accused of plagiarism

• Chu's proposal is an almost verbatim copy of Miller's, according to copies of the proposals provided to the Houston Chronicle.

• Miller's [2003] grant request begins, "We propose a two-year program to develop a prototype system designed to detect biological warfare agents (BWAs) by probing their linear and nonlinear dielectric properties."

• Chu's begins: "We propose a three-year program to develop a prototype system designed to detect biological warfare agents (BWAs) by probing their linear and nonlinear dielectric properties.”

• His proposal did not mention Miller or Miller's work.

Reading: Sources for Literature• PubMed – My NCBI – Habits

– http://www.ncbi.nlm.nih.gov/sites/entrez– http://www.ncbi.nlm.nih.gov/sites/gquery

• HighWire Press– http://highwire.stanford.edu

• MDAnderson Cancer Center– http://www3.mdanderson.org/library/online-journals/jrnls.html

• HAM/TMC (are you Registered!)– http://ca3cx5qj7w.search.serialssolutions.com/

• Rice U.- Fondren Library– http://ps4ps6lm2r.search.serialssolutions.com/

• Faculty of 1000 – http://www.f1000biology.com/browse/

• Science Podcasts-Nature, Science, PBS• Faculty/Experts in field/Peers

Strategies: Parts of a scientific article:Title

Authorship

Abstract

Introduction

Methods

Results

Discussion

References

Background

Question

Experimental Approach

Results

Answer

Significance

Modified from S. Marriott

Title/Running title

Authorship

Science 30 January 2009: Vol. 323. no. 5914, pp. 630 - 633 DOI: 10.1126/science.1166175

Survival from Hypoxia in C. elegans by Inactivation of Aminoacyl-tRNA Synthetases

Lori L. Anderson,1 Xianrong Mao,1 Barbara A. Scott,1 C. Michael Crowder1,2*

Hypoxia is important in a wide range of biological processes, such as animal hibernation and cell survival, and is particularly relevant in many diseases. The sensitivity of cells and organisms to hypoxic injury varies widely, but the molecular basis for this variation is incompletely understood. Using forward genetic screens in Caenorhabditis elegans, we isolated a hypoxia-resistant reduction-of-function mutant of rrt-1 that encodes an arginyl–transfer RNA (tRNA) synthetase, an enzyme essential for protein translation. Knockdown of rrt-1, and of most other genes encoding aminoacyl-tRNA synthetases, rescued animals from hypoxia-induced death, and the level of hypoxia resistance was inversely correlated with translation rate. The unfolded protein response was induced by hypoxia and was required for the hypoxia resistance of the reduction-of-function mutant of rrt-1. Thus, translational suppression produces hypoxia resistance, in part by reducing unfolded protein toxicity.

Background

Question

Approach

Results

Answer

Purpose: To help the reader understand and interpret the contents of the paper.

Content: Any background information and motivations essential to understand the specific study should be stated here.

Terms: Uncommon or novel terms to the general scientific community are defined, and prior knowledge / information are cited here.

Introduction - Background of Study

Modified from S. Marriott

Topic Position

Stress Position

Supporting Sentences

For studies in which the results of one experiment determine what the next experiment will be, the story consists of a repeated four-part pattern:

• Subheading

• Question/Topic position

• Experiments• Results

• Answer/Stress position

Results

Modified from S. Marriott

www.biosciencewriters.com

• Transition/Background

• Question/ Topic position• Experiments

• Results• Answer

• Interpretation/Transition• Stress position

Results Continued

S. Marriott

A Discussion has three parts:The beginning states the answer to the question and provides evidence to support the answer.

Does not begin with a summary of the results.Does not begin by repeating the Introduction or writing a new Introduction.

The middle explains the answer, thus indicating how the answer fits in with previous work.

The end states recommendations, applications, implications, or speculations, thus indicating the importance of the answer.

Generalities:The discussion includes alternative explanations of results.

Topic sentences tell the overall story of the study – found at the beginning of every paragraph, either alone or in combination with transition words, phrases, or clauses, repeated key terms, and other techniques of continuity.

The answer should be clearly identifiable at both the beginning and the end of a Discussion.

In each paragraph, supporting sentences should be organized to support the topic sentence.

Does / should not include tangential topics.

The Discussion

Modified from S. Marriott

Reader Expectations• Every unit of text, no matter the size, serves a single function or

makes a single point.

• Topic position: Old information - the information that begins a sentence establishes a perspective for viewing the sentence as a unit.

– Provides linkage (looking backward) and context (looking forward).

• Supporting sentences: knowledge to support the topic position.– Subject - Verb Separation: Readers expect a grammatical subject to be followed

immediately by the verb. – Anything of length that intervenes between subject and verb is read as an interruption, and

therefore as something of lesser importance.

• Stress position: New information - saved the best for last.– Readers naturally emphasize the material that arrives at the end of a sentence.

– Commas and semicolons are used to create units of thought in long sentences. Modified from George D. Gopen and Judith A. Swan, Scientific American 1990

“If the reader is to grasp what the writer means,the writer must understand what the reader needs”

• Readers do not simply read; they interpret. All sentences are

infinitely interpretable, given an infinite number of interpreters.

• As the complexity of the context increases moderately, the

possibility of misinterpretation or non-interpretation increases

dramatically.

• Being conscious of these locations, we can better control the

degrees of recognition and emphasis a reader will give to the

various pieces of information being presented.

George D. Gopen and Judith A. Swan, Scientific American 1990

Strategies – Skimming an article

• Title (Not always true/justified – quality of journal)

• ISI citation index: Journal Citation Reports & Citation Index• http://admin-apps.isiknowledge.com/JCR/JCR?SID=S266COIkpLd1pCKFmJd

• http://apps.isiknowledge.com/UA_GeneralSearch_input.do?

product=UA&search_mode=GeneralSearch&SID=S266COIkpLd1pCKFmJd&preferencesSaved=

• Abstract (Not always informative/justified – journal type)

• Figures/Legends (Results)

• Discussion

• Introduction

• Methods

Strategies – Scanning for Information

1. What was the major purpose of the study? (Intro)

2. What source/organism(s) was studied? (Intro/Methods)

– Is it the same one you would study?

3. What variables were studied? (Intro/Results/Figures)

– Are they the same ones you would use?

4. Did any conditions differ between other studies and this one?

(Methods/Results)

5. What were the major results of the study? (Results/Figures)

– Pay special attention to figures and the author’s description of them.

6. Determine what the author considers the major contribution of

the study. (Abstract/Discussion)

Strategies – Concept Mapping

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

• to generate ideas (brain storming, etc.)• to design a complex structure (long texts, hypermedia, large web

sites, etc.)• to communicate complex ideas• to aid learning by explicitly integrating new and old knowledge• to assess understanding or diagnose misunderstanding

http://users.edte.utwente.nl/lanzing/cm_home.htm

Strategies: Cornell Notes

• Promotes active learning – Each major point on the right ought to have a question or keyword on the left.

• Left column: write questions and keywords that relate to the information recorded on the right

• Right column: make notes, draw graphs and record the important information from the lecture or reading

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

1/4 3/4

•notes

•draw graphs

•record important info.

•write questions

•keywords

•Summary •conclusions

Evaluation – PitfallsSome literature is better than others!

Some authors are smarter than others, some are right more often – sometimes these are not the same people – Opinion Leaders

It is an art of storytelling – most often not chronological.

Some articles are driven by special interests (tenure/promotion/business).

Data are plentiful but truth is elusive!Be skeptical but not cynical.

Demand excellence – be critical but be fair.

Find balance between believing everything and believing nothing.

IntimidationScientific Jargon – Get a review

Use medical dictionary – e.g. http://www.thefreedictionary.com/

Consult technical literature – e.g. http://www.ambion.com/techlib/resources/RNAi/index.html

Scientific Jargon: the Mysterious Language of Science and Medicine

It has long been known….. I didn’t look up the original reference.

A definite trend is evident…..These data are practically meaningless.

While it has not been possible to provide definite answers to the questions….. An

unsuccessful experiment, but I still hope to get it published.

Three of the samples were chosen for a detailed study….. The other results didn't

make any sense!

Typical results are shown….. This is the prettiest graph I could muster.

These results will be in a subsequent report….. I might get around to this sometime,

if pushed / funded.

The most reliable results are obtained by Jones….. He was my graduate student; his

Thesis depended on this.

Correct within an order of magnitude ... Wrong!

Modified from Dyrk Schingman, Oregon State

What is a logical argument?

Premise: If you overslept, you’ll be late. You aren’t late. Therefore:1. You did oversleep.2. You didn’t oversleep.3. You’re late.4. None of these follows.

Premise: If you overslept, you’ll be late. You didn’t oversleep. Therefore:1. You’re late.2. You aren’t late.3. You did oversleep.4. None of these follows.

An argument is a set of statements consisting of premises and a conclusion; normally the premises give evidence for the conclusion.

Modified from Introduction to Logic, Harry Gensler, 2001

Valid and Sound Arguments• An argument is valid if it would be contradictory to have the premises all true

and conclusion false.

• An argument is sound if it is valid and has every premise true.

Valid and true premises!If you’re reading this, you aren’t illiterate.You’re reading this.You aren’t illiterate.

First premise false!All scientists are millionaires.Joe is a scientist. Joe is a millionaire.

Conclusion doesn’t follow!All millionaires eat well.Joe eats well. Joe is a millionaire.

Modified from Introduction to Logic, Harry Gensler, 2001

When we try to prove a conclusion, we try to give sound arguments. We must make sure of two things: our premises are true, and our conclusion follows from our premises.

Evaluation – Questions asked, answered and if a persuasive

scientific argument is made – Significance

• Defined the specific problem of research.

– Cited reviews and articles in high impact journals.

• Indicated the importance and the hypothesis.

• Placed the problem in the intellectual context of field.

• Gave reasons and detailed the experimental methods.

• Detailed accomplished and expected results (Statistics).

• Clearly identified the potential unique contribution (Significance).

• Suggested possible theoretical or practical relevance of findings (Biases

vs Implications).

• Did they answer the question? Was the right question asked?

The Scientific & Medical Literature

Reading/Strategies• Sources• The parts & Reader

Expectations• Strategies

– Skimming– Scanning– Concept mapping– Cornell notes

Evaluating• Pitfalls / Jargon• Questions • Significance• Scientific Arguments

– Valid/Persuasive

– Statistics

– Other Biases

• Scientific Method • Philosophy• Ethics