Barry Braun, Ph.D. Basics of Study Design The Basics of Study Design Barry Braun, PhD, FACSM Associate Professor Director, Energy Metabolism Laboratory.

Barry Braun, Ph.D. Basics of Study Design

The Basics of Study DesignThe Basics of Study Design

Barry Braun, PhD, FACSMAssociate ProfessorDirector, Energy Metabolism LaboratoryDepartment of KinesiologyUniversity of MassachusettsAmherst, MA

H

H

OH

OH

CH2OH

H

OH

OHH


A fairy taleA fairy tale

While boardsailing in Belize, physician/ scientist Dr. Dulcinea Toboso gets hit on the head by her mast and knocked unconscious. She wakes up in a hut where she is cared for by a tribe of people who share a remarkable characteristic; every person is lean and toned, even though they eat massive meals and do absolutely no exercise. They tell her the secret is the bark of a rare tree that only grows in the misty cloud forests that hide the interior of the island. The bark smells like elephant feces and somehow, tastes even worse.


Though it is strictly forbidden, Dr. Toboso leaves with several

kilograms of bark hidden in her bathing suit. She

flies to San Francisco and heads to her laboratory to isolate the active ingredient, which she plans to market as "Bark-a-lounge", a dietary supplement designed to cause fat loss and muscle growth without any need for exercise. As a conscientious scientist, she decides to do a research study to show how well it works. She writes the study design on her prescription pad and orders her long-suffering assistant to do the following study:


A group of 12 men she knows from her gym will participate in the study. They will weigh themselves at home and then come to the laboratory so their body fat can be measured using skin fold calipers. Then they will do as many pushup and situps as they possibly can. They will be given 30 doses of "Bark-a-Lounge" in pill form and told to take 2 per day for about 15 days. Then, they will re-weigh themselves, come back to the lab tohave body fat re-measured and do as many pushups and situps as possible. Dr. Toboso is sure that the men will lose fat but gain strength after taking "Bark-a-Lounge" for 15 days.


ObjectiveObjective

Although we have to give Dr. Toboso credit for even considering actually subjecting her product to scientific testing, many of you recognize that her study design is not optimal. The overall goal of this lecture is to allow you to recognize the strengths and the flaws in published studies and media reports. If you plan to conduct your own studies, this lecture will aid you in designing them in a way that maximizes their contribution to the body of scientific knowledge that is used to enhance the performance of athletes and the health of the general public.


Plan of attackPlan of attack

Part 1: “True Lies” What kind of study? Epidemiology vs. experiment;cross sectional vs. longitudinal, association and causality, validity and reliability

Part 2: “Of Mice and (Wo)Men”: Humans, animals or cells? Controlling confounding variables vs. real world application.


More plan of attack

Part 3: “Sub-divide and conquer”How do you attack big important questions? One big study or many small ones?

Part 4: “The Color of Money” Can the funding source affect the studydesign? The results?

Part 5: “You can’t always get what you want”All studies have flaws. Why continue to do them?


Some useful termsSome useful terms

Subjects: participants in a study (usually only used when participants are human)

Variable: Something that can be measured. Independent variables are controlled by the investigator (research scientist). Dependent variables are not.

Treatment: What subjects are “exposed” to. Also called exposure or condition.


Outcomes: The dependent variables. Theanswers to the question you are interested in.

Control group or condition: What the treatment or exposure is compared with. Can be the initial state (baseline) or can be a group that is either given no treatment or a non-functional placebo.

Relative to starting weight (baseline), what is effect on body weight (outcome) when I give 100 people (subjects) three pints of ice cream per day for 6 months (treatment) as compared with 100 people who get no ice cream (control group)?


Epidemiological StudiesEpidemiological Studies

One or more characteristics of a population (e.g. weight or blood lipids or dietary habits) are assessed (usually by using questionnaires but other techniques used as well). Subjects are not asked to change behavior or subjected to treatments like exercise or diet change.

Researchers do not control the experimental conditions; they are trying to understand behavior or physiology or metabolism in a “natural” setting.


Cross Sectional

The variables of interest are measured once. E.g., survey 600 subjects (300 W and 300 M) and measure height. Exposure is gender and the outcome is height.

Mean (average) height for men = 175 cmMean height for women = 165 cm

Based on your data, you might conclude that men are taller than women.


Note that EVERY man was not taller than EVERY woman. There is a lot of variation in human height (let’s say men in your sample ranged from 155-195 cm and women from 148-185 cm).

But the average or mean height for men (175 cm) is greater than the mean height for women (165 cm).

148 165 175 195


Because there is so much variation in height within each gender (about 30 cm in your sample) compared to the mean DIFFERENCE in height (only 10 cm), you need to study a lot of subjects to see a difference between men and women that accurately represents the population.


Although very useful to illustrate a relationship between exposures and outcomes, a problem with observational studies is that you often can’t determine if the exposure caused the outcome.

Let’s say you are interested in whether doing a lot of aerobic exercise lowers the risk for getting cancer;in particular, skin cancer. You send out surveys to hundreds of people asking about their exercise habits and whether they had skin cancer. This is a case-control study; it compares people who got a disease (“cases”) with those who didn’t (“controls”).


Retrospective studiesRetrospective studies

You could do this study “retrospectively”, that is, you could look through medical records, find cases of skin cancer, and mail surveys to the people you identified asking them about their exercise habits.

The downside to this approach is that you depend on people’s memory of their past habits. You might minimize this problem by having people mail you their training diaries but many will be non-existent or incomplete and you have no way to determine whether or not they are accurate.


Prospective studiesProspective studies

You can also do this study prospectively. You start with a group of individuals who DON’T have the disease and track them for some period of time. Then, you look for differences between people who got the disease vs. those who didn’t.

You might randomly contact 5000 people from the phone book and assess their exercise habits every year. At the end of 5 years you would see who got skin cancer and if there was a relationship between time spent exercising and a diagnosis of skin cancer.


The advantage of a prospective design is that the subjects are followed “longitudinally”, that is; over time; rather than cross-sectionally; which only gives a single “snapshot” at one time point.

But to get meaningful comparisons you need to have a fairly large number of people who get the disease so that you can separate them into groups that differ by exercise habits. And some of the subjects will move away or lose interest over time. So to get accurate results often requires recruiting and tracking thousands of people for multiple years.


Questions and answersQuestions and answers

Lets say that your results show that people who run and cycle and swim > 20 hours/week have higher rates of skin cancer than people who don’t exercise at all. Can you conclude that triathlon training causes skin cancer? Alert the media!

Most triathletes spend an enormous amount of time outdoors with a lot of skin exposure to the sun. So is it exercise that causes more skin cancer or is it more exposure to UV radiation from the sun. Unless you collected data on sun exposure in your survey, you would have no way to know


Isolating the outcome of interest

With enough subjects and enough information there are statistical methods to “separate” the key variables. E.g., if you had good data on both exercise habits and sun exposure you would see that if you “remove” or factor out the sun exposure variable, there is no longer any association between exercise habits and skin cancer. So it is sun exposure, not exercise, that increases the risk for skin cancer.


Take another example. Let’s say you want to test the hypothesis that a high intake of fat increases the risk for heart disease. You would need to:

1. accurately identify the men and women in the population who get heart disease

2. accurately assess how much fat is in the diet of each person

3. compare dietary fat in people who get heart disease with dietary fat in people who don’t


This graph (I made it up) says that the number of people who get heart disease increases as the amount of fat in the diet increases.

What are potential problems with this story? Well, did we measure what we thought we were measuring?

% o

f p

eop

le w

ho

g

et

he

art

dis

eas

e

0

20

40

60

80

10 30 50 70 dietary fat as a % of total kilocalories


ValidityValidityValidity refers to the accuracy or truthfulness of a measurement. In other words, are you actually measuring what you think you are measuring?

This can be obvious (using a body weight scale to measure body fat), less obvious (are lower blood lipids after starting exercise training due to training or accompanying weight loss?) or very subtle (do athletes perform better when given carbohydrate during exercise because the sugar does something directly or because they think they should do better when given carbohydrate?)


Measuring physical activity

Activity monitors are a good example of how difficult it can be to develop tools that yield valid measurements of physical activity. There are many types of activity monitors available; pedometers, accelerometers, etc.

If you are a scientist interested in accurately measuring daily physical activity how valid are these tools?


For example, you decide that collecting physical activity information using questionnaires is too subjective and prone to bias so you decide to measure it objectively using an activity monitor that is worn on the hip and is sensitive to motion. You give the accelerometers to 20 people and measure their activity for 7 days to assess their physical activity. 10 of your subjects are world class cyclists and 10 are typical college students. After 7 days your measurements indicate the college students are more active than the elite cyclists! How can this be?


Since the activity monitor only measures movement in the vertical plane, the 600 miles each of your cyclists covered during the week on their bicycles was not detected as movement by the monitor.

This is an extreme case but researchers are constantly forced to consider “am Ireally measuring what I need to measure?”.


What do your subjects eat?

One of the most common measurements attempted in Sport Nutrition is diet analysis. It seems straightforward; you collect information from subjects about what they eat over the course of a few days and enter the foods into a database which spits out grams of carbohydrate and protein and thiamine and iron and vitamin C, etc.

In reality, the measurement is fraught with potential inaccuracy.


Sources of potential error

How do you account for portion size? Estimate based on showing the subjects plastic food models before you start the study? Have them weigh their food? Better but they have to carry their scales everywhere with them. What about combination foods? How do they tell you ingredients and portion sizes of the seafood paella they had at their best friends wedding? And how do you know they are remembering to report everything they ate?


And the process of having to weigh their food and write everything down changes their typical behavior. People avoid foods that are difficult to record accurately and start choosing easy things like prepackaged foods that are conveniently labeled.

Diet records are often inaccurate even in the hands of experienced users. Many subjects under-report their actual food intake by hundreds of kilojoules/day. In contrast , women with eating disorders may OVER-report actual food intake.


Internal ValidityInternal Validity

Chance: what is the chance that the outcome you observe could occur even with NO association between the exposure and outcome you measure?

Measured statistically and reported as a “p-value” showing probability of obtaining the result by chance. Commonly define p-value <.05 (5%) as “statistically significant”. This means there is a 95% chance that the observed effect is NOT due to chance alone.

Is this good enough? Is it too restrictive?

http://images.google.com/imgres?imgurl=www.gserc.usm.edu/images/poker.jpg&imgrefurl=http://www.gserc.usm.edu/gaming.htm&h=174&w=194&prev=/images%3Fq%3Dpoker%2Bflush%26svnum%3D10%26hl%3Den%26lr%3D%26ie%3DUTF-8%26oe%3DUTF-8


What are the consequences of getting it wrong?

Willing to accept an error rate higher than 5% if the consequence is getting the wrong sandwich.

Not willing to accept error rate greater than 0.1% ifconsequence is landing on jagged rocks.

Every reader will have to use their own judgment regarding their comfort level with a given probability that the results are due to chance. Most journal editors have a comfort level right at 5%.


Bias – a systematic error that misrepresents the association between the treatment and outcome. Investigators may design the study in a way that makes it more likely to get a particular outcome.

Or, in conducting the study, they may treat the subjects in one group differently than in the other group (e.g. more encouragement during a maximal exercise test with the treatment than the placebo)

Subjects can bias a study as well. Food intake is often not accurately reported; e.g. faulty memory or wanting to supply the “right” answer.


ReliabilityReliability

Reliability refers to the reproducibility of a measurement. Measurement tools (surveys, activity monitors, etc) are often tested extensively before being used in studies to determine if the values they report are reproducible. Reliability is the main reason researchers often need to make multiple measurements over several days .


ReliabilityReliability

Reliable but not validNeither Reliable AND Valid

xxxxx

xxxxx

x

x

x

x

x

x

x

It is important to be clear on the distinction between validity and reliability. A measurement can be reliable but not valid; i.e., it measures incorrectly every time. Investigators require results to be both reliable and valid.


Reliability influences # of measurementsReliability influences # of measurements

Some measurements, e.g. maximal oxygen consumption (VO2max) are very reliable. You can measure VO2max on different days, different times of day, before or after a snack, and the results will almost always be within a few % of each other.

On the other hand, resting metabolic rate varies day to day and is very sensitive to time of day, food intake, exercise, room temperature, etc. Need very controlled conditions and have to repeat measurements at least 3 times


Back to the made-up graph which indicates that the number of people who get heart disease increases as the amount of fat in the diet increases.

What are other potential problems with this story? Did account for all the other confounding variables?

% o

f p

eop

le w

ho

g

et

he

art

dis

eas

e

0

20

40

60

80



A confounding variable is associated with both the exposure and the outcome and that affects the association between the exposure and outcome.

more exercise hours per week

more skin cancer

more sun exposure

The relationship between exercise and skin cancer is confounded by strong relationships between exercise and sun exposure and between sun exposure and skin cancer.

Trying to minimize confounding variables is the most difficult and time-consuming part of study design


Can we accurately measure the rate of heart disease (probably) and the amount of fat in the diet (much more problematic)?

Do other factors need to be considered?

* gender (true for men AND women?), * age (maybe elderly people eat more fat)* ethnicity (directly or indirectly)* other “risky behavior” (smoking, lack of exercise, less frequent physicals, etc.) in people who eat

more fat in diet?


Can you consider all the other factors?

Clearly not b/c we don’t even know what they all are (e.g. there is a lot of recent evidence that the conditions a fetus encounters in utero can have an impact on adult-onset disease).

Even if you could, does a positive relationship between 2 things (as 1 goes up, the other also goes up) prove that one causes the other?


During this time period (2005), there was strong association between the distance from Earth to Saturn and the price of gasoline. Did gasoline prices rise because Earth was getting farther from Saturn?

The relationship is a coincidence: Association does not mean causality

pri

ce o

f g

as

olin

e

distance from the Earth to Saturn


So, epidemiological studies are difficult to design in a way that gives you clear, definitive answers.

To get a sharper picture of the causal relationships between diet and health or performance you can do an experimental study.

Take a group of healthy people, feed them different amounts of fat, and see who gets heart disease?


Experimental StudiesExperimental Studies

The key difference from an observational study is that the investigator actively manipulates the treatment instead of letting things happen by chance. Because the experimental conditions are controlled, there is a much greater chance that the outcomes are directly related to the treatment.

A disadvantage is that by manipulating the conditions, the results may have less direct relevance to what happens in the “real-world”


Experimental StudiesExperimental Studies

In experimental research, study subjects (whether human or animal) are selected according to relevant characteristics and then assigned to either an experimental group or a control group. The subjects in the experimental group receive treatment and the control group receives no treatment or a placebo. If you do this correctly, you can assume that differences between the groups at the end of the study were caused by the treatment.


Experimental: Cross SectionalExperimental: Cross SectionalExperimental studies can be cross-sectional (multiple groups getting a single treatment) or cross-over (one group getting multiple treatments including control). In a cross-sectional design, subjects are randomly assigned to either a treatment or a control group. They are exposed to the treatment or control for a period of time and then the outcome is compared between the two groups. Let’s say you wanted to test whether consuming only simple sugars for 28 days would cause more synthesis of muscle glycogen compared with a “normal” diet.


Your cross-sectional design might look something like this:

Baseline test of muscle glycogen synthesis

28days

Group 1

Group 2

Groups randomly assigned

Re-test of muscle glycogen synthesis


Assigning subjects to groupsAssigning subjects to groups

One of the keys to doing this right is to ensure that the 2 groups of subjects are as similar as possible. To do this, subjects are usually randomly assigned to the placebo or control group.

An alternative is to match subjects in each group on some key characteristics (e.g. age, weight, training status, aerobic capacity). This helps to distribute any characteristics that might influence the results across the groups.


An example of why randomization is important can be seen in the following example:

Researchers want to determine if a high fat diet during marathon training can improve performance. They do a baseline (before any treatment) test of aerobic fitness to all of the potential subjects. Then they assign them to different groups; 20 to the high-fat diet group and 20 to the high-carbohydrate diet group. Then they train them using the different diets for 12 weeks.


At the end of that time, they redo the test of aerobic fitness and find that the high-fat group has improved considerably more (increased VO2max from 45 to 52 ml/kg/min) than the high-carbohydrate group (only increased from 68 to 70 ml/kg/min). They report in all of the media outlets that runners can gain twice the training effect by using a high-fat diet. Is this reasonable?


Notice that the baseline VO2max was considerably higher in the high-fat group. Runners were clearly not randomly assigned; the high-carbohydrate group seems to have contained really fit elite runners (whose VO2max is already about as high as it can be) and the high-fat group look like mainly novice runners (who can improve a lot with training).

If the groups had been randomly assigned, the baseline VO2max would have been similar in the 2 groups. In that case, a larger improvement in the high-fat group could be interpreted as due to the diet (assuming everything else had been done right!)


BlindingBlinding

Randomization is often blinded to limit experimental bias (an interest in having a particular result). Blinding is used to prevent bias from influencing the behavior of both the investigators and the subjects. There are two types of blinding, single blind and double blind. In a single blinded study the investigators know which treatment the subjects are getting but the participants do not. In a double blinded study, a neutral third party assigns the groups and neither the investigators nor the participants are aware of the group assignments.


A drawback of cross-sectional study design is that no matter how well you “match” the 2 groups on important characteristics like age, height, weight, fitness, etc., there is no way to do this perfectly.

Two groups may be similar but they can’t be identical, meaning “inter-individual variability” (genetic and other differences between people) will be a limitation to showing clear differences between the treatment and the control groups.

Wouldn’t it be great if you could clone each subject and use their clone in the other group?


Experimental: Cross OverExperimental: Cross OverIn a cross over design, subjects serve as their own controls. Half of the subjects get the treatment and the other half get placebo. Then the same subjects undergo the opposite protocol.

Baseline test of muscle glycogen synthesis

28days

½ of group

order of treatment randomly assigned

1 month washout

Re-test of muscle glycogen synthesis

Final test of muscle glycogen synthesis

28days

½ of group


Washout periodWashout period

A potential problem with the cross-over design is that effects of the first condition (e.g. treatment) may have an impact on the response to the second treatment (e.g. control). The solution is to put a “washout” period between the 2 conditions to allow the effects of the first condition to disappear.

This washout period may be long (months for some interventions like training or lipid-soluble anabolic agents). This makes the study very lengthy and it can be difficult to keep subjects in the study.


External ValidityExternal Validity

Also referred to as generalizability; meaning how applicable are the results to the general population. To increase the external validity, investigators can study subjects varying in gender, race, ethnicity, age, weight, etc. By doing this, it is more likely that results can be applied to the general population.


OvergeneralizingOvergeneralizing

Many classic studies in nutrition (for example; the response to semi-starvation and re-feeding; human protein requirements) were performed almost solely using Caucasian, male, healthy subjects in their 20’s and 30’s.

Nutritional requirements were generalized from those studies to the entire population, despite few data on women, children, ethnic/racial minorities or people with underlying health problems


Trade-offsTrade-offs

All major funding agencies now mandate inclusion of women and minorities or require a strong justification for not doing that.

Why not include as many types of subjects as possible in order to maximize the external validity?

Increasing external validity also means increasing the number of potential confounding variables. In some studies, it is more prudent to use a specific population to minimize confounding variables


““Basic” research studiesBasic” research studies

Experiments under highly controlled conditions are often necessary to confirm observations or uncover how a process works (the mechanism of action). They may be conducted in vitro (e.g. with cell populations on culture plates) or with animals.

These studies allow the investigator to isolate one variable of interest without confounding variables such as environmental factors, genetic variation, and differences in dietary or physical activity patterns.


One of the advantages of doing studies using cells or animals is that tissues not available in humans can be isolated (e.g. whole muscle, liver, heart, etc.) and life spans are much shorter. For example, if we were to do our study of high fat diets and heart disease in mice instead of humans, the study would take a couple of years instead of decades.

And researchers could sacrifice the mice at the end of the study and look directly at the effects of the diets on their arteries, muscle, liver, etc.


Due to differences in physiology and the fact that animals are routinely exposed to levels of compounds far higher than those humans typically encounter, results from studies with animals are not directly generalizable to humans.

In addition, there are moral issues regarding animal experimentation that can’t be ignored. Some people feel strongly that no experimentation on animals is ever justified. Some people have no problem at all with scientific experimentation on animals.


The great majority of individuals, both within and outside the scientific community see this as a complicated issue. There are benefits to animal research (potentially lifesaving cures for human disease; many dogs were sacrificed in the hands of Banting and Best before they were able to isolate and purify the insulin that has saved the lives of millions of people with diabetes).

And certainly costs (nobody enjoys the idea of submitting creatures to experimental procedures that often end with their death).


And the type of animal is certainly a factor in people’s discomfort with animal research: few people object to research on flies, a few more to fish or frogs, many become uncomfortable with experiments on mice, rats, and rabbits, and even more people feel strongly about research on cats, dogs and primates.

To balance these competing forces, universities and other research organizations follow strict guidelines to help ensure that research on animals is conducted in the most humane possible way


Researchers are required to justify why the research is essential (disease yes, performance no), to use statistical analysis to minimize the number of animals they intend to study ,and to maximize the comfort and well-being of the animals in their care.

As new experimental and mathematical modeling techniques are developed, the justification for doing research on animals is expected to diminish in the near future.


Human ExperimentationHuman Experimentation

The moral issues of experimentation extends to humans as well. Before organizations began to regulate the conduct of experiments on humans, experiments were sometimes done without subjects consent and with little regard for their health or well-being.

Human research in most countriesis regulated to ensure that subjects can truly give informed consent to the procedures and that potential benefits outweigh risks


Potential subjects have to be recruited in ways that are not coercive and they must be in a position to refuse to participate or to leave the study partway though without adverse consequence (so no prisoners, people who are institutionalized, children unless with parental consent).

What about students in a class being taught by the researcher? Grad students in the lab? The researcher has to convince an institutional review board that participation or non-participation will have absolutely no consequences with respect to their grade in the class or graduation, etc.


Review boards weigh the potential benefits from the research with the stress, physical and mental discomfort, time commitment, etc. that the subjects are required to undergo.

During the study itself, procedures must be in place to ensure that health and well-being of the subjects are a higher priority than the data.

Subjects are often compensated financially for their participation; it is important that compensation be sufficient but not excessive (i.e. coercive)


Because the priority to maximize health and well-being of the subjects and to ensure they are not coerced into continuing participation in a study can conflict with the need to collect vital research data, doing human studies in a way that both prioritizes subject well-being AND maintains maximal scientific rigor is very difficult.


So let’s return to this made-up association. Could you do an experimental study in which you recruit subjects without heart disease, feed them several different amounts of dietary fat and look at the relationship between dietary fat and the rate of heart disease over time?

% o

f p

eop

le w

ho

g

et

he

art

dis

eas

e

0

20

40

60

80



Yes. But would require studying hundreds of people for decades, providing all of their meals and controlling dozens of other things that affect risk for heart disease (like smoking and exercise and aspirin use and on and on).

This would cost tens of millions of dollars, take several decades and would still be almost impossible to do b/c most volunteers would leave the study


So, how do you design a study that can answer an important question and that is doable in a reasonable time frame and for a reasonable amount of $$?

You have to take a big important question and reduce it to a much smaller, more focused question. You have to do a series of small studies, each one building on the one before, until you accumulate enough evidence to support or disprove your idea.



Matching subjects on key characteristicsMatching subjects on key characteristics

To compare whether men responded to cookie cream similarly to women, you plan to do a 2nd group composed of men. What kind of men?

Well, you can recruit men matched to the characteristics of the women. How about VO2max? OK, but a woman with VO2max of 60 ml/kg/min is often a lean athlete in hard training whereas elite male athletes have a higher VO2max. So men and women matched on VO2max will usually differ on body fat and training


How about body fat? OK, but an average body fat for a man, let’s say 15% would be very low for a woman, and again you are likely to end up with trained female athletes with very high VO2max and moderately trained, recreationally active men.

There is actually almost no way to match men and women for both aerobic capacity AND body fat.

Need to choose the one that is MOST critical. Or another characteristic that CAN be matched (e.g. training status)


Recruiting subjects for studies requires balancing many competing factors. A more diverse subject pool gives you more generalizability but also more confounding variables, increasing the number of subjects required.

A more homogeneous group (e.g. highly trained, college-age women in the luteal phase) reduces the confounding variables and allows you to do the study with fewer subjects but also makes the results less generalizable


The trade-offs illustrate why studies at all levels of generalizability are required to answer important questions. Epidemiological studies using large, heterogeneous sample sizes can point to interesting associations that are worth pursuing (e.g. more physical activity is associated with lower rates of diabetes).

Basic scientists can look at potential mechanism (e.g. isolated rat muscle electrically stimulated to contract takes up more sugar than resting muscle)


In between are human experimental trials ranging from simulating the rat study (putting in arterial and venous catheters in the leg of a volunteer to see if exercised human muscle also takes up more sugar for the blood) to testing different intensities and durations of physical activity on groups of free-living people to determine which combination has the biggest impact to reduce the risk for diabetes.

The best study designs build on the results that have come before and add another key piece to the jigsaw puzzle.


New York Times, 5-27-01New York Times, 5-27-01

“Coke formed a partnership with Procter & Gamble earlier this spring.

The companies are now preparing to introduce a drink called Elations. Each bottle of Elations contains 1500 milligrams of glucosamine, a dietary supplement that has been popular among people with arthritis for years.”

Procter officials insist that sound science is what distinguishes Elations from the many herbal concoctions currently transforming the market.”


New York Times, 5-27-01 cont’d

“The National Institutes of Health is conducting a comprehensive 4-year study on glucosamine. But neither Coke nor Procter felt they could afford to wait for the results. ““The game will be over if anybody isn't in it by then," said the assistant director of Procter's Nutrition Science Institute””.

Can research be done in a way that balances needs of the scientific community (a line of research studies that tell the whole story) with needs of industry (no research or a single study showing the product works)?


For industry, For industry, enoughenough research is .... research is ....sufficient to convince enough consumers to buy the product that $$ from sales exceeds the costs of manufacture, distribution and advertising.

To do more violates the interests of employees, shareholders and, in terms of price, consumers.

Doing more than the minimum research needed to maximize sales is not only unnecessary but even incompatible with interests of the company.


For academic scientists, For academic scientists, enoughenough research is …..research is …..

First: efficacy (does it work?) safety (does it harm?) but also:

Research scientists are charged with understanding context: mechanism of action, effects on other metabolic pathways etc.

Doing less than the minimum research required to understand the physiological context is incompatible with responsibilities as scientists.


Is there a way to meet halfway?

YES, in the sense that both groups share the same basic goals of optimizing safety, health, and performance

NO, in the sense that there are fundamental disagreements about who (target population), what (top priorities), why (knowledge/sales), when (how soon), and how (single study vs. line of research) research is done


A group of 12 men she knows from her gym will participate in the study. They will weigh themselves at home and then come to the laboratory so their body fat can be measured using skin fold calipers. Then they will do as many pushup and situps as they possibly can. They will be given 30 doses of "Bark-a-Lounge" in pill form and told to take 2 every day for about 15 days. After 15 days they will re-weigh themselves and come back to the laboratory to have body fat re-measured and do as many pushups and situps as possible. Dr. Toboso is sure that the men will lose fat but gain strength after taking "Bark-a-Lounge" for 15 days.

A fairy tale revisitedA fairy tale revisited


Doing studies correctly is hard. Why keep doing them?

“It’s supposed to be hard. That’s what makes it great. If it was easy, anybody could do it.”

Barry Braun, Ph.D. Basics of Study Design The Basics of Study Design Barry Braun, PhD, FACSM Associate Professor Director, Energy Metabolism Laboratory.

Documents

following study

kind of study

research study

oh h oh h slide

kilograms of bark

bark smells

doses of bark

body fat remeasured