Making Observations: Assessing External Validity

Assessing External Validity

Clearly, then, different studies rely on different sorts of data—sometimes a random sample, sometimes a maximum variation sample, sometimes a single case. There is similar diversity in the situations that studies examine. In some studies, we want the situation to be representative of the broader world; for example, we hope our study of aggression on the playground will tell us about aggression in other settings. In other studies, though, there’s no need for representativeness because our hypothesis can be tested only by examining some unusual or even artificial circumstance—one in which we can observe patterns that would otherwise be hidden.

If we do want our study to reflect the broader world, we need to ensure its externalvalidity—that is, to make sure the study does represent the world as it exists outside ofour investigation. To ensure external validity, we obviously need the sample of people in the study to be representative of the broader population. We also need to make sure the circumstances of the study accurately reflect those in the broader world. Imagine, for example, that while studying aggression in children, we observe the children’s behavior when they’re tired and cranky at the end of a long, hard school day. In this setting, we might get a distorted picture of the children because in this situation they’re quite likely to misbehave. As a result, we probably shouldn’t draw conclusions from the study about how children behave in other circumstances (Figure 1.7).

External validity also depends on what’s being investigated. An investigator inter-ested in the human visual system can probably study American college students and draw valid conclusions about how vision works in all humans. This extrapolation from the data would be valid simply because the properties of the visual system are rooted in the biology of our species, so we can generalize widely from a relatively narrow data set. But we’d have to be much more cautious in our generalizations if we studied the same college students in hopes of learning about, say, human romantic fantasies. In that case, our results might tell us little about anyone other than the particular group studied.

Be aware, however, that questions of external validity need to be resolved through research. For example, consider laboratory studies of memory. These studies often involve college students memorizing either lists of words or brief stories. Can we draw conclu-sions from these participants, and this task, about how (for example) eyewitnesses will recall a crime or how medical patients will remember a doctor’s instructions? To find out, we might need to study actual eyewitnesses or patients and see if the principles derived from the laboratory studies apply to these samples as well. This analysis would then help us decide, in future studies, whether we could generalize from laboratory findings.