Making Observations: Monitoring Demand Characteristics

Monitoring Demand Characteristics

External validity also depends on another consideration: We need to make sure that our study itself doesn’t affect the behaviors we hope to examine. One concern here is that people’s behavior sometimes changes when they know they’re being observed. They may be trying to present themselves in the best possible light, or maybe they’re just feel-ing self-conscious about being observed and act differently as a result.

A study’s demand characteristics can also alter people’s behavior. This term refers to any cues in a situation that make participants think one response is more desir-able than another. Sometimes the demand characteristics are communicated in the way questions are phrased (“You do brush your teeth every morning, don’t you?”). Sometimes they’re conveyed more subtly—perhaps without realizing it, the investi-gator smiles and is more encouraging when the participants answer in one way rather than another.

In some studies, demand characteristics influence an entire data pattern. For exam-ple, how often do people drive their cars through red lights? To find out, we might col-lect responses on a questionnaire that asks people directly about their driving habits. (“In the last month, how often have you . . .”) But the results from this study would surely underestimate the frequency of this traffic violation: we’re asking about an ille-gal activity, so people are likely to give us a less candid but more “desirable” response (“I never run red lights.”).

In other cases, demand characteristics can create artificial differences between the groups being compared. Imagine that an investigator is comparing the problem-solv-ing skills of American and Japanese students, but he’s warmer and more encouraging to the Japanese participants than to the Americans. In this situation, any differences we observe between the groups might be due to the students’ country of origin, or they might be due to this differential treatment. Because of this ambiguity, we could draw no conclusions from these data.

Investigators take several steps to minimize a study’s demand characteristics. First, they try to phrase questions and instructions as neutrally as possible, so that partici-pants can’t identify any response as preferred or “better.” If questions are likely to be embarrassing or provoke anxiety, the investigators do everything possible to diminish these concerns and guarantee that all information gained from the participant will remain confidential. Investigators are also careful to treat all study participants alike. In many studies this equal treatment is guaranteed by the use of a double-blind design, in which neither the person who actually collects the data nor the study participants themselves know the nature of the hypotheses at stake—or, in some studies, the nature of the groups being compared (Figure 1.8). This approach ensures that the investigators treat the various groups of participants alike and that participants in the various groups have similar expectations about the procedure.