Making Observations: Defining the Question

Defining the Question

Science begins with observation—but what should we observe? If we are interested in, say, how young children play together (Figure 1.1), it’s just not possible to observe every-thing about their playtime. Everything (if interpreted literally) would include each child’s every blink, every breath she takes, every time she scratches her ear or touches her hair, and much, much more. A lot of this information is unlikely to be of interest to our study, and so we’d be buried under an avalanche of unnecessary details that might make us lose track of the more useful information.

It seems, therefore, that we need to choose which aspects of the children’s behavior we care about and which aspects we can ignore. In making this choice, we’re usually guided by the questions we hope to address through our observations: Is a child’s behav-ior influenced by the size of her playgroup? If so, we need to count how many children are involved in the group. Does the weather shape the child’s style of play? To find out, we’d need to note the weather and somehow describe the “style.”

Moreover, it’s not enough just to have a question. We also need to formulate the question in a way that leaves no doubt about how we’re going to link the question to the evidence we collect. To make this idea concrete, imagine that we wanted to ask: Which sex is more aggressive in its playtime—boys or girls? In tackling this issue, we’d need to be clear from the start about what sorts of actions would count as “aggres-sive.” Otherwise, we might observe children for hours and still have no way to answer our question.

This requirement to be specific is usually met by making sure that any research project begins with a testable hypothesis—a specific claim about the facts, framed in a way that will allow an unambiguous test. Testability, in turn, is guaranteed by ensur-ing that the hypothesis is falsifiable—stated so that we’re clear at the start about what pattern in the evidence could show the hypothesis to be false.

We can see how important these ideas are by considering hypotheses that are not testable. Think about the forecasts that astrologers offer (Figure 1.2)—for example, “There will soon be major changes in your life.” This prediction is open-ended in ways that prevent any linkage to specific observations: What exactly is meant by soon, or major changes? There’s plenty of room for debate about how these terms should be defined, and so there’s room for debate about what sorts of observations would or wouldn’t falsify the forecast. As a result, we have no way to check the forecast against the facts, no way to tell if the claim is right or wrong. For scientific purposes, this situation is unacceptable.

This example also makes it clear that a testable hypothesis requires well-defined terms (Figure 1.3). To see how this plays out, let’s press on with the question about whether the sexes differ in their aggression on the playground. To address this question, we’ll need to develop an operational definition— one that translates (“operationalizes”) the variable we want to assess (aggres-sion) into a specific procedure or measurement. We’ll also need the.

operational definition to have construct validity—that is, it must truly reflect the variable named in our hypothesis. Thus, for example, we wouldn’t operationalize aggression on the basis of how often someone frowns. This wouldn’t be valid, because sometimes peo-ple frown for reasons other than aggression and sometimes they don’t frown when they’re being aggressive. So it would be better to operationalize aggression as a count of actions like kicks, punches, and bites. This would count as an operationalization because these behaviors can be directly observed in the data.

These points about operationalization and validity are concerns about how we define our dependent variable—so called because the investigator usually wants to find out whether changes in this variable depend on some other factor. In our case, we want to know whether a child’s aggression levels depend—at least partly—on his or her sex; so the dependent variable would be our measure of aggression. The independentvariable, in contrast, is the factor whose effects we hope to examine. If we’re askinghow boys and girls differ in their aggression, then we have a single independent variable—sex—that can take either of two values (male or female).

In many studies, the dependent variable is operationalized in terms that can be eas-ily and objectively measured. If, for example, we’re measuring someone’s memory capac-ity, our dependent variable might be the percentage of correct answers on some test. If we’re studying how sensitive someone’s hearing is, we might count the number of faint tones that person can detect. A quality like aggression, though, might require a differ-ent sort of yardstick because deciding whether an act is aggressive can be a subjective matter. (Was that shove playful, or was it a hostile push? Was that kick deliberate, or just an accident?) In dealing with this subjectivity, our best option is to use a panel of judges who assess the study’s participants on the relevant dimension. We could, for example, videotape the children at play and then show the recording to the judges. They would rate each child’s aggression on, say, a 7-point scale where a rating of 1 means “not at all aggressive,” and 7 means “extremely aggressive.”

By using a panel of judges, we can check on the subjectivity of the assessment (Figure 1.4). If a child’s play seems aggressive to one judge but not to another, it confirms the concern that this is a subjective matter. In that case, we can draw no conclusions from the study; we’ll need to find some other way to measure our dependent variable. But if the judges agree to a reasonable extent, then we can be confident that their assessments are neither arbitrary nor idiosyncratic—and this would allow us to use the results of judges’ assessments as the study’s dependent variable.