Motivations for Content Selection: An Application of Open-ended Questions in an On-Line Environment - The Web Journal of Mass Communication Research

Salma I. Ghanem and Dixie Shipp Evatt

[WJMCR 5:4 September 2002]

Sections:

Abstract

For all of their sophistication and speed, polls often do little to illuminate the reasons behind the opinion. Why and how do individuals come to care about some issues over others? Why are some issues salient but others are not? The study reported here uses open-ended questions to try to get at the underlying methodological problem and examines what individuals say motivates them to either select or to ignore specific content.

Results lend credence to the idea that people are unlikely to select stories to read if they do not relate to the story. People are more likely to select a story to read if they either identify with it on some personal level or if they feel that the issue is of some social consequence.

Introduction

Since the advent of public opinion polling in the United States some 65 years ago, the science of opinion measuring has become quite sophisticated.1 Today, we can watch a Presidential debate and within minutes get a “snap” poll telling us how the candidates were perceived. The voting booth no sooner closes than the results of exit polls allow commentators to call a race. Politicians get daily “tracking” polls on issues on which issues and proposals are resonating with voters.

Yet, for all of their sophistication and speed, polls often do little to illuminate the reasons behind the opinion. Why and how do individuals come to care about some issues over others? Why are some issues salient but others are not? McCombs tackled this by posing a question set as a follow-up to the MIP (most important problem) in public opinion polls in Austin, Texas. His findings, based exclusively on close-ended responses, suggest that self-interest and personal feelings, what he calls motivations, are keys to understanding salience of issues.2 While his was a valuable first step, the study reported here takes a different approach by using open-end questions to try to get at the same underlying methodological problem: Is it possible to conceptualize salience or motivations in a way that can illuminate public opinion measurement?

Related Studies

Much of the literature on public opinion research accepts, apparently with little challenge, the notion that opinions as expressed by the public through polling are superficial, “top of the head” and volatile. Indeed, the idea that the initial response of human beings to an issue, person or idea is largely impressionistic is an essential ingredient in some of the earliest discourse about public opinion in the United States. For instance, James Bryce, the 19th century political theorist, believed an individual’s initial spur-of-the-moment reaction to a public issue is not based on conscious reasoning. He said it is only after the initial affective arousal that the individual seeks additional information and begins to formulate a definite view on the issue.3

In his book on mass opinion, Zaller says citizens do not carry around fixed attitudes on issues. Rather they construct “opinion statements” on the fly based on ideas that are for one reason or another most immediately salient to them.4 This suggests that when confronted by a public opinion poll question, people are more likely to give a spontaneous reply than a reasoned response.

The fact that the opinion is spontaneous does not mean that it lacks validity. It means that some force or motivation, either personal or social, brought that issue to mind and the individual was comfortable enough with the thought to use it in answering the question. In other words, salience can be as simple as what catches the eye or as complex as an individual’s own needs or interests. This is the same concept that Taylor and Fiske used when they coined the term “top of the head” phenomenon saying that the causal attributions people make, the opinions they express and the impressions they form of others are often shaped by seemingly trivial but highly salient information. They say, “attention within the social environment is selective. It is drawn to particular features of the environment either as a function of qualities intrinsic to those features (such as light or movement) or as a function of the perceiver’s own dispositions and temporary need states.”5

While there is agreement that salience influences opinion, the mechanics of the process are less well understood. Taylor and Fiske’s observations about salience suggest two sources of salience: self and environment, particularly active or highly visible elements of the environment. As applied to mass communication research, the second half of this equation seems to fit well with the agenda-setting hypothesis that says simply shining the light of media attention on an issue and tracking it through stages of activity will increase its salience.

Agenda setting theory has long linked the media and the public agendas. Over the last 30 years or so, researchers have replicated the studies, explored the contingent conditions, investigated the intervening variables, linked the theory to other theories, examined the theory from both the aggregate and individual levels and some at times have brutally criticized the theory.6 In addition, a group of scholars are now examining the second level of agenda setting and are linking agenda setting to more recent theoretical developments in the field such as framing and priming.7 In most studies on agenda setting, media exposure was either assumed or just measured. Media exposure, a necessary antecedent for any type of media effects was seldom investigated in agenda setting. Agenda setting deals with the transfer of salience from the media to the public agenda. In the first level of agenda setting, the focus is on object salience while the emphasis in the second level is on attribute salience.8 In both levels, exposure is a necessary condition.

The study reported here examines the motivations behind content exposure decisions by respondents. This study also advances a methodology in this field by using open-end questions to define the motivations behind content exposure decisions by respondents.

Some research on exposure and on the motivations behind media exposure was conducted in the early days of agenda setting. Weaver for example found that there is a positive relationship between need for orientation and media exposure. The two factors involved in the need for orientation are (a) relevance or interest and (b) uncertainty regarding a particular issue. Relevance or interest were measured by a series of closed-ended questions which measured primarily civic duty or political interest.9 Just recently, personal relevance was added in the exploration of the need for orientation.10

Wanta also discusses differences behind people’s interest in a particular topic and provides the reader with two scenarios of people interested in the same topic but with divergent motivations. In one case, the interest aroused from personal involvement while the other was based on concern with societal problems which is similar to the sense of civic duty explored by McCombs.11 According to McCombs the strong response to civic duty observed in his study was expected because of its assumed strong social desirability bias. Researchers are well aware that although, in theory, every citizen should have a strong sense of civic duty and a feeling of obligation to keep up with current affairs, many do not. There are strong individual differences in the civic duty of keeping informed.12 A sense of obligation to stay informed is found to correlate with newspaper reading and viewing national television news.13

McCombs examined five motivations in the formation of the public agenda: self-interest, civic duty, avocation, emotion and peer influence. The five motives were measured by seven follow-up items to the most important problem (MIP) question.14 The motives emphasized by the respondents were civic duty, emotion and avocation while self-interest and peer influence were rather weak.

His study, however, is based exclusively on close-ended questions. Previous survey research has shown that the public demonstrates a broader scope of knowledge when allowed to respond to open-ended questions that do not presuppose a “right” or a “wrong” answer.15 Open-ended questions allow respondents to offer information about issues and ideas that are most salient to them as opposed to being bound by pre-selected categories. This method gives individuals the opportunity to talk about issues in language that makes sense to them. After a series of experiments, Geer concluded that open-ended questions of this type, on balance, measure important concerns of respondents. He says such questions do not necessarily result in expression of superficial concerns. Nor, he says, are answers overly influenced by information the respondents recently learned.16

Method and Research Questions

This study examines what individuals say motivates them to either select or to ignore specific content. It also relies on analysis of open-ended responses. Analyzing open-ended responses or the “construction procedure” is more sensitive to situational and individual differences and reduces the desirability effect often associated with the “selection procedure.” The selection procedure is basically close-ended responses where respondents have to select from given categories of responses.17 This study is a content analysis of content data drawn from a series of media effects experiments conducted between 1997 and 2000. The results of the individual experiments were reported elsewhere.18 This ongoing line of research has resulted in a database that contains 1,125 cases of which 707 contain responses to open-end questions that seek to understand underlying motivations for selecting a story.

In each of the experiments, subjects first read a news story. They were then asked: “Emerging technology may someday make it possible for each individual to, in effect, create his or her own daily newspaper by selecting stories from a range of topics on-line. If you had such technology available to you, how likely are you to have chosen the news article you just read?” [Very Likely (4); Likely (3); Unlikely (2); Very Unlikely (1)]. This formulation removed any choice of medium from the respondents and, instead examined only their likelihood of attending a specific story.

Subjects were then asked to explain in their own words the reason behind their likelihood for selecting the story. Researchers then transcribed responses from the handwritten questionnaires into a file that identified the respondent by case file number only. In this way, analysis relied only on the words and characterizations provided by respondents since no other part of the questionnaire was visible to the researchers who were asked to code the data. Unlike the McCombs research where categories were assigned and subjects asked to respond to each one separately, the categories for this study were developed from an inductive process of text analysis of the naturalistic statements themselves. Categories and coding instructions were refined in several stages to assure that they were mutually exclusive and exhaustive. The categories developed from this process were as follows:

1. Empathizer:
A response that typifies this category would express a sense of concern or interest about the topic because of sympathy or a shared sense of sadness or joy for what was happening to the people in the story.

2. Social Generalist:
Responses that typify this category would focus on consequences to society or nations as a whole rather than on individual classes or persons. Responses in this category will usually not mention solutions, but rather describe problems and/or their causes.

3. Virtuous Savior:
Responses that typify this category would demonstrate a strong interest in correcting or rectifying social ills or problems experienced by individuals, societies or nations. Very often, the response would suggest solutions to problems rather than manifestations of problems.

4. Identifier:
A person who typifies this category would express a sense of concern or interest about the topic because of personal relevance to his/her life, avocation or future plans. This person looks at the situation from an individual rather than a social perspective.

5. Ethnocentric or Selfish:
This person will express the lack of interest or concern about the topic because it lacks personal relevance to his/her life, avocation or future plans or to others like him/her or a nation.

6. Stylist
The comments of a person who typifies this category would focus on structural aspects of the story, writing style, headline or presentation rather than the content per se. This category was used only when the statement mentioned the form of the message as opposed to the content of the message.

There are some similarities between the categories used in this study and the categories used by McCombs. The “empathizer” focuses more on feelings and echoes the McCombs “emotion” category. Both the “social generalist” and the “virtuous savior” express a sense of “civic duty” or societal concern as given by McCombs. The “identifier” in this study is similar to the way McCombs used “avocation,” and “ethnocentric/selfish is similar to McCombs category of “self interest.”

While there are similarities, there are also differences. One category used by McCombs (peer influence) was not found in this study because it did not emerge from the open-ended data.” Interestingly, peer influence is a category that did not yield significant results in McCombs’s study. In addition, this study added one category not found in McCombs, that of “stylist.”

The stylist was necessary for this study because some responses only dealt with the presentation of the story rather than the information provided in the story.

Coders were told to assign a response to the category that best described the respondent’s motives as reflected in his or her reasons as described in the whole statement. Coders were cautioned to focus on the manifest content of the words used by respondents. For this analysis, they were told that the valence or direction of the comment (whether likely or unlikely to choose a story) was not as important as the way in which the response justified or explained the decision. When more than one element was present, coders were asked to pick the one that in their judgment was dominant.

Two coders coded the same sample of open-ended responses and intercoder reliability was measured at .83 (using Holsti’s method). After coding was complete coded data were merged with the larger set to allow analysis and comparison.

Findings

The first step in analysis was to examine the relationship between likelihood of choosing a story and particular motivations. Although the respondents were offered four response categories, for analysis categories were collapsed into dichotomous categories. The categories very likely and likely were combined and unlikely and very unlikely were combined.

As shown in Table 1, nearly nine out of ten of respondents (88.3 percent) who were not likely to select a story indicated either an ethnocentric/self interestedreason (72.7 percent) or a reason dealing with style (15.6 percent) for their choice. The motivations for selecting a story fell mostly in the other four categories. This breakdown seems to lend credence to the idea that people are unlikely to select stories to read if they do not relate to the story. People are more likely to select a story to read if they either identify with it on some personal level or if they feel that the issue is of some social consequence.

In terms of demographics, as shown in Table 2, males and females exhibit different patterns of motives in their open-ended responses. The percentage of ethnocentric/self interested males is higher than the percentage of females in the same category and women needed to identify more with the story than men (18.5 vs. 10.0 percent) did.

A second demographic variable, age, was also analyzed. As shown in Table 3, younger subjects proportionally tended to exhibit greater ethnocentric/selfinterested motivations than did older subjects.

Since all respondents were full-time students enrolled in institutions of higher education, another variable that produced interesting patterns was the choice of major. For instance, the majority of law students were social generalists (50 percent) and the majority of science and math students were ethnocentric/self interested (54.5 percent). Education students had the highest percentage of identifiers (32 percent) while engineering students had the highest percent of stylists (24 percent).

Discussion

After subjects in this study read a media message, their interest in the issue presented was often expressed in terms of personal relevance. One demographic that produced a difference was gender. Women were, as a group, less likely to focus on self-interest and more likely to project themselves into the situation by identifying with the issue or the people in the news story. This underlying motivation may illuminate the gender gap that frequently is detected in political polls.

Age is another variable that warrants further study. Because most of the subjects were college age students and the data suggest that as individuals mature they develop a stronger social consciousness further research applying this method to older subject pools is warranted.

The other demographic reported here, choice of major, needs further study. Unfortunately, the sample size for some categories of majors is too small to draw strong inference. These data are, however, suggestive. For example, the fact that engineering students had the highest percentage of stylists might suggest an important personality characteristics or predisposition, such as interest in the physical properties of objects, that influences opinion.

One of the advantages of the method used in the study reported here was the development of categories for motivations that were drawn A from the subject’s own words and feelings. As this line of research moves forward, these data will provide a useful validity check against research that relies on closed-ended questions.

These data are suggestive of a pattern of motivations that, when married to other research, can move us toward more sensitive measuring instruments to better understand the varied and interrelated factors (from rigid structures such as culture to volatile states such as mood) that affect opinion. Evatt (1999) likened these influences to a Calder mobile. Not only does such a model offer three spatial dimensions, but it also embodies the idea that movement in one space or factor invariably creates movement in another. In this kind of model, even the small pieces can trigger vibrations in the largest. With the exception of those permanent conditions or states of being, such as gender, the influences are measurable as both dependent and independent variables. This study adds one piece of evidence to the varied motivations within the matrix of variables that must be considered.

The same dynamic that led scholars to conclude that attitudes influence opinions and opinions influence attitudes probably is at work for any combination of two or more of the antecedent variables or motivations that come into play in opinion formation. Whether in the context of commercial marketing, social issues or surveys or opinion polls, scholars look upstream to the same general set of influences to explain effects and predict outcomes. In many ways that is how opinion processes can be visualized: functionally related instead hierarchically ordered. For instance, the suggestion in these data that the power of individual experience and preferences may be primary could be useful in defining related theoretical constructs such as obtrusiveness in agenda-setting theory.18 This field of research deserves greater attention from scholars who seek to measure and understand influences on public opinion.

About the Author:Salma I. Ghanem is associate professor and chair of the Department of Communication at the University of Texas-Pan American. Dixie Shipp Evatt is an assistant professor in the Newhouse School of Public Communications at Syracuse.