By Doreen Marchionni, Hans K. Meyer and Esther Thorson
WJMCR 46 (February 2014)
Introduction | Literature Review | Independent Variables | Dependent Variables | Results | Discussion | Conclusions
Hostile-media-effects research suggests some news consumers identify in even the most even-handed news stories as against their own position. The present experiment searched for hostile media effects in people’s responses to news stories and blogs that were either consistent or inconsistent with their own socio/political orientation. Contrary to hostile-media predictions, people did not view news stories as less credible after reading blogs that did not match their views. These results are interpreted in terms of the large influence of psychological variables on the processing of news and blogs: social presence and coorientation.
Key words: hostile media effect, hostile media perception, third-person effect, blogs, media psychology, computer-mediated communication
Even as blogs exploded on the Internet, professional journalists and the mainstream American press remained slow to adopt them. Many of these online journals, or “web logs” consisting of links and postings from most recent to oldest, developed a reputation over the years for taking aim at the mainstream media, often with biting political comment. Thanks to the Internet and special self-publishing software, citizen journalists from myriad streams of life could take on legacy news media for what many characterized as unacknowledged biases, shallowness and arrogance, among a host of other complaints.
Today, as mainstream newsrooms continue to experience devastating losses in audience and advertisers, blogs are flourishing, unbounded by the physical, financial and legal constraints of traditional publishing and broadcasting. A definitive count of blogs worldwide is difficult, in part because many start then quietly disappear. One possible measure comes from blog search engine Technorati, which maintains a directory of roughly 1.3 million active blogs, with roughly 150,000 on politics alone. According to one of the site’s recent annual State of the Blogosphere reports, blogging is no longer the purview of upstart citizens, either. Many blog on behalf of corporations, sometimes in search of new clients. (Reflecting this change, Technorati’s latest annual study, for 2013, is now titled the Digital Influencer Report and focuses on brand marketing.) In addition, about one-third of bloggers work in the mainstream media, which once viewed blogs as the “enemy” 1.
Now mainstream media are embracing this form of writing. Blogs are prominently featured on many newspaper sites. The most successful ones by professional journalists share some of the same characteristics as blogs by non-journalists, including opinion and distinctive author voice2. At the heart of this endeavor are questions fundamental to more than a century of mainstream journalism and its core values: What happens to people’s perceptions of professional journalists’ credibility and expertise when those journalists pull down the veil of objectivity and blog with voice and attitude? The reluctance of the mainstream press to adopt blogs because of such a threat may be justified. Or has the Internet simply re-defined notions of credibility?
This controlled experiment seeks to explore these and other impacts by testing two types of news messages –– traditional news stories and blogs –– the latter of socially liberal or conservative bent; by introducing the psychological variables of social presence (perceived humanness of journalist) and coorientation (perceived similarity to journalist) as mediators; and by assessing credibility of the stories and the Web sites that carry them as dependent variables. To test the perceived influence of blogs and stories, our research borrows heavily from the rich tradition of hostile-media-effects research and Reeves and Nass’ theory of media reality3.
Hostile media effect
An examination of the hostile-media-effects literature reveals little substantive research on blogs, which makes this area ripe for theorizing. The hostile media effect describes the tendency of people who are highly involved in an issue to see traditional, mainstream news coverage of that issue as biased against their own viewpoint. The question arises as to whether that effect holds for opinionated blogs by mainstream journalists now increasingly common on news sites. Taken as a whole, the literature suggests blogs might help elicit an even stronger hostile perception, given their typically opinionated nature, when read together with traditional news stories.
In the first published experiment demonstrating the effect in mainstream news, Vallone, Ross and Lepper 4 showed news broadcasts of the conflict in the Middle East to Arab and Israeli students and found that both groups saw the news as biased in favor of the other side, while nonpartisans saw the same content as neutral. Subsequent research explored this same divisive issue, often showing strong support for the effect5. The effect also emerged for other issues. Christin, Kannaovakun and Gunther6, for instance, demonstrated hostile media effects with highly partisan participants – UPS managers and Teamsters – during the 1997 United Parcel Service strike.
Since the Vallone, Ross and Lepper study7, scholars have published findings from at least a dozen experiments. Key in many of these studies is a person’s strong sense of group identification and membership, either social or political in nature, though some studies argue such identification may not be necessary: Those without strong identification may still view coverage as hostile to their point of view8. In a meta-analysis, or “analysis of analyses,” Hansen and Kim9 concluded the hostile effect exists whether participants are highly involved or not, but the level of involvement affects the magnitude of the effect, or effect size.
Gunther and Schmitt10 suggest research on hostile media perceptions has been growing in part, perhaps, because it shows the critical role of audience variables in mass communication processes, offering alternatives to traditional notions that the media have powerful, homogenizing effects on everyone. The authors also noted similar logic behind the third person effect, which posits people will perceive more influence of an undesirable communication on others than on themselves11>, with that perceived influence increasing as the audience expands or becomes more distant from the self12. In the case of hostile media effects, Gunther and Schmitt’s13 experimental study on the genetically-modified-food debate revealed that if partisans consider messages only in terms of their own opinions, they will see the messages as neutral or favorable, while viewing the same messages as biased in a hostile direction when considering influence on others, in support of the third person effect14.
One of the most important findings of Gunther and Schmitt’s15 study is the effect seems to occur only when the message is distributed via a massmedium, as opposed to a message in a student essay that few read. In other words, the distribution source triggers a different perceptual process in partisan readers. The authors argue more research needs to flesh out this effect in a variety of communication forms.
When people process news, they look for a real human behind it16. For years, scholars struggled to understand why news consumers consistently rated TV news as more credible than newspapers, despite the latter’s relative comprehensiveness. As it turned out, consumers were using different criteria in assessments. In the case of TV, they judged the on-air personality, to whom they often responded positively, while in the case of newspapers, they judged the faceless institution, which they perceived as cold and less trustworthy17 . Along those lines, Reeves and Nass18 suggest that people’s interactions with computers, television and other media are fundamentally social, much like interactions in real life. Their evidence includes at least 35 studies that recreated a range of social and natural experiences but with media taking the place of real people and places19. Key to their conception of the “media equation” is that not only is the technology of the medium important but also the psychology of those who use it. Media-equation research falls squarely in line with decades of research into the dynamics of interpersonal communication, from dyads to large-group interactions, and to the concept of coorientation: how people relate to each other depending on perceptions of others’ ideas and values. As such, this experiment examines the concepts of social presence and coorientation in relation to people’s perceptions of traditional news stories and blogs.
Short, Williams and Christie20 introduced social presence more than 30 years ago drawing on scholarship that seeks to explain the social phenomena of mediated environments. They defined presence as “the degree of salience of the other person in the interaction and the consequent salience of the interpersonal relationship”21. Because of the range of research, the concept has no true disciplinary home and numerous definitions22. Indeed, researchers have explored everything from social responses to computers23to presence in virtual reality24.
Underlying much of this research is Reeves and Nass’25 idea that people treat media as though they were human. They argue this happens because human brains evolved in a world in which all perceived objects were real and only humans possessed human-like shapes and human-like characteristics, such as language, emotion and personality. Anything that seemed real or possessed human characteristics was a real human26. Nass and his colleagues’ research around “Computers Are Social Actors” (CASA) found people automatically apply social rules in their interactions with computers as if the machines were human<27. More recent research along these lines has explored normative group influences in computer-mediated communication28; computers that convey empathic emotions29; and the creation of social presence through machine-generated voices30.
This experiment focuses on presence research concerning interpersonal communication in an online environment. Personal-communication researchers identify three dimensions of social presence: (1) source attention, defined as the degree to which the source is focused on relative to other cues, (2) co-presence, or the feeling of existing with another person, and (3) mutual awareness or psychological involvement –– the feeling of being “known” by another 31. This stream of interpersonal-communication research defines social presence as the degree of psychological involvement or salience of real people communicating through a mediated environment. This definition is similar to Short, Williams and Christie’s32 by characterizing social presence as a feature of a medium, not the user. They argued that the social presence of a medium varied according to the number of social cues it offered.
Many current researchers, however, define social presence not as a characteristic of the medium but rather how participants use the medium to communicate, such as for social interaction33. Consistent with this approach, the present experiment draws on features of both the medium and user, defining social presence as a measure of a psychological feeling of distance that can vary depending on the characteristics of the medium and the message. We focused on how journalists can alter the characteristics of an online news article in order to increase perceptions of social presence — that is, whether audiences view journalists as real people, not automatons.
In light of the preceding discussion on how readers respond to media messages depending on their own political views and their social responses to messages, we offer the following hypotheses in each condition:
H1a: Scores for blog social presence will be higher than for news-story social presence.
H1b: Scores for blog social presence will be higher when the blogs’ opinion matches that of the participants’ political affiliation.
H1c: Social presence will be higher for news stories when blogs match participants’ political affiliation.
Most research on coorientation, or how people relate to each other based on shared ideas, has been conducted since the mid-1960s and is an eclectic synthesis of five older schools of thought dating back to 1902, including the symbolic interactionist approach34. Contemporary research has looked at everything from teenagers’ coorientation behavior toward pop music35 to the ways scientists view newspaper reporters based on personal contacts with them36.
In a special edition of American Behavioral Scientist devoted to explicating coorientation, Wackman37 identified three indicators of coorientation: 1) Agreement, or the similarity between two people’s cognitions about an object; 2) Congruence, or the similarity between one person’s cognition about an object and estimate of another person’s cognition about that object; 3) Accuracy, or the similarity between one person’s estimate of another’s cognitions about an object and that other person’s actual cognitions about the object.
Because the journalists in this experiment are not real, and therefore agreement and accuracy cannot be measured, this study uses a congruence measure to capture participants’ perceptions about their own cognition and the supposed writers’ ideas about various topics. This study seeks to expand on recent findings that suggest perceived story credibility could in part be a product of coorientation38.
H2a: Scores for coorientation will be higher for blogs than for news stories.
H2b: Scores for blog coorientation will be higher when the blogs’ opinion matches that of the participants’ political affiliation.
H2c: Coorientation will be higher for news stories when participants read blogs that match their political affiliation.
Because of the news media’s longstanding reliance on the ideal of objectivity and belief that objectivity leads to higher credibility, increasing social variables in stories potentially could harm people’s trust of the media. Definitions of media-related credibility abound in the literature. Generally, credibility is defined as a multidimensional construct that involves the perceived believability of a message (article), source (journalist or media company) or medium (newspaper, Web site, radio station, etc.). Partly in response to findings that people rated TV news as more credible than newspapers, despite the lack of depth and completeness, Gaziano and McGrath39 created a 12-item scale that included questions measuring fairness and community concern and that loaded onto a single factor, which they labeled “credibility.”
However, Meyer40 found their results indicated two factors, believability and community concern, and created a scale reflecting both. Many current credibility measures draw on both scales, including this experiment. Thus, this study defines perceived credibility as a function of 1.) factualness and accuracy (believability dimension) and 2.) concern mainly with the community’s interest (community-affiliation dimension)41. Because the literature has not been clear about just what readers were rating when they reported on news media credibility, this experiment measures perceived credibility at the level of the story and the source (organization).
H3a: Story credibility will be higher for news stories than for blogs.
H3b: Story credibility for matched blogs will be higher than for non-matched blogs.
H3c: Participants will give lower story credibility scores to news stories after they read a blog that does not match their political affiliation.
H4a: Organization credibility will be higher for news stories than for blogs.
H4b: Organization credibility for matched blogs will be higher than for non-matched blogs.
H4c: Participants will give lower organization credibility scores to news stories after they read a blog that does not match their political affiliation.
The study relied on a 2 (news story vs. blog) by 2 (conservative blog vs. liberal blog) repeated-measures, within-subjects, full-factorial experimental design. We recruited a convenience sample of 152 college students from a large Midwestern university in the United States who voluntarily signed up to get extra credit in class. Their volunteering met the requirement of random selection of participants needed for a controlled experiment. We sought a total sample size of approximately 140-150 based on our previous research that tested the same variables in similar contexts with significant effects. We handed out cards in a computer lab that assigned each person to a different version of the study online. Each version showed each story presented in different orders by topic and by condition (see details below), thus meeting the requirement of randomly assigning participants to different conditions needed for a controlled experiment. Each respondent read a total of four articles (two blogs and two traditional news stories), though the two blogs for each person had either conservative or liberal stances. Because a question at study’s end asked for each person’s political affinity, we could then assess whether the randomly assigned blog matched the person’s self-described affinity. The within-subjects design allowed each participant to read two stories in each of the news conditions and to serve as his or her own control, minimizing the number of participants needed.
Of the sample, 147 completed the entire study. For the five who did not, we replaced their missing values with the regressed-mean, which predicts what the respondents’ score for this variable would have been based on their other responses. We collected demographic information from participants as background information though the experimental design did not require it. (The study tested only for the effect that manipulating the story presentation may have had, not the effect of gender, race or year in school.) For interest, more than 49% of the respondents were freshmen, 33% were sophomores, 14% were juniors, 3% were seniors, and 1% were graduate students. The average age of all participants was 19. The sample predominantly was female (70%) and white (80%) but did include African-Americans (4%), Asians (3%) and Hispanics (9%). The political affinity of participants was 52% socially liberal and 36% socially conservative. About 12% reported they were neutral.
Article type (traditional vs. blog, manipulated)
The four stories covered topics of potential interest to students: professors’ use of plagiarism software, college drinking, steroids in college sports and job prospects after college. All articles were roughly 8-10 newspaper column inches, each fitting on a computer screen, to control for story length as a potential confound. We varied each story by subject such that sometimes it took the form of a blog and sometimes a news article. Participants were randomly assigned to websites that presented the stories in a counterbalanced order to help ensure story order did not influence results. The experiment took most students less than 30 minutes to complete.
We obtained the articles from news websites, but manipulated them to make them apply to the university and to conform to experimental conditions. We changed all names and media identifiers, but we used mastheads and other typographical features from news websites in the Midwestern state to help with external validity, or to provide the sense of a real news-reading experience in the computer lab. However, the mastheads represented smaller newspaper sites the researchers thought most participants would not necessarily have strong opinions about.
Traditional news story (manipulated)
This news-story condition received little to no manipulation as these were news stories selected for their factual approach to the topic. We ensured these stories contained no language that would elicit participants’ feelings of social presence or coorientation with the reporter. We also ensured the story refrained from taking any kind of a political stand. In other words, we made sure the stories upheld the objective standards and format of traditional inverted-pyramid news stories based on how we wrote and presented them.
Blog (conservative vs. liberal, manipulated)
The blogs, on the other hand, took on a conversational tone that clearly expressed the author’s point of view with evaluative language spoken in first-person. Each blog was clearly labeled as such and included an “editor’s note” indicating it was written by a newspaper staff writer. The blog clearly conveyed the personal opinion of its writer. We manipulated key words and phrases in each blog to convey the writer’s political affinity but left all else the same to control for the effect of socially conservative and socially liberal blogs.
Political affinity (7-point scale, measured)
We measured participants’ political affinity with a single question at the end of the study that asked, “On the following scale, please rate your general stance on social issues, such as universal health care, unemployment and welfare.” On the scale, “1” indicated liberal while “7” indicated conservative and “4” indicated neutral. For hostile-media-effect tests, we broke this scale down to liberals (1-3) and conservatives (5-7) and eliminated neutral responses.
For the following measures, scales consisted of five-option, Likert-style questions (strongly agree to strongly disagree). We worded some questions negatively to help minimize patterned responses, then reverse-coded those items prior to statistical analyses.
Social presence (measured)
This variable was measured using a scale developed by Tamborini and Skalski43 and adapted to the current study to apply to a reader-reporter relationship: “I felt like I got to know the reporter,” “At times, I felt like the reporter was in the room with me” and “I thought of the reporter while reading the article.” We computed reliability measures on each variable separately for news stories and blogs. Cronbach’s alpha for news was. 536 while for blogs it was .595. While alpha scores were low, we relied upon the previously tested scale for reliability.
The scale to measure coorientation was based on conceptual definitions in the literature, particularly Wackman’s (1973): “I felt like this reporter probably is a person kind of like me,” “I understand the story’s issue in the same the reporter does,” “I see myself as quite different from this reporter,” “I think this reporter has my interests at heart,” and “I would find it quite difficult to talk with this reporter on this issue.” Reliability scores were .795 for news stories, .850 for blogs.
Story credibility (measured)
Story credibility was measured using four questions modified from Gaziano and McGrath’s (1986) study: “The article was accurate,” “I believe what I read in the article,” “I can trust what I read” and “I’m not sure the article told the whole story.” Reliability scores were .773 for news stories, .891 for blogs.
Organization credibility (measured):
Organization credibility was measured using items that represented dimensions of community interest, in keeping with Meyer’s44 credibility conceptualization. The final scale included the following four questions: “This company probably cares about readers like me,” “Its reporters seem to be well-trained,” “The company seems in touch with the average person” and “The company probably thinks it’s important to publish quality reporting.” Reliability scores were .780 for news stories, .875 for blogs.
The study began by drawing upon the within-subjects design to test mean-score differences in social presence, coorientation, article credibility and organizational credibility, as well as the effect of reading blogs that matched or did not match participants’ stated political affiliation. We conducted a repeated-measures ANOVA with story type (blog or news) as the comparison group and whether the blog that participants read matched their stated political opinion as an interaction effect. Mean-score comparisons are based on estimated marginal means from the repeated measures ANOVA because they take into account the fact the measures are not independent. Statistically significant F-scores existed on the main grouping variable (story type of news or blog) for all four dependent variables, but offering support only for H1a, H3a, and H4a
Table 1 Repeated-measures ANOVA comparing blogs and news stories and/or news and blogs by political affiliation and the type of blog stories participants read (either liberal or conservative)
|Story Type (blog or news)||1||77.881||390.834**|
by Matched Political Opinion
|Story Type (blog or news)||1||15.604||49.081**|
by Matched Political Opinion
|Story Type (blog or news)||1||36.368||111.944 **|
by Matched Political Opinion
|Story Type (blog or news)||1||16.541||67.340**|
by Matched Political Opinion
* p < .05. ** p < .01.
Regardless of political affiliation, mean social presence scores for blogs were consistently higher than those of news stories across political affiliation and blog type, while mean scores for story and organizational credibility were higher for news stories than blogs. Though coorientation’s F-score was significant, it did not support H2a. Coorientation scores for new stories were higher than they were for blogs, or the opposite of what was predicted. This could be because of the opinionated nature of the blog posts.
Estimated marginal means offer additional insight into the interaction effects of story type (blog or news) and whether the blog matched political affiliation (see Table 2), or the remaining hypotheses tests.
Statistically significant F-scores existed for coorientation, story credibility and organizational credibility, offering support for H2b, H3b and H4b: coorientation, story credibility and organizational credibility will be higher for blogs when the blog matches the participant’s political affiliation.
Table 2 Independent samples T-test for blogs and news stories comparing whether participants had read a blog that matched their political opinion
|Social Presence||No Match||81||3.4797||.52425||-.84|
|Story Credibility||No Match||81||2.5881||.77062||-2.57**|
|Organization Credibility||No Match||81||2.9767||.73143||-2.24**|
|Social Presence||No Match||81||2.4637||.49178||-1.03|
|Story Credibility||No Match||81||3.4675||.45525||1.04|
|Organization Credibility||No Match||81||3.5935||.42909||.72|
* p < .10 ** p < .05 *** p < .01
No differences emerged for social presence on matched blogs, showing no support for H1b. ANOVAs also showed no support for H1c, H2c, H3c, and H4c: social presence, coorientation, story credibility, and organizational credibility will be higher for news stories when the blogs participants read matched their stated political affiliation. Mean-score differences for news showed no significance for social presence, story credibility and organizational credibility. Coorientation mean scores, meanwhile, differed significantly for news stories after participants read a blog that matched their political affiliation, but again in the opposite direction predicted.
These findings paint a nuanced though surprising portrait. We found significant differences on a variety of scores, but only half supported our hypotheses. In the case of hostile media effects, we found no support on key measures of credibility.
Looking more specifically at the psychological measures, social presence means were nearly identical when comparing matched and unmatched blogs, as well as matched and unmatched news stories. Social presence scores for blogs, though, far exceeded those of news stories, explaining 72 percent of the variance in story type. Readers noticed the person behind the story in a blog but appeared unaffected by the blog’s opinion. The opposite occurred with coorientation: Readers generally cooriented with news more than blogs. Between blogs, though, participants identified mostwith those that matched their predispositions, as predicted. Coorientation also revealed the study’s largest mean differences between blogs and news stories when political affiliations did not match. This suggests a reader may have an easier time relating to a journalist after reading something that matches the reader’s political affiliation, which could be a precursor to the hostile media effect. Scores for coorientation, or perceived similarity to another, are of particular interest. Meyer, Marchionni and Thorson45> found coorientation to be a powerful predictor of story credibility in both traditional online news stories and blogs. Together these studies suggest that, irrespective of the hostile media effect, audiences respond well to articles in which they identify with the author.
The hostile media effect is most commonly demonstrated by differences in credibility assessments. Perhaps not surprisingly, readers perceived blogs that matched their predispositions as having more story and organization credibility than unmatched blogs. But participants who read unmatched blogs did not then perceive news stories as having less story and organization credibility, revealing no hostile media effect. Indeed, readers found news stories considerably more credible than blogs even when the blogs matched readers’ predispositions. For all news stories, mean scores for story and organization credibility were relatively high and essentially the same, regardless of whether participants read matched or mismatched blogs. In other words, statistical significance emerged in the differences between matched and unmatched blogs and not in the differences between news stories and matched/unmatched blogs.
The implications are important, particularly for perceptions of credibility, journalism’s grail. Participants’ biases did not influence how credible they viewed news stories and the organizations behind the news. However, participants connected better in some ways with blogs than news, as revealed in scores for social presence and coorientation. After years of reluctance, newspapers are now experimenting with allowing reporters to blog, offering their own observations on news events and in their own voice, in defiance of traditional core values of impartiality and detachment. Our findings suggest readers definitely notice the person behind the blogs, as evident in social presence scores. But readers appear to still find news stories more trustworthy. Newspaper managers’ reluctance to let reporters blog may have been sound. This echoes findings in Meyer, Marchionni and Thorson’s46study, which showed weak scores for perceived expertise and credibility in blogs. In the present study, participants did seem to bond better with blogs, though, and this and other studies suggest connection concepts may be increasingly important, at least with younger news audience.
In short, this study reveals that what news audiences consider credible may not be changing so rapidly. This is particularly striking given the youth of our study’s participants, all born in the digital age and likely familiar, if not comfortable, with blogs. This may be a glass half-empty argument, though. Some might say moderate scores for measures of credibility in various blog conditions might not have even been conceivable just a few years ago. Fundamental changes are underway, perhaps with more to come, in how audiences view news. News managers would be wise to carefully explore the potential of blogs to bring in younger audiences, the most coveted for mainstream media these days, and look for ways to build credibility with them. One way might be to involve them more intimately in a conversation as both reader and source47. In separate experiments, Marchionni48found coorientation to be a powerful predictor of readers perceiving a story as conversational, or co-created by journalist-audience, as well as credible. In online news at least, that single variable of perceived similarity to another may be re-shaping the traditional contours of the hostile-media effect.
As with all empirical studies, this one has limitations. Participants included journalism news majors, and they may have been pre-disposed toward traditional news formats, despite their youth. The experiment’s within-subjects design mediated the influence of demographic factors, as participants’ responses were compared only to their other responses and not to those of other participants. As may be obvious, though, while we cannot generalize to the newspaper reading population from any experiment, we can generalize to relationships between variables and theorizing about them, as Lang (1996), Shapiro (2002) and others note, even with student subjects. Future research also should address whether more polarizing topics in stories and blogs, such as abortion, might yield more significant effects among partisans, as has happened in many third-person and hostile-media studies.
In the end, the hostile media effect suggests a powerful test of how political affinity might influence a person’s perception of newspaper credibility and expertise in a vastly-changed, 21st-century news environment. This study, however, suggests that effect in some cases may be tempered by measures of connection more than political affiliation.
Marchionni is a visiting assistant professor of communication at Pacific Lutheran University in Tacoma. Meyer is an associate professor at the E.W. Scripps School of Journalism at Ohio University. Thorson is associate dean of graduate studies and research at the School of Journalism at the University of Missouri.