Sampling Method and Unit of Analysis
This study analyzed the depiction of the war across different communication forms/modalities. Units were gathered in a purposive sample. Data gathering started on July 11, 2004 and ended July 19, 2004. This was a Sunday through the following Monday period to accommodate gathering two samples from weekly news products.
Samples were collected from ten categories, both traditional and non-traditional communication forms/modalities. This includes: national newspapers, regional newspapers, weekly news magazines, radio news programs, political talk radio, nightly television newscasts, television news magazines, political talk television shows, late night entertainment talk shows and Internet news Web sites. Due to the drop in newspaper readership and television viewership in the last decade (Fallows, 1996; Hulin-Salikin, 1987; Pew Research Center, 1996), this study featured a number of non-traditional media such as late night entertainment talk shows and Internet Web sources. Experts maintain that non-traditional media fills the void with audiences for information gathering left by declining newspaper and nightly television newscasts use levels (Just, Crigler, Alger, Cook, Kern & West, 1996; Kern, Just & Crigler, 1997; Rehm, 1996).
The samples were then content analyzed. Content analysis is one of the most prominent research tools in mass media studies (Berelson, 1952). Content analysis is widely used in mass media research due to its efficiency and effectiveness in analyzing the content of the media message (Wimmer & Dominick, 1983). For the auspices in research to the media’s perception of the war in Iraq, analysis of media message content must be systematic and objective in order to avoid viewing balanced articles as only those that are favorable to the military. Berelson (1952) also states that content analysis is a research technique that is objective, systematic, and quantitative in description. Krippendorff (1980) states four advantages to content analysis: (1) Content analysis is an unobtrusive technique; (2) Content analysis accepts unstructured material; (3) Content analysis is context sensitive and is thereby able to process symbolic forms; (4) Content analysis can cope with large volumes of data.
Berelson’s (1952) definition of content analysis has four distinctive characteristics. The first feature is objectivity, from which the research acquires scientific character. An objective study will be precise and reliable, such that the same result would be secured if different people conducted the same research. The second feature, systematization, which involves selecting of the appropriate procedure to analyze all relevant content while eliminating any biased analysis, and designing of a reliable system of analysis to secure data relevant to hypotheses or research questions. The third characteristic of Berelson’s definition is quantitativity, which requires that the research data lend itself to numerical recording. According to Berelson, this is the most important feature of content analysis, differentiating the research from other ordinary reading. Manifest content is the fourth feature, and is opposite of latent content. Manifest content is apparent content, or the obvious meaning of content that can be understood by different people in generally the same way.
Each sample was selected based on coverage in Iraq in three categories (1) politics, (2) military, and (3) life in Iraq. Units in the politics category involve war coverage focusing on the President or members of his administration (i.e. Secretary of State, Secretary of Defense), and other foreign governments involved in the war. Units focusing on military involve life stories of individual military members, battles, death tolls, military unit stories and military member’s families, this also includes coverage of coalition forces. Finally, the life in Iraq category deals with the new interim Iraqi government, descriptions of how life is in Iraq now, civil engineering stories (building schools, hospitals), training of Iraqi police force, etc.
National newspaper war coverage was examined using the following exemplars: USA Today, The New York Times, and The Wall Street Journal. These mediums were all chosen because of their national influence, they also have an ability to shape what regional media outlets choose to cover (Audit Report, 1995). The regional newspapers analyzed were the Washington Post, L.A. Times, and The Chicago Tribune. The Washington Post was selected because it is a prominent East Coast paper, the L.A. Times because it is a prominent West Coast newspaper (Bruce, Craig, & Haigh, 2004), and The Chicago Tribune because of it influence in the Midwest.
Articles were collected everyday. Each the paper was published within the data gathering dates (Sunday, July 11, 2004 through Monday, July 19, 2004). The newspapers were delivered to the study site (they were not gathered off the paper’s Web site). This ensured the correct measure of prominence within the newspaper. The unit of analysis was articles and photos. They were selected if they discussed/showed war in Iraq, to include: the politics concerning Iraq within the United States, U.S. military coverage, and stories about life in Iraq. This did not include editorials, sports, or comic strips. Only news articles/photos concerning U.S. politics, U.S. military, and life in Iraq during the selected timeframe were analyzed. Photographs accompanying a story were included as a part of the story, thus they were coded as one unit.
The next communication form was prominent news magazines: Time, Newsweek, and U.S. News and World Report. They were selected because they are long-standing, news-based magazines with national circulation (Simmons Study, 1994). The unit of analysis was articles covering the war. Each story including its photos was analyzed. If the story had sidebars, they were analyzed separately from the story.
To analyze how radio news covered the war, this study selected 5-minute news broadcasts from both National Public Radio (NPR) and America Press (AP) Network News. National Public Radio was chosen because it is broadcasted nationally and available in most rural areas. NPR is a 24-hour a day, talk radio station with a 5-minute newscast at the top and bottom of every hour (i.e. 12:00 and 12:30). America Press Network News is not a stand-alone station, it provides a 5-minute newscast to its subscribers, who are radio stations across the country. The unit of analysis was each story discussing the war in Iraq.
This study also examined political talk radio shows. The three exemplars chosen were “The Rush Limbaugh Show”, “The O’Franken Factor” hosted by Al Franken and Sean Hannity’s “Tuned Into America.” The shows were selected base on Arbitron ratings, with “The Rush Limbaugh Show” the most listened to political talk radio program followed by “Tune into America” and “The O’Franken Factor”, respectively (Talkers Magazine On Line, 2004). Each program is three hours long. This study sampled one hour of each show everyday it aired (Monday-Friday). The unit of analysis was segments, banded by commercial breaks.
Another communication form was nightly television newscasts. This featured 30-minute newscasts from each of the major news networks, “NBC Nightly News”, “CBS Evening News”, “ABC World News Tonight” and CNN “News from CNN”. A newscast from each network was chosen to get a non-discriminate sample of national news coverage (Mediamark Research, 1993). The study broke the newscasts down into individual stories and examined each story for its depiction of the war. The unit of analysis was individual stories within the newscasts. This included anchor read stories, also known as readers, anchor lead-in and reporter package, and anchor reader with sound bite.
The study also examined television news magazines. Again, each of the major networks was represented: NBC’s “Dateline”, CBS’s “60 Minutes” and “60 Minutes II”, and ABC’s “20/20”. These four programs were selected to ensure a complete sample of news magazines; these are the four most popular news magazine shows in the United States (Nielson, 1996). They are weekly and biweekly programs. The unit of analysis was each story, including anchor lead in, discussing the war or Iraq.
Another communication form was political television talk shows. The study observed shows that represent two major news agencies, MSNBC and FOXNEWS. MSNBC’s “Hardball” with Chris Matthews and FOX NEWS’s “The O’Reilly Factor” with Bill O’Reilly were chosen because they are a representative of the variety of political talk shows available to viewers. Each show is an hour in length and airs every weekday. The entire show was evaluated. The shows were broken up into 5-minute segments minus commercials and each of those sections were analyzed.
Another communication forum was late night entertainment talk shows. This includes NBC’s “The Tonight Show” with Jay Leno and CBS’s “The Late Show” with David Letterman. These two shows were selected for analysis because they are the two most popular late night shows (Nielson, 1995). Each show is an hour long and airs every weeknight. The sample evaluated the first five minutes of both shows, focusing on the introductory monologue at the beginning of every show. The unit of analysis was each joke of the monologue pertaining to the war or Iraq.
This study also focused on the Internet news Web sites. The Web sites selected for analysis were The Drudge Report (www.drudgereport.com) and WorldNet Daily News (www.worldnet.daily.com). To avoid content overlap that may skew study results, this study did not include any news Web sites from the major television networks (i.e. www.cnn.com, www.msnbc.com). Nor does the study include any news Web site from primarily search engine Web sites (i.e. www.google.com, www.yahoo.com). The Drudge Report and WorldNet Daily News are Web sites whose sole purpose is news dispersion. The sample frame for the Web sites was articles that were accessed with one mouse click only. This helped define the prominence and limit the timeliness associated with each article. The study went to each home page and looked at the articles listed and if they covered the war in Iraq and the article could be reached with one mouse click, it was evaluated for the study. The unit of analysis was the articles covering the war in Iraq. The articles were gathered at 5 o’clock in the evening each day.
Coding Training and Codebook
One written coding instrument was developed to code all units of analysis: Print, radio, television and Web sites. The instrument explained the procedures for individual coders. It also had categorical data entry, i.e. what program was coded? What day did it air/was published? The tool also had sections to divide the subject matter of the units into three categories and three interval scales to describe the story content. Each unit of analysis was coded by two members of a communications graduate studies program. There were nine coders in all. Each coder analyzed two categories. Training was held for each pair of coders. The same codesheet was used for each media. Training was conducted for seven hours on samples of articles and television/radio programs not used in the sample.
Rosenthal’s (1984) formula for reliability was used to determine intercoder reliability. Intercoder reliability was as follows: television news magazines (a=.98; television late night talk show (a=.91); television political talk shows (a=.98); political talk radio (a=.96); weekly news magazines (a=.96); radio news (a= .98); regional newspapers (a= .57); national newspapers (a=.80); and online news sources (a=.37).
Variables measured
The independent variables used in the analysis were communication forums/modalities including: national newspapers, regional newspapers, print news magazines, nightly television newscast, television news magazine, late night television entertainment shows, political talk television, national radio news, political talk radio, and national news websites. There were several dependent variables: overall tone of coverage about the war, the relative use of episodic or thematic framing, and the extent to which feelings of happiness, sadness, anger and fear were expressed within a story. The instrument also had three interval scales and forced coders to select a category to determine if the story/broadcast covered the war by focusing on U.S. politics, U.S. military, or life in Iraq. Coders selected a category for each unit of analysis, and then completed the interval scales with this category as the focus.
The overall tone of the coverage depicting the war in Iraq was assessed using a global attitude measure using one, 5-interval scale including: negative/positive, foolish/wise, worthless/valuable, unfavorable/favorable, bad/good, unacceptable/acceptable. The method was developed by Burgoon, Cohen, Miller and Montgomery (1978) and has a track record of 25 years with strong reliability. A single item, 5-interval scale was used to measure the extent to which a story employed episodic or thematic framing. The measure was developed by Pfau, Haigh et al. (2004) as an operationalization of episodic/thematic framing scale.
The affect measure was developed by Dillard, Plotnick, Godbold, Freimuth, and Edgar (1996), items ranged from 0 (none of this feeling) to 5 (a lot of this emotion). The items tapped four dimensions: happiness (cheerful and happy), sadness (sad, dismal, and dreary), anger (angry, irritated, and annoyed), and fear (fearful and scared).
This study used descriptive statistics and a MANOVA to test for the variations in how the nine communication forums differed in their coverage in the depiction of the war in Iraq. Significant omnibus tests were followed with planned comparisons for predicted effects and with Scheffe post-hoc for unpredicted effects. For all tests, the significance level of .05 was set.