Survey Research









Response Effects

Response Effects refer to the phenomenon where something about the survey process impacts the responses given by survey respondents. While it is impossible to eliminate all response effects, it is possible to reduce or eliminate the structural causes of many response effects (that is, when the structure of the survey, such as the order of questions or alternatives, the wording of questions, etc. is the cause) and minimize the impact of others (for example, response effects caused by the race or sex of interviewer can be minimized by randomization in the sampling process)


Framing refers to the use of language, words, or phrases which are likely to lead survey respondents to give certain types of answers.

Example of Framing

In one poll of adults 18 and older in York County, SC conducted in 2003, two versions of one question were included to demonstrate the phenomenon of framing. Respondents were randomly selected to receive one of the versions. In a series of questions where respondents were asked to Strongly Agree, Agree, Neither Agree nor Disagree, Disagree, or Strongly Disagree with a statement, half of the respondents received Version A of one statement while half received Version B.

Version A: "I don't mind if the government keeps tabs on regular people."

Error creating thumbnail: Unable to save thumbnail to destination

Note: The majority of respondents value their privacy and do not approve of the government keeping tabs on them.

Version B: "I don't mind if the government keeps tabs on regular people if it helps keep us safer from terrorism."

Error creating thumbnail: Unable to save thumbnail to destination

Note: Now, the majority of respondents appear to be willing to sacrifice their privacy for perceived safety.

Side by side comparison of Versions A and B:

Error creating thumbnail: Unable to save thumbnail to destination

By creating a frame of "terrorism" for the question, respondents gave a different answer than they otherwise might. Whether you attribute the cause to placing terrorism at the "top of the head" of the respondent<ref>Template:Cite bookfckLR</ref> or causing them to consult a different core predisposition<ref>Template:Cite bookfckLR</ref>, the result of altering their response is the same.

Example of Framing Related to Word Choice

In the fall of 2008, the United States Congress was considering The Troubled Asset Relief Program (TARP), the Bush administration's plan to purchase assets and equity from financial institutions in hopes of stabilizing the financial industry. It was signed into law on October 3, 2008. In September 2008, three separate national surveys (Pew, LA Times/Bloomberg, and ABC News/Washington Post) sought to gauge public opinion regarding this program. Each survey used different language to describe the program.

Pew wording: "As you may know, the government is potentially investing billions to try and keep financial institutions and markets secure. Do you think this is the right thing or the wrong thing for the government to be doing?"

Pew results: Right 57%, Wrong 30%

LA Times/Bloomberg wording: "Do you think the government should use taxpayers’ dollars to rescue ailing private financial firms whose collapse could have adverse effects on the economy and market, or is it not the government’s responsibility to bail out private companies with taxpayers’ dollars?"

LA Times/Bloomberg results: Should 31%, Should not 55%

ABC News/Washington Post wording: "Do you approve or disapprove of the steps the Federal Reserve and the Treasury Department have taken to try to deal with the current situation involving the stock market and major financial institutions?"

ABC News/Washington Post results: Approve 44%, Disapprove 42%

Impact of Word Choice on Survey Results
Poll Wording Favor Program Do Not Favor Program
Pew "investing" 57% 30%
LAT/Bloomberg "bail out" 31% 55%
ABC News/Post "steps" 44% 42%

Certainly, there was no intention to bias the results as all three polls have an established national reputation for reliability.  Additionally, at first blush, none of the questions seem to have any obvious problems with wording.  Nonetheless, the results vary dramatically from one poll to the next.  Part of the reason may be the affective connotations attached to the different key words.  "Investing" often has positive connotations.  People think of "investing" as something positive -- storing away something for the future or improving something.  The results of the Pew poll show that a solid majority of Americans liked the idea of "investing" in the security of national markets and financial institutions.  The term "bail out," on the other hand, often has negative connotations.  Many people associate getting bailed out with saving someone from a silly or stupid mistake caused by an action they should not have taken.  It is often seen as undeserved and necessary because of imprudence.  The results of the LA Times/Bloomberg poll show that a definite majority of Americans were against a "bail out" for companies that had gotten themselves in trouble.  Finally, there usually aren't heavy positive or negative connotations with taking "steps."  The language here is much more neutral.  The ABC News/Washington Post poll results show a country evenly divided on the "steps" proposed to "deal with the [...] situation."

It could be that framing from simple differences in word choice is more likely to occur for issues on which individuals do not hold strong opinions or are difficult for an individual to understand.  Any survey question long enough to adequately explain TARP, giving the respondent enough information to fully understand the program, would have been far too long to practically include in a survey.  As such, the respondent may be more likely to look for, or pick up on, inadvertant cues in the question wording.  "Investing" provided a slight positive cue and showed results favorable of TARP.  "Bail out" provided a slight negative cue and showed results unfavorable of TARP.  "Steps" provided no positive or negative cue and show results that could be interpreted to mean that the public was either evenly divided on TARP or were uncertain due to a lack of familiarity with the specifics of TARP.


Michael Parkin defines priming as the "psychological process in which exposure to a stimulus activates a concept in memory that is then given increased weight in subsequent judgment tasks. Priming works by making the activated concept accessible so that it can be readily used in evaluating related objects."<ref>Template:Cite bookfckLR</ref>

Example of Priming

In surveys, Priming response effects are often the result of question order. Imagine a survey that seeks to measure individuals' concerns about the economy. If respondents are asked what they consider to be the most important problem facing their country or state after they have been asked a series of questions regarding economic concerns, they may be more likely to cite the economy at the "most important problem."

Priming due to question order may also change the criteria used for judgement or evaluation. Asking a series of questions about the economy before asking respondents to evaluate the U.S. president may result in higher evaluations than might otherwise be expected in economic good times and lower than expected evaluations in an economic downturn. Similarly, asking for a presidential evaluation after a series of questions regarding foreign military engagements may result in the individual subconsciously creating their evaluation of the president based on their attitudes regarding foreign policy rather than any domestic issue.

In fact, for these reasons, most political surveys ask questions regarding approval or evaluation for candidates or elected officials, as well as questions regarding the most important problem or issue facing the country or state, somewhere near the beginning of the survey. Another example would be asking questions regarding religious beliefs and church attendance prior to asking questions about issues that are often religiously charged, such as abortion of same sex marriage.





<references group=""></references>

Discussion questions



  • [[Def: ]]
  • [[Def: ]]
  • [[Def: ]]