Elements of research design
- 1 Objectives
- 2 Introduction
- 3 Types of Studies
- 4 What is a Research Design?
- 5 Research Design Approaches
- 6 Operationalizing a Research Design
- 7 Conclusion
- 8 References
- 9 Discussion questions
- 10 Problems
- 11 Glossary
- Types of Studies
- Why is this Study Important?
- What is a Research Design
- Research Design Approaches
- Operationalizing a Research Design
- Research Methods
Several prominent Political Science scholars have stated that "the lack of focus on research design in social science statistics is as surprising as it is disappointing..."<ref name="KKV">King, Gary; Keohane, Robert; Verba, Sydney (1995). American Political Science Review 89 (2). </ref>. They note that experiments in the social sciences are fairly uncommon, but that researchers can have a significant effect on "the value of our qualitative or quantitative information, even without statistical corrections, by improving the design of our research"<ref name="KKV" />. Therefore, while social scientists often do not have the luxury of a controlled research environment, they do have the ability to conduct good research. This starts with giving research design the proper attention.
Types of Studies
The purpose of conducting a study is to find answers to questions or to explore new and innovative ways of solving problems; it is a systematic way to conduct research. All studies are not created equal. How sophisticated a study can be conducted and how entailed a research design can be developed depends on the amount of information available or attainable.
There are three main types of studies:
1) Exploratory Study:
This type of study is conducted when not much is known about the subject matter at hand or no information is available about how similar problems have been researched or solved in the past. These studies are often necessary in order to understand the nature of the problem (an essential element in research design), since there may be little or no previous work to provide guidance. Once the data reveal a pattern or a story, other characteristics of research design can be applied. Studies that collect data through observations or interviews (i.e.qualitative studies) are often exploratory in nature.
When Tea Party Rallies began popping up all over the country, reporters started going to the events to report on what was happening. They interviewed participants to find out about their participation in these events. This exploratory research did not begin with any theory about how the events fit into the broader political context. Rather they only sought to figure out what was happening.
2) Descriptive Study:
This type of study is undertaken in order to describe the characteristics of variables or identify patterns among variables. Often with these studies a researcher may not have well defined hyptheses at the onset and is relying on the study to help generate testable hypotheses. This type of study estimates rather than explains.
After the Tea Party Movement became active during the Health Care Reform debate, and then started mobilizing during the 2010 midterm elections, we became more curious about who these people were. The New York Times and CBS decided to conduct a poll so that they could measure the characteristics of Tea Partiers in a more systematic way. They concluded that Tea Party members tend not to be minorities, tend to be well educated and tend to be conservative.<ref>Kate Zernike and Megan Thee-Brenan (14 April 2010fckLR). "Poll Finds Tea Party Backers Wealthier and More Educated". New York Times. http://www.nytimes.com/2010/04/15/us/politics/15poll.htmlfckLR. Retrieved 7 July 2011fckLR. </ref>
This prompted one pundit to conclude that they are white, bright and right.<ref>Frank James (15 September 2010fckLR). "Who Is The Tea Party? Republicans By Another Name". National Public Radio. http://www.npr.org/blogs/itsallpolitics/2010/09/15/129876488/who-is-the-tea-party-republicans-by-another-namefckLR. Retrieved 7 July 2010fckLR. </ref> From that we cannot conclude that people join the Tea Party Movement because they are white or because they are well educated or because they are conservative. It is simply a description.
3) Explanatory Study:
This is the most complete type of study. In this type of study (as the name suggests), the purpose is to try to "explain" something (relationship, event, outcome, etc.). This is the central goal of (political science) research. In this type of study the principal elements of research design are applied.
Polls have shown that supporters of the Tea Party are racially intolerant. <ref name="resentment">Christopher S. Parker (Christopher S. Parker). "2010 Multi State Survey of Race and Politics". http://depts.washington.edu/uwiser/racepolitics.htmlfckLR. </ref> Some have challenged these findings by claiming that Tea Partiers' conservative politics are mistaken for racial intolerance. A study was conducted by the University of Washington's Institute for the Study of Ethnicity, Race and Sexuality that accounts for both the effect of ideology and support for the Tea Party on racial resentment. This study found that after controlling for conservatism, support for the Tea Party remains a valid predictor of racial resentment.
File:Racial resentment scale.pdf <ref name="resentment" />
Why Is This Study Important?
Besides identifying the type of study being conducted, a study must also have purpose. From this point forward we will assume that the studies being conducted will be of the explanatory variety- those being done with the purpose of explaining something. However, simply explaining "something" is not enough. What is being explained is important:
- 1) What questions does this study propose to answer? Not only does a study require inquiry (a set of questions to answer),but these questions should be interesting.
- 2) Can you identify a missing piece of the literature? What is it that you are bringing to the table that others have not?
- 3) What is the purpose of the project? State objectives (or questions) clearly and concisely early in the study.
- 4) What are the hypotheses? Once you have questions, you need hypotheses about the answers
What is a Research Design?
The best way to conceptualize a research design is to see it as a road map or a plan of action. A research design helps answer the questions that were identified in the section above ("why is this study important?"). In other words, it shows how a researcher intends to accomplish the goals of the study and and the generalizability of these results <ref name= "Butt">Johnson, Janet Buttolph; Joslyn, Richard A. (1995). Political Science Research Methods. CQ Press. </ref>. A research design addresses the six parts of the research process:
- 1. Topic Selection
- 2. Research Question
- 3. Research Methods
- 4. Data Gathering
- 5. Analysis
- 6. Results
Research Design Approaches
Operationalizing a Research Design
Once the question (or questions) has been identified and, of course, the researcher has concluded that the puzzle is interesting (and will revolutionize political science), how will the answer(s) to the the question(s) be found? The assumption will be that Gary King et al. will be pleased with the strength of the research design to this point. The next steps are formulating hypotheses, developing concepts, identifying variables, mapping out causal relationships, deciding what level of measurement (precision) variables fall under, and testing the rigor of these measurements.
HypothesesA hypothesis is a testable statement about a relationship; a cause and effect.
A common example of this relationship is voter turnout and education. The hypothesis suggests that people with less education are less likely to vote. Using data to examine this relationship is testing the hypothesis. If the data shows that this explanation is incorrect, then the relationship between education and voting will not be empirically associated.<ref>Pollock III, Philip H. (2009fckLR). The Essentials of Political Analysis. CQ Press. </ref>
Source: NES 2006
Types of Hypotheses
What Makes a Good Hypothesis?
Developing Good Concepts
Concepts are the way we are able to describe the world around us. If we are interested in the characteristics of a group or why certain people behave a certain way, we use concepts to describe these characteristics and/or behavior. If the U.S. Congress is made up of a certain number of males and females and this number is majority Republican, the terms "male," "female," "majority," and "Republican" are all concepts that describe the make-up of the Congress (although not as precisely as it should be described).
Concepts are developed through a process by which a society or several societies (groups) agree to a particular name for a given behavior or activity over a period of time. This process is continuous and arbitrary, and thus, does not ensure that the same name will be given to the same phenomena or behavior. <ref name="Butt"/>
A classic political example of this would be the term "democracy." This term has evolved to have different layers and different meanings. A whole political science class can probably be devoted to the CONCEPT of democracy.
Therefore, it is important when defining concepts that a researcher be as precise as possible. Sometimes one presumes that their understanding of a concept is the same one that everyone else has. This may not necessarily be the case. More precision and less assumption is the way to go.
How can concepts be measured, especially given the process by which they come about? They cannot. In order to be measured, concepts must be translated into something. Concepts are translated into variables. And as we know from the way concepts are the developed, they contain a great deal of variation (hence, the name "variable" is quite appropriate!). Thus, a variable is an observable characteristic of a concept that can be empirically measured.
What is your age?
What is your education level?
What is your income?
What is your ideology?
These are all examples of variables.
Discovering Causal Relationships
Accuracy of Measurement
Face validity vs. construct validity