Citation: Huitt, W. (2003, June). Assessment, measurement, evaluation, and research: Types of studies in scientific research. Educational Psychology Interactive. Valdosta, GA: Valdosta State University. Retrieved [date], from http://www.edpsycinteractive.org/topics/intro/research.html
Return to: | Science: A Way of Knowing | Ed Psyc Interactive: Courses |
Type of studies
There are three types of studies utilized in the process of scientific research, corresponding roughly to the three major purposes of science. The first type is descriptive in which the researcher attempts to document what is actually occurring. The study may be either qualitative (descriptions in words) or quantitative (descriptions in numbers). The researcher has no control over the phenomena of study, but simply records what is observed or reported.
The second type of study is referred to as correlational. In this study, the research relates the level of one variable to a corresponding level of another variable in an attempt to discover any relationships between them. The purpose of this type of study is to predict the level of one variable by knowing the level of a second variable. The researcher has only moderate, if any, control over the variables in this type of study. The correlational study is the most often used study in educational psychology as well as in many areas of the social and behavioral sciences. (See Huston, 2002 for an excellent example of the type of data produced by mainly correlational studies.)
The third type of study is referred to as experimental (see also). In this type of study, the researcher randomly assigns subjects to at least two groups (experimental and control). In the experimental group, the researcher manipulates the level of one (independent) variable and observes the corresponding change, if any, in the level of another (dependent) variable. The purpose of this type of study is to determine if there is a causal relationship between the two variables. (See Zaslow et al. (2002) for an excellent example of the type of data produced using experimental studies.)
Experimental studies
An experiment consists of assigning participants to at least two groups, i.e. experimental and control, and administering or manipulating an independent variable in the experimental group while holding conditions constant and equivalent for the control group. The groups (assumed to be equivalent in all respects initially) are then compared on the dependent variable (performance or change in behavior) to determine the likelihood or probability that the independent variable caused these changes in the experimental group. Note the phrase "likelihood or probability" is used since the experimenter is NEVER absolutely certain that the independent variable was the actual "cause" of the observed changes. Alternative explanations exist. Among the most obvious is the possibility that the two groups differed prior to the administration of the independent variable.
There are three fundamental ways that such an occurrence may be ruled out. Subject assignment to groups can increase the likelihood that there are no important initial differences between the experimental and control groups. The subjects may be randomly assigned to the groups thus distributing individual differences evenly across the groups or they may be assigned on the basis of matching on important factors such as age, ability, gender, SES and others. It has been argued that random assignment, particularly where the groups are large, is a more powerful technique and more likely to assure nonsystematic differences between groups, i.e. important extraneous factors are randomly distributed in both groups.
The second method of equating groups consists of statistical accommodations applied to groups consisting of different individuals. Initial group differences may be assessed by pretesting and, if found to be different on the dependent variable prior to applying the independent variable, then using the analysis of covariance to test the differences between groups on the posttest data (same measure) provides a statistical adjustment in the final results based upon initial group differences. The statistical procedure has been characterized as asking the question "What would the data be like if they weren't like they are?" Such a procedure is superior to examining gain scores and is most often used where random assignment of subjects to groups is not possible because the groups are intact already, e.g. educational groupings, or the study is addressing the effect of the treatment on so-called "classification variables" which cannot be manipulated, e.g. gender, ethnicity or SES, and the like.
A third way in which initial group differences may be avoided is achieved by controlling or eliminating them through the experimental design. In the definition of an experiment stated previously, called a two group design, it is assumed that different individuals are assigned to the two groups. Using the same individuals for the experimental and control groups assures group comparability and importantly decreases error variance which results in a clearer picture of the impact of the independent variable. If individuals are to be used twice, then exposure to either condition initially, i.e. order of conditions, could change the results such that order alone could be the causative agent rather than the independent variable. Therefore, the experimental design provides for counterbalancing the order that the subjects receive the conditions.
For example, half of the subjects receive the control condition first followed by the experimental condition while the other half receive the experimental condition and then the control. Whatever the potential effect of having a particular order of conditions has on the outcome is canceled or offset by combining the results for both experimental subgroups (E then C; C then E) and both control subgroups. However, sequence of conditions could interact with the independent variable in a complex manner to produce misleading results.
Research Designs
A excellent overview of experimental and quasi-experimental research designs is provided by Dawson (1997). In general, "true" experiments involve the manipulation of variables and the assignment of subjects in a manner which assures group comparability. They are preferred over other approaches since there are few threats to either internal or external validity, i.e. the validity of the experimental conclusions. Quasi-experiments are efforts to assess the impact of variables over which the experimenter does not have control and/or assessing the effect of a variable under circumstances where subject assignment to groups is not controlled. The accompanying tables developed by Campbell and Stanley (1963) illustrate the differences among "preexperimental" designs which are at best, demonstrations of possible effects of a variable, and true experimental and quasi-experimental designs.
Two quasi-experimental designs widely used in developmental and educational psychology are the longitudinal and cross-sectional designs. Both approaches have their advantages and disadvantages. The longitudinal, a favorite among early developmentalists, is expensive, time consuming, and subject to bias in sample selection, attrition of subjects and experimenters, testing, history, statistical regression, and instrumentation problems. Its strength derives from the singular capability of tracing the development of unique traits across time within the same individuals. The cross-sectional approach avoids many of the problems of the longitudinal, but is subject to risk from the cohort effect, particularly where the age differences among groups are large, e.g. 10 years. This, of course, allows extraneous factors to creep into the data which falsely suggest change due to development rather than cultural events. In an effort to either control for or assess the extent of the above possibilities associated with these designs (principally history vs. cohort effects), the following designs have emerged.
Internal and External Validity
The most important aspect of this presentation for developmental and educational psychology are the threats to the validity (both internal and external) of experimental conclusions since developmental and educational researchers are usually interested in studying classification variables or intact groups such that quasi-experimental designs are often involved.
Resources:
Return to: | Science: A Way of Knowing | Ed Psyc Interactive: Courses | Home Page |
All materials on this website [http://www.edpsycinteractive.org] are, unless otherwise stated, the property of William G. Huitt. Copyright and other intellectual property laws protect these materials. Reproduction or retransmission of the materials, in whole or in part, in any manner, without the prior written consent of the copyright holder, is a violation of copyright law.