Introduction
-
Meta-analysis is a statistical technique for amalgamating,
summarising, and reviewing previous quantitative
research. By using meta-analysis, a wide variety of questions can be
investigated, as long as a reasonable body of primary research studies exist.
Selected parts of the reported results of primary studies are entered into
a database, and this "meta-data" is "meta-analyzed", in similar ways to
working with other data - descriptively and then inferentially to test certain
hypotheses.
-
Meta analysis can be used as a guide to answer the
question 'does what we are doing make a difference to X?', even if 'X' has
been measured using different instruments across a range of different
people. Meta-analysis provides a systematic overview of quantitative
research which has examined a particular question.
-
The appeal of meta analysis is that it in effect combines
all the research on one topic into one large study with many participants.
The danger is that in amalgamating a large set of different studies the
construct definitions can become imprecise and the results difficult to
interpret meaningfully.
-
Not surprisingly, as with any research technique, meta-analysis has its
advantages and disadvantages. An advantage is its objectivity, and yet like any research, ultimately its value depends on
making some qualitative-type contextualizations and understandings of the
objective data.
- Meta-analysis has been used to give helpful insight into:
- the overall effectiveness of interventions (e.g., psychotherapy, outdoor education),
- the relative impact of independent variables (e.g., the effect of different types of therapy), and
- the strength of relationship between variables.
- To get more introduction to meta-analysis, go to
Effect Sizes & Confidence
Intervals
-
Meta analysis reports findings in terms of effect sizes. The effect
size provides information about how much change is evident across all
studies and for subsets of studies.
- There are many different types of effect size, but they fall into
two main types:
-
standardized mean difference (e.g., Cohen's d or
Hedges g) or
-
correlation (e.g., Pearson's r)
- It is possible to convert one effect size into another, so each
really just offers a differently scaled measure of the strength of an
effect or a relationship.
- The standardised mean effect size is basically computed as the
difference score divided by the standard deviation of the scores.
- In meta-analysis, effect sizes should also be reported with:
- the number of studies and the number of effects used to create the
estimate.
- confidence intervals to help readers determine the consistency and reliability
of the mean estimated effect size.
-
For more information about calculating effect sizes and
confidence intervals, see:
-
Tests of statistical significance can also be conducted and on the effect
sizes.
-
Different effect sizes are calculated for different constructs of
interest, as predetermined by the researchers based on what issues are of
interest in the research literature.
-
Rules of thumb and comparisons with
field-specific benchmarks can be used to interpret effect sizes.
According to an arbitrary but commonly used interpretation of effect size
by Cohen (1988), a standardised mean effect size of 0 means no change,
negative effect sizes mean a negative change, with .2 a small change, .5 a
moderate change, and .8 a large charge. Wolf (1986), on the other hand,
suggests that .25 is educationally significant and .50 is clinically
significant.
Using Effect Sizes in Primary Studies
-
Meta-analysis methodologies, particularly effect sizes, are
also applicable to primary research. For example, effect sizes are particularly useful in
program evaluation studies. For more information:
How to Conduct
a Meta-analysis
Meta-analytic Studies of
Psychological Interventions
Hattie, J. (1992). Self-concept.
NJ: Lawrence Erlbaum.
Lipsey, M. W., & Wilson, D. B. (1993). The
efficacy of psychological, educational, and behavioral treatment.
American Psychologist, 48, 1181-1201.
Smith, M. L., Glass, G. V., & Miller, T.
I. (1980). The benefits of psychotherapy. Baltimore: Johns Hopkins University Press.
Meta-analysis Methodology References
Bushman, B. J., & Wells, G. L. (2001).
Narrative impressions of literature: The availability bias and the
corrective properties of meta-analytic approaches. Personality and
Social Psychology Bulletin, 27, 1123-1130.
Cohen, J. (1988). Statistical power
analysis for the behavioral sciences (2nd ed.). New York: Academic
Press.
Glass, G. V. (1976). Primary, secondary, and
meta-analysis of research. Educational Researcher, 5, 3-8.
Glass, G. V. (1977). Integrating findings:
The meta-analysis of research. Review of Research in Education, 5,
351-379.
Wolf, F. M. (1986). Meta-analysis:
Quantitative methods for research synthesis. Beverly Hills, CA: Sage.
|