A Definition of Single Source Bias in Social Science Research

During the early part of the last century, Thorndike (1920) found that traits such as intelligence, skill, and reliability were highly correlated within subjects. Without immediately realizing it, he observed and identified a problem in social science research that even today has not yet been solved. He wrote:

Those giving the ratings were unable to analyze out these different aspects of the person’s nature…and rate each [attribute] in independence of the others. Their ratings were apparently affected by a marked tendency to think of a person in general as rather good or rather inferior and to color the judgments of the qualities by this general feeling. (p. 25)

Thorndike called this phenomenon “the constant error of the ‘halo’” (p. 28), which later became known as the halo perceptual error.

Single source bias defined

What Thorndike observed was a form of common method variance, defined as overlapping variability due to the way data were collected rather than from a real, substantive relationship. A form of common method variance, single source bias arises when overlapping variability is due to data collected from a single source (Campbell & Fiske, 1959).

People are complicated; collecting data from them is inaccurate and unreliable. Humans use cognitive schema to generalize evaluations (Dipboye & Flanagan, 1979; Mitchell, 1985). The result is difficulty distinguishing multiple attributes of the same object, idea, or person. Although an efficient scheme for cognitive processing, it is definitely less accurate when it comes to forming perceptions. When data are collected from self-reports, the dangers of single source bias are most prevalent.

Effects of single source bias

Single source bias leads researchers into thinking that a genuine relationship exists between two variables when the methods used to collect data are to blame. Since it is impossible to partial out the single source bias relationship from a substantive one, a researcher can never really know the true nature of the relationship (or non-relationship) between and among variables collected from a single source (Podsakoff & Organ, 1986).

Single source bias is far more a problem than most researchers realize. Too often, journal editors and reviewers are unaware of single source bias and fail to recognize its potential to confound findings in an empirical study. Baugh, Hunt, and Scandura (2006) said it best when they wrote:

What was once expressed as a concern about a manuscript is now gaining virtual "fatal flaw" status. Where it was at one time sufficient to acknowledge the issue and beg forgiveness in the discussion of limitations of the research, this approach no longer suffices. (p. 165)

When designing a study, be sure to ask yourself whether data from a single source might be responsible for the results you anticipate. Consider collecting data from multiple and externally verifiable sources. Avoid designs that rely exclusively on self-reports. You will discover that when editors and reviewers suspect single source bias in your research, it will be increasingly difficult to find a journal that will publish your manuscript.

References

Baugh, S. G., Hunt, J. G., & Scandura. T. A. (2006). Reviewing by the numbers: Evaluating quantitative research. In Y. Baruch, S. E. Sullivan, & H. N. Schepmyer (Eds.), Winning reviews: A guide for evaluating scholarly writing (pp. 156-172). New York, NY: Palgrave Macmillian.

Campbell, D. T. & Fiske, D. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105.

Dipboye, R. L., & Flanagan, M. F. (1979). Research settings in industrial and organizational psychology: Are findings in the field more generalizable than in the laboratory? American Psychologist, 34(2), 141-150.

Mitchell, T. R. (1985). An evaluation of the validity of correlational research conducted in organizations. Academy of Management Review, 10(2), 192-205.

Podsakoff, P. M., & Organ, D. W. (1986). Self-reports in organizational research: Problems and prospects. Journal of Management, 12(4), 531-544.

Thorndike, E. L. (1920). A constant error in psychological ratings. Journal of Applied Psychology, 4(1), 25-29.