Top Banner

Click here to load reader

Populations and Samples Anthony Sealey University of Toronto

Jan 04, 2016




Populations and Samples Anthony Sealey University of Toronto. - PowerPoint PPT Presentation

  • Populations and Samples

    Anthony SealeyUniversity of Toronto

    This material is distributed under an Attribution-NonCommercial-ShareAlike 3.0 Unported Creative Commons License, the full details of which may be found online here: You may re-use, edit, or redistribute the content provided that the original source is cited, it is for non-commercial purposes, and provided it is distributed under a similar license.

  • Populations and Samples

    Often researchers are interested in making general claims about relationships between particular political concepts.The complete set of all things to which the specified relationship is thought to apply is referred to as the population of the analysis.

  • e.g. What was the population being analyzed when we investigated the relationship between gender and attitudes towards same-sex marriage?

  • While the population is what is being analyzed, it is often impractical to gather information on the complete set of things included in the population. For this reason, researchers often gather information about a subset of the population referred to as a sample and try to draw inferences about the population based on the information gathered from the sample.

  • In many respects, the best possible type of sample is a random sample, because randomization generally ensures that samples are representative and allows us to determine the likelihood that a given sample is unrepresentative. In many instances, however, non-random sampling techniques are more convenient and sometimes even preferable.

  • Non-Random Sampling Techniques 1) Systematic Sampling 2) Stratified Sampling 3) Cluster Sampling 4) Purposive Sampling 5) Deviant Case Sampling 6) Snowball Sampling

  • Measurement, Sampling and Error

    Notice that we now have two possible sources of error from the process of operationalizing our concepts.The first source of error comes from the measurement process (measurement error). The second source of error comes from the sampling process (sampling error).

  • However, it is possible (although potentially dangerous) to think of sampling error as a type of measurement error.

  • It is also worth drawing attention to the fact that in quantitative analysis, the availability of measures often drives the selection of measures.

  • e.g. Measuring attitudes towards feminism in the World Values Survey:

    Compare outlooks on the statement: D059 On the whole, men make better political leaders than women do. with outlooks on this statement: D062 A job is alright but what most women really want is a home and children.

  • Now lets compare data availability:

  • Now lets compare data availability:

    little data is missing for femism1

  • Now lets compare data availability:

    all the data is missing for femism2

  • So what do we do? We use femism1 (D059) not because its a more valid measure than femism2 (D062), but because femism2 isnt available.

  • Finally, it is important to note that in many instances, the operationalization of measures is often highly controversial and affected by the values and beliefs that scholars bring to their research. e.g. Relative vs. absolute measures of poverty.

  • Measurement clip from:

    The Gapminder Foundation

  • Credibility, Transferability and Validity

    Validity is a concept most easily identifiable with quantitative research.The term has a wide range of possible meanings in the field of research methods, but the central idea revolves around notions of accuracy and truthfulness.

  • First, we can think of measurement validity. For a measure to be valid, it must accurately represent the concept that it is intended to operationalize.

  • One aspect of measurement validity is face validity. A measure has face validity if it is an appropriate operationalization of the concept.

    e.g. Which has greater face validity as a measure of animal rights activism: whether someone owns a pet or whether an individual donates to animal shelters?

  • The text also discusses the ideas of convergent and divergent validity. These notions of validity can be applied to indicators. Indicators are said to have convergent validity if the variables are thought to be indicators of the same measure and they yield similar results for most cases.

  • e.g. The indicators opposition to same-sex marriage and opposition to abortion rights are said to have convergent validity if they are thought to be indicators of a measure of moral traditionalism and they yield similar results for most cases.

  • Indicators are said to be divergently valid if the variables are thought to be indicators of the same measure but have reverse directionalities and they yield opposing results for most cases.

  • e.g. The indicators support for same-sex marriage and opposition to abortion rights are said to have divergent validity if they are thought to be indicators of a measure of moral traditionalism but have reverse directionalities and they yield opposing results for most cases.

  • We can also apply the notion of validity to studies themselves. One such application is the idea of external validity. An analysis is said to have external validity if its findings can be generalized from the sample included in the analysis to cases outside the sample.

  • Credibility and transferability are concepts that have been developed by qualitative researchers in as parallels to the notions of measurement and external validity in quantitative research.

  • Qualitative research is said to be credible if the data used in the qualitative account fits the world being described; the qualitative account must be believable. Qualitative research is said to be transferable if the findings can be applied to other contexts.

  • Dependability and Reliability

    Another important characteristic of quantitative measures is that they should be reliable. A measure is said to be reliable if it consistently obtains comparable results in a variety of instances of measurement.

  • In qualitative research, the analogous attribute is often described as dependability, but again refers to the idea of a consistency between the collected data and the conclusions drawn (the results). Another way of thinking about this is to ask: would the results be consistent if the analysis of the collected data is repeated by other researchers?

  • Confirmability and Replicability

    Some qualitative researchers also draw a distinction between the ideas of confirmability and replicability. Such a distinction is quite subtle, however, and probably exaggerates the extent to which quantitative analyses are actually replicable.

  • The key idea for both is to ask: if we were to redo the study again, would the conclusions drawn be the same again?

  • Terminological Summary

    Quantitative Research Qualitative Research

    Measurement ValidityCredibilityExternal ValidityTransferabilityReliabilityDependabilityReplicabilityConfirmability

  • Validity, Reliability and BiasAs we have seen, the concept of validity has a broad range of possible applications.However, two important criteria by which to conceptualize validity involve reliability and biasedness. Valid measures should be both reliable and unbiased.

  • A reliable or consistent estimator is one that tends to produce estimates that do not differ significantly from each other (i.e. the variance of the estimates is low). An unbiased estimator is one for which the average of all possible sample statistics is equal to the population parameter that it is estimating.

  • e.g. #1:

    Reliable but Biased

  • e.g. #2:

    Unbiased but Unreliable

  • e.g. #3:

    Biased and Unreliable

  • e.g. #4:

    Reliable and Unbiased

  • A Schematic Representation of Some Aspects of the Concept of Validity

  • Validity




    Face Validity

    Unbiased- ness



    See course description. *See course description. *See course description. *See course description. *

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.