Data Collection Methods

Document Review

    Use when
    Program documents or literature are available and can provide insight into the program or the evaluation

    Advantages

    • Data already exist
    • Does not interrupt the program
    • Little or no burden on others
    • Can provide historical or comparison data
    • Introduces little bias

    Disadvantages
    • Time consuming
    • Data limited to what exists and is available
    • Data may be incomplete
    • Requires clearly defining the data you’re seeking

Observation

    Use when
    You want to learn how the program actually operates—its processes and activities

    Advantages
    • Allows you to learn about the program occurring
    • Can reveal unanticipated information of value
    • Flexible in the course collecting data

    Disadvantages
    • Time consuming
    • Having an observer can alter events
    • Difficult to observe multiple processes simultaneously
    • Can be difficult to interpret observed behaviors

Survey

Use when
You want information directly from a defined group of people to get a general idea of a situation, to generalize about a population, or to get a total count of a particular characteristic

Advantages
• Many standardized instruments available
• Can be anonymous
• Allows a large sample
• Standardized responses easy to analyze
• Able to obtain a large amount of data quickly
• Relatively low cost
• Convenient for respondents

Disadvantages
• Sample may not be representative
• May have low return rate
• Wording can bias responses
• Closed-ended or brief responses may not provide the “whole story”
• Not suited for all people—
e.g., those with low reading level

Interview

Use when
You want to understand impressions and experiences in more detail and be able to expand or clarify responses

Advantages
• Often better response rate than surveys
• Allows flexibility in questions/probes
• Allows more in-depth information to be gathered

Disadvantages
• Time consuming
• Requires skilled interviewer
• Less anonymity for respondent
• Qualitative data more difficult to analyze

Focus Group

    Use when
    You want to collect in-depth information from a group of people about their experiences and perceptions related to a specific issue.

    Advantages
    • Collect multiple peoples’ input in one session
    • Allows in-depth discussion
    • Group interaction can produce greater insight
    • Can be conducted in short time frame
    • Can be relatively inexpensive compared to interviews

    Disadvantages
    • Requires skilled facilitator
    • Limited number of questions can be asked
    • Group setting may inhibit or influence opinions
    • Data can be difficult to analyze
    • Not appropriate for all topics or populations

TRAINYOUCAN NETWORK ONLINE STATUS