Survey Research

Psy 242

Open-ended vs. closed-ended questions

- open-ended: harder to score, but more likely to find out what respondents really think

- closed-ended: easier to score, but may not have the respondent's true answer listed as a choice. Should be:

o Mutually exclusive - no overlap in the

o Exhaustive - the response options cover all possible answers

Likert scales vs. yes/no questions

- Likert scale

o Numbers indicate degree of agreement with a statement

o Ordinal or interval scale

o Advantages: Can get finer grained information, detect more subtle differences

- Yes/No

o Nominal scale

o Advantages: Forces a choice - avoids the problem of respondents answering "maybe" or "sort of" to everything

Avoid ambiguity - address a single question per item

Avoid bias

- response bias: Have some questions for which “yes” means you like pets, some for which “yes” means you don’t

- social desirability bias – Try to avoid words with negative connotations and word the questions as neutrally as possible

- Sequence of items – Could seeing one question bias the answer of another question if it came afterwards?

Reliability and Validity

- reliability = consistency

o inter-item reliability of a scale: usually measured by “split-half” reliability or by Cronbach’s alpha (will be covered in lab)

o scale = a combination of a set of questions that all measure the same construct or idea

- validity = measuring what it is supposed to

o more complicated to evaluate than reliability

o empirical validity = correlation with a criterion or a related measure

o Face validity = appears to measure the concept it is supposed to; the weakest form of validity, but what you will probably have to rely on in your surveys for project 1.

Sampling

o Haphazard sample = chosen with unsystematic "hit or miss" methods; virtually worthless

o Convenience sample = nonrandom sample chosen for practical reasons

§ What distinguishes convenience from haphazard samples? Whether you have a clear idea of what population the sample is representative of (who you can generalize the results to).

o Probability sample = one in which the researcher knows the probability of each person in the population being sampled

§ Random sample = each person in the population has an equal and independent chance of being sampled

§ Systematic sample = a probability sample that is not randomly selected (eg: every 20^th name in a list).

§ Stratified random sample = representing 2 subpopulations in a pre-determined proportion (eg: 50% white, 50% black)

§ Cluster samples = randomly selected groups rather than individuals

· Multi-stage sampling = sampling clusters within clusters (eg: randomly select 5 states, then randomly select 5 counties in each, then 5 zip codes in each, etc.)

- “Sampling Frame” – the population that the sample actually comes from; generally no sample is a perfectly unbiased one; always wind up excluding some portion of the intended population from the sampling frame.

Correlation

- Surveys are often used to look for relationships between two variables or constructs

- These relationships are often measured as correlations