IB Theory of Knowledge 1.8.6 Surveys, Interviews and Self-report Notes

1.8.6 Surveys, interviews and self-report

What a survey answer can tell you

1. A tick in a box is a piece of behaviour before it is a piece of information. Somebody read a question, worked out what it was asking, decided what to say, and put a mark down.
2. Three people can select the same option for incompatible reasons: one because it describes them, one because it came nearest to describing them, one because the other options looked worse to admit to.
3. If the dataset retains only the selected option, those reasons are lost. Once the answers are added up, the total combines people who may have understood or used the categories differently.
4. Ask what shaped the answer: the wording, the setting, the presence of an interviewer and the question the respondent thought they were being asked.
5. How reliable are questionnaire findings when wording, leading questions and selection affect the answers? Distinguish these influences from deliberate misreporting, and consider how each could be checked.
6. A self-report survey records what people report about their behaviour, beliefs or experiences. Explain what evidence connects those reports to the particular claim you want to make.

Standardised questions and adaptive interviews

1. A survey can use standardised questions to improve comparability. Some surveys vary order or route respondents to different questions, so check what each person was asked before comparing answers.
2. An interview can be structured or allow follow-up questions. In a flexible interview, the researcher can clarify a question and investigate detail that a fixed form might miss.
3. Fixed wording helps comparison across many participants, while adaptive questioning can reveal differences in meaning. Neither guarantees that everyone understands a question in the same way.
4. A semi-structured interview is an attempt to hold both at once, with a fixed core and room to follow what comes up. It reduces the loss on each side without removing it.
5. Surveys and interviews can both collect self-reports. In either format, the account depends on what the person can recall, understand and is willing to share.
6. Willing and able fail separately. Somebody may decline to report what they know perfectly well, and somebody may report in good faith about a state they have no reliable access to, such as why they made a choice.

Social desirability bias in self-reports

1. Social desirability bias occurs when people's answers are shaped by what they regard as acceptable or favourable. This pressure can operate in an interview or on a form whose readers are only imagined.
2. It can lead to over-reporting of approved behaviour or under-reporting of disapproved behaviour. Such systematic errors need not cancel as the sample grows.
3. In its 2016 analysis of reported turnout in the 2014 US election, Pew Research Center found more respondents said they had voted than had a matching voter-file record. Pew considered both over-reporting and errors in the records or matching process.
4. Disclosure can depend on who sees an answer. Speaking to an interviewer, completing a paper form and using a private screen are different conditions whose effects need testing.
5. Anonymity, self-completion and neutral wording can reduce some pressures to give an acceptable answer. They do not guarantee accuracy, and the useful question is how the method affects reporting.
6. Evidence about social pressure can suggest the likely direction of one source of error. It does not by itself establish the direction or size of the survey's total error, since other influences may pull differently.

Closed and open questions preserve different information

1. A closed question gives respondents a fixed set of answer options, which makes coding and comparison easier.
2. The options reflect the designer's categories. Someone whose experience fits none of them may select a poor match, use an "other" option if offered, or leave the question unanswered.
3. An open question hands the categories back. What comes out is in the respondent's own words and can carry a distinction the form never anticipated.
4. To turn open answers into counts, researchers code them into categories. Examine the coding rules and how disagreements were handled; those decisions can be reported for others to inspect.
5. A rating scale also needs interpretation. The gap between four and five on a seven-point scale need not mean the same thing to every respondent. Averaging ratings requires assumptions about what the scale measures and how its values can be compared.
6. Choose the format to fit the claim. Closed questions can support counts, while open questions can reveal meanings the researcher had not anticipated. Coded open answers can also be counted, and a study can combine formats.

Natsal-3: privacy, reporting and the limits of a survey answer

1. Natsal-3 is the third British National Survey of Sexual Attitudes and Lifestyles. Its fieldwork ran from 2010 to 2012, it interviewed 15,162 people aged 16 to 74, and the findings appeared in The Lancet in 2013.
2. Interviews took place in people's homes, with the interviewer working through a laptop. The most sensitive questions were moved into a module the participant completed on that laptop directly, with the interviewer unable to see the answers.
3. The self-completion module was designed to make sensitive disclosure easier. That choice addresses a possible source of reporting bias, but the use of the module alone does not establish how much it changed any particular answer.
4. Men in Natsal-3 reported more opposite-sex partners on average than women. Equal partnership totals would be expected if both sides of every partnership belonged to the same fully observed population, but a national survey has age, residence and participation boundaries as well as reporting error.
5. Mitchell and colleagues examined several possible contributions to the gap: sampling, unusually high reported counts, counting versus estimating, and attitudes that might affect reporting. These factors can be investigated without assuming the entire difference is caused by dishonesty.
6. The recorded answer establishes what a participant reported under those conditions. Interpreting it as evidence of behaviour requires attention to recall, definitions and the population covered.

Access to private experience against the risk of an unverifiable account

1. A person has first-hand access to pain and experience that an outside observer lacks. Researchers can examine related behaviour or physiological signals, but those measures do not automatically capture what the experience felt like.
2. Self-report provides evidence about that first-person experience. Its value depends on the question, the person's ability to describe it and the way researchers interpret the account.
3. Another person cannot report the same experience from the same first-person position. That limits direct checking, while leaving room to compare the account with relevant observations or records.
4. Partial checks exist and are worth stating in an argument. Consistency across repeated asking, agreement with what other people say about the same person, and agreement with conduct that can be observed each narrow the possibilities without closing them.
5. When may a researcher use their own experience as evidence? Compare the checks needed for that use with the checks applied to a participant's account, and explain what each can establish.
6. A unique account can be relevant evidence even when the event cannot be repeated. Evaluate its recording, context and corroboration instead of assuming unrepeatability alone makes it unusable in either the human or natural sciences.

What a self-report proves depends on the claim an essay is making

1. Match the evidence to the claim. A self-report can support an inference about attitudes, experience or behaviour, but its strength depends on recall, disclosure and other checks. Explain the inference and qualify it where those checks are missing.
2. Two researchers can agree on the recorded survey answers while disagreeing about what they establish. Compare their reasons and evaluate the evidence each uses to connect the answers to the claim.
3. What must be added to a person's account before researchers treat a claim based on it as well established? Consider which standards fit the kind of knowledge being sought.
4. Compare studies through a shared question. Explain what a survey and an interview each establish, rather than listing several findings without analysing their methods.
5. The weakest use is a percentage as decoration. A figure quoted to open a paragraph, with no account of who was asked or how, borrows authority the argument has not earned and leaves a reader nothing to weigh.