Every sample-based survey is subject to non-response either for the reason of refusal or problems of locating the household: household has moved, person died, address recorded incorrectly, etc. Whichever is the reason, it must be properly reflected in the interview result code - a categorical variable set by the interviewer to indicate the outcome of the visit to the household.
Survey Solutions doesn’t do sampling. A sampling specialist should perform sampling for your survey. The focus of the below discussion is not on how to properly select a replacement Y for a non-responding household X, but on:
- what mechanisms should be added to the questionnaire to accommodate one or the other strategy?
- what should be the instructions given to the field staff?
- what information should be prepared at the beginning of the survey? which information will be required later?
- how the replacements may be tracked and audited?
Below we discuss different protocols that the HQ may impose for the cases when a household must be replaced. In all of the versions below, it is assumed that the field staff has made all possible efforts to convince the household to participate, but didn’t get an interview.
Suppose we could estimate (from previous survey, from pilot data, etc)
that there will be an R=80% response to our survey. If we target
T=10,000 completed interviews, the HQ should create then
- Manual replacement
For T targets exactly T assignments are created by the HQ. If a household is refusing to participate, it’s assignment is closed with the corresponding interview result code. A new assignment is than manually created by the HQ in replacement of the original.
- Semi-automatic replacement
Daily snapshots of data are downloaded from the Survey Solutions server
automatically, utilizing the API. Automatic scripts verify the interview
result codes, and for all households that have refused and not replaced
so far generate a new preloading file, which is then manually uploaded
by an HQ user to create replacement assignments in batch mode.
- Shadow replacements
For every assignment (A) that is generated as part of the sample, a second assignment is generated as a backup (B) for the case of non-response. Interviewers are instructed to proceed to A-households, and in case of non-response, to the corresponding B-household. Unused B-assignments are closed with the interview result code “unused spare assignment”.
- Interviewer spares
For all the assignments (A) in the given enumeration area the interviewer also receives a number of replacement assignments (B), usually smaller in number (for example for 10 As receives 3 Bs). When a non-response is encountered among A, the interviewer must proceed to the next available B. Unused backup assignments are closed with the interview result code “unused spare assignment”.
- Supervisor spares
For all the assignments (A) in the given enumeration area the supervisor receives a number of replacement assignments (B), usually smaller in number (for example for 10 As receives 3 Bs). Only As are distributed to the interviewers, while Bs remain assigned to the supervisor. When a non-response is encountered among As (the supervisor receives a completed assignment of type A with a non-response result code), the supervisor assigns the next available B to the same interviewer that has reported the non-response. Unused backup assignments are left in the status “assigned to supervisor”, or closed with the interview result code “unused spare” by the fictitious interviewer.
- Sample mode + census mode
The same survey is administered in sample mode and in the census mode. If an interviewer encounters a non-response, she can create a replacement assignment and enter identifying information herself. To avoid interviewer’s biases, a firm rule must be specified on how to select the replacement, e.g. “next household in order of enumeration”.
- In place replacement
In this strategy the questionnaire itself is modified to accommodate the replacement address. This can be done by triggering the “Replacement address” section to open when the interview result code is “non-response in target household, interview obtained from a replacement household”.
In every strategy outlined above where the interviewer is allowed to enter the address of the replacing household, there is a vulnerability that the interviewer will enter the replacement address incorrectly. A two-man rule may be introduced, where the interviewers are issued a checksum computed by the supervisor for a given replacement address, or a conditional section is locked until replacement is approved by the supervisor.
In those strategies where the interviewers are issued the replacement assignments right away, it is possible to password-protect the assignment with an individual password (generated at HQ and visible to the supervisors, but not to the interviewers). In that case interviewers may only proceed to the replacement interviews when such password is communicated to them by their respective supervisor or an HQ-user.
If you don’t know which strategy to prefer, discuss whether connectivity is expected in the enumeration areas, whether the full list of the households is available before the interviewers proceed to the enumeration areas, and other factors which will help you to shorten the list of available options.
If unsure, select strategy #6 as it provides the best protection against the interviewers’ biases, albeit at a cost of requiring communication.
There are potentially other strategies, which extend the ones outlined above or achieve the same results using additional external media/lists/cards/maps etc. Depending on the nature of the survey, number of visits and other factors one or the other strategy may be more suitable, however the final quality of data will depend on how the replacement strategy has defended against the interviewers biases.