Skip to navigation
Webpage last modified: 2011-Nov-28
Interviewers play a critical role in surveys, as they implement the survey design. They are often required to perform multiple tasks with a high level of accuracy. In a face-to-face household survey, the interviewer may be required to physically locate the household and to update the sampling frame. In both telephone and face-to-face surveys, the interviewer has to contact the household, explain the purpose of the study, enumerate household members, select the respondent, motivate the respondent to participate, ask questions in the required manner, put the respondent at ease, and accurately record the respondent's answers as well as any other required information. Depending upon the survey topic and survey context, the interviewer may be required to perform additional tasks, such as bio-measure collection or oral translation.
Interviewers can influence responses through their personal attributes and their behaviors ("interviewer effects"). These guidelines present strategies to optimize interviewer efficiency and minimize the effect interviewer attributes have on the data through appropriate recruitment, selection, and case assignment; they also present strategies to minimize the effect that interviewer behaviors have on sampling error, nonresponse error, measurement error, and processing error through training.
Figure 1 shows interviewer recruitment, selection, and training within the survey production process lifecycle (survey lifecycle) as represented in these guidelines. The lifecycle begins with establishing study structure (Study, Organizational, and Operational Structure) and ends with data dissemination (Data Dissemination). In some study designs, the lifecycle may be completely or partially repeated. There might also be iteration within a production process. The order in which survey production processes are shown in the lifecycle does not represent a strict order to their actual implementation, and some processes may be simultaneous and interlocked (e.g., sample design and contractual work). Quality and ethical considerations are relevant to all processes throughout the survey production lifecycle. Survey quality can be assessed in terms of fitness for intended use (also known as fitness for purpose), total survey error, and the monitoring of survey production process quality, which may be affected by survey infrastructure, costs, respondent and interviewer burden, and study design specifications (see Survey Quality).
The structure and composition of the interviewing staff must be established during the design and planning phases of the project because these decisions will determine the number and type of interviewers required, the training protocol, sample assignment, and most efficient methods of supervision.
Since data collection staff quality has a major impact on the quality of the data collected, it is important to attract and retain the most qualified interviewers possible.
The quality of an interviewer-administered survey depends, to a large extent, on the quality of the interviewers and their supervisors. It is important, therefore, to recruit and select the best possible people for the job. In addition, selecting candidates who are well suited for the job may lead to lower interviewer turnover and reduced survey costs.
Newly hired interviewers and supervisors require basic training in techniques for successful interviewing before they receive specific training on the study on which they will be working. Research indicates that interviewer training helps improve the quality of survey data by: (1) reducing item nonresponse , (2) increasing the amount and accuracy of information obtained , and (3) increasing survey participation by teaching interviewers how to identify and respond to respondents' concerns .
Interviewers and supervisors need to be very familiar with the study's protocols and questionnaire in order to carry out their tasks. Depending upon the survey, they may need to learn the instrument's branching structure, the study's requirement for field coding, or the use of a respondent booklet, show cards, or other visual materials. There may be special instructions for implementing all or part of the survey that deviate from the standardized interviewing covered in general interviewer training. Interviewers should also be knowledgeable about the project objectives so that their actions help, not hinder, the overall goals. Both newly hired and experienced interviewers require training specific to the study at hand.
Comprehensive documentation helps analysts correctly interpret the data and assess data quality; it also serves as a resource for later studies.
Research indicates that the interviewer design effect may be even larger than the design effect attributable to geographic clustering . This is especially true in some international studies where cultural and other factors contribute to large interviewer variances. Interviewer variance occurs when response errors of persons interviewed by the same interviewer are correlated; therefore interviewer variance is part of the correlated variance component of the total variance (other correlated variances stem from coders, editors, supervisors, and crew leaders).
The intra class coefficient, pint, is a measure of the ratio of interviewer variance to the total variance and is defined as:
The value of pint is theoretically always between 0 and 1 although calculated estimates of pint may sometimes be negative. In this case, they are usually treated as zeros. When pint for a particular variable is 0 or is negative, we interpret this to mean that the interviewers have no effect on the variance of responses to that variable; the larger the value of pint, the larger the effect of interviewers on the variance of the particular variable.
The interviewer design effect (deffint) is a measure of the effect of interviewers carrying out multiple interviews, compared to what you would get if there was a different interviewer for each respondent, all else being equal (if the addition of more interviewers increases costs such that supervision or training must be reduced to compensate, interviewer variance may actually increase).
where m is the average number of interviews per interviewer.
Thus, even a small interviewer variance (pint,) can have a significant effect on the variance of a survey estimate if m is large. The interviewer variance contribution is usually not included in textbook variance estimation formulas. Interviewer variance leads to a loss of sample information when the effective sample size neff, defined as n/deffint, is smaller than the actual sample size n.
Standardized interviewing aims to reduce interviewer variance.
The following example shows how to calculate the number of interviewers required for a hypothetical study. The example makes the following assumptions:
Make the following calculations:
The Survey of Health, Ageing and Retirement in Europe (SHARE) utilizes a model agenda for training interviewers in participating countries. While the content of this agenda is SHARE study specific, it might provide a useful basic template for other similar cross-national survey efforts. Organizations may add country-specific items to the model training agenda (e.g., tracing/locating steps that should be followed in their country and any relevant cultural considerations).
|Topic||Purpose||Panel: Time (minutes)||Baseline: Time (minutes)|
|Introduction, Welcome, and Logistics||Set the stage for this intense training.||15||30|
|SHARE Project and Questionnaire Overview||Explain the goals of the project and the importance of baseline and longitudinal sample||45||45|
|Sample Overview||Understand how the sample was selected, sample eligibility, and response rate requirements.||30||60|
|GIT Requirements||Cover minimal GIT requirements, including when and how to contact sample, probes, feedback, etc.||60||60|
|Overview of the Sample Management System||Learn how to operate the SHARE electronic sample management system, assign result codes, and enter call notes. Introduce noncontact mock scenarios and test results.||60||90|
|Longitudinal Sample Management System||Introduce splitters, deceased, new eligible respondents, and additional result codes.||30||Na|
|Proxy Interviews||Explain how to identify and interview proxy respondents.||30||45|
|Nursing Homes||Explain how to contact respondents in nursing homes and to work with gatekeepers / potential proxy respondents.||30||Na|
|Overview of the Blaise Program||Explain the Blaise program conventions, including different types of questions, question wording, data entry, interviewer instructions, etc.||45||45|
|SHARE Questionnaire Walk-Through||Describe SHARE modules. Conduct a scripted review of the questionnaire, including spawning of additional line. Address main questions and issues that arise with different sections. Longitudinal: Describe longitudinal differences. Explain preloads. Address different questions arising from reinterviews.||330||240|
|End-of-Life Interviews (EOL)||Cover the concept of the EOL interview, approaching respondents, and administering the interview. Explain how to record these in the Sample Management System (SMS).||30||Na|
|Drop-Off||Describe drop-off and the procedure for identifying and labeling drop-off appropriately. Explain procedure for administering drop-off and how to record these in SMS.||45||45|
|Physical Measurements; "Certification"||Have each interviewer demonstrate the ability to conduct physical measures.||30||60|
|Response Rates and Contact Efforts||Explain the importance of response rates and the reiteration of required contact effort per line. (Longitudinal: review only) Longitudinal: Cover panel care and effort requirements, including tracking effort.||45||90|
|Gaining Respondent Cooperation||Review eight concerns that interviewers are likely to encounter. Practice quick answers to several concerns. Note that longitudinal sample is more likely to encounter different types of resistance.||90||90|
|Practicing Household Introductions||Have interviewers team up in groups of 10 or so and each take a turn introducing the study.||Optional||90|
|Pair-wise Questionnaire Walk-through||This is an opportunity for interviewers to go through the questionnaire with a fellow interviewer. Use an abbreviated script. Switch at the half-point mark and complete the interview.||90||130|
|Pair-wise EOL Interview||Practice administering the End-of-Life interview.||45||Na|
|Administrative Wrap-Up||Answer outstanding questions.||30||30|
|Certifier Notes for the Individual Certifications|
|CERTIFIER INSTRUCTIONS: Score each item 0, 1, or 2. 0 = Inadequate performance; 1 = Needs Improvement; 2 = Met Expectations. Use the Errors column to tally the number of times the interviewer makes general interviewing technique (GIT) errors in reading, probing, feedback, or clarification. Note question numbers of errors when possible.|
|On time and prepared for certification||Sample Management System running and ready to interview; for face-to face interview, have respondent materials ready, including copy of letter and brochure.|
|Correctly completing household listing/enumeration and screener||Make sure that the interviewer has completed the household listing/enumeration correctly; if not, tell him/her how to correct and proceed. The interviewer will have to recertify on the screener portion if this happens.|
|Use of GIT probes and clarification||Should use standard GIT protocol as indicated; 1 - 2 errors - score 1; 3+ errors - score 0.|
|Use of neutral feedback||Interviewer should provide feedback for at least 30% of responses. Non-standard feedback counts as an error.|
|Verbatim question reading||Include pronunciation and emphasis in evaluation; 1-3 errors - score 1; 4+ errors - score 0.|
|Data Entry||General comfort with navigating in Blaise.|
|Post-interview process & contact person information||Interviewer should confirm all contact information for respondent and enter information for required number of contact persons.|
|Contact attempt record||Interviewer should enter a final contact attempt note which you will check before scoring. If 1 or 2 items are missing - score 1. If more than 2 items are missing - score 0.|
|Total possible = 16 Certified = 12or higher Re-Certify = 10-11 Administrative Review will be required if score is less than 10.|
|GENERAL COMMENTS: Provide specific examples and question numbers of problem areas when possible. Note the way in which the interviewer
administered the informed consent and reads the script to explain the need for obtaining information for contact persons.
|Debriefing with Interviewer by [NAME]:||Date:|
|Notes: Include summary of recertification plan and retraining or practice interviews needed. Make note of areas that need close review on taped interviews.
|Feet flat on floor/legs uncrossed||2|
|Loose clothing/ no more than one layer||4|
|Arm on the table (or supported) at heart level||2|
|Use of correct cuff size||4|
|Tube of cuff hanging at inner crease of arm||4|
|Start at 180 SBP (Systolic Blood Pressure)||2|
|Re-inflation no sooner than 30-45 sec.||4|
|Re-inflation to first SBP + 20||1|
Total incorrect Blood Pressure: ___________
Max incorrect to pass: 3 (4 or more needs recertification)
|Heels to wall||2|
|Place sticky properly on wall||2|
|Orange triangular ruler on top of head, parallel to floor (fat edge against the wall)||4|
|Place metal tape measure properly and straight for accuracy in measuring height||4|
|Remove sticky from wall when done||1|
|For leg length, ask respondent to locate bony prominence and hold metal tape in place there; keep tape straight||4|
Total incorrect (Height): ___________
Max incorrect to pass: 3 (4 or more needs recertification)
|Place scale on firm floor||4|
|Remove bulky clothes||2|
|Tap red label on scale; wait for "000.0"||4|
Total incorrect (Weight): ___________
Max incorrect to pass: 2 (3 or more needs recertification)
|Ask respondent to identify umbilicus (navel) and hold cloth tape in place there||3|
|One layer of clothing||3|
|Tape snug but not tight||1|
|Check that tape is horizontal all around the R||3|
|Ask respondent to take normal breath and exhale, holding breath at end of exhalation||1|
|Record to nearest centimeter||3|
Total incorrect (Waist): ___________
Max incorrect to pass: 2 (3 or more needs recertification)
|Take measurement at respondent's side||2|
|Place cloth tape at level of maximal protrusion of gluteal muscles||1|
|Tape snug but not tight||3|
|Check that it is horizontal all around the respondent||3|
|Move tape up and down to make sure measurement is taken at greatest diameter||1|
|Ask the respondent to take normal breath and exhale, holding breath at end of exhalation||1|
|Record to nearest centimeter||3|
Total incorrect (Hip): ___________
Max incorrect to pass: 2 (3 or more needs recertification)
 Alcser, K. H., & Benson, G. D. (2005). The SHARE train-the-trainer program. Ch. 6 In A. Boersch-Supan & H. Juerges (Eds.), The survey of health, ageing and retirement in Europe - Methodology. Mannheim, Germany: Mannheim Research Institute for the Economics of Aging.
 Alcser, K. H., & Benson, G. D. (2008). Training for SHARE Wave 2. Ch. 8.3 in A. Boersch-Supan &, A. Brugiavini, H. Juerges, et al. (Eds.). Health, Ageing and Retirement in Europe (2004-2007). Mannheim, Germany: Mannheim Research Institute for the Economics of Aging.
 Axinn, W. G. (1989). Interviewers and data quality in a less developed setting. Journal of Official Statistics, 5(3), 265-280.
 Axinn, W.G., Fricke, T.E., & Thornton, A. (1991). The micro-demographic community study approach: Improving survey data by integrating the ethnographic method. Sociological Methods Research, 20(2), 187-217.
 Beatty, P. (1995). Understanding the Standardized/Non-Standardized Interviewing Controversy. Journal of Official Statistics, 11(2),147-160.
 Belli, R., Shay, W., & Stafford, F. (2001). Event History Calendars and Question Lists. Public Opinion Quarterly, 65(1), 45-74.
 Biemer, P. & Lyberg, L. (2003). Introduction to survey quality. New York, NY: John Wiley & Sons.
 Billiet, J., & Loosveldt, G. (1988). Improvement of the quality of responses to factual survey questions by interviewer training. Public Opinion Quarterly, 52(5), 190-211.
 Börsch-Supan, A., Jürges, H., & Lipps, O. (2003). SHARE: Building a panel survey on health, aging and retirement in Europe. Mannheim, Germany: Mannheim Research Institute for the Economics of Aging.
 California Health Interview Survey (2007). 2005 Methodology series: Report 2 — Data collection methods. Los Angeles, CA: UCLA Center for Health Policy Research, 2009
 Cannel, C. F. et al. (1977). A summary of studies of interviewing methodology.Vital and Health Statistics: Series 2, Data Evaluation and Methods Research. (No. 69. DHEW Publication No. (HRA) 77-1343). Washington, DC: U.S. Government Printing Office. Retrieved December 29, 2009, from www.eric.ed.gov/ERICWebPortal/custom/portlets/recordDetails/detailmini.jsp?_nfpb=true&_&ERICExt Search_SearchValue_0=ED137418&ERICExtSearch_SearchType_0= no&accno=ED137418.
 Cantave, M., Kreuter, F., & Alldredge, E. (2009). Effect of pay structure on interviewer productivity. Poster presented at the 2009 AAPOR meetings.
 Christopher, S., McCormick, A. K. H. G., Smith, A., & Christopher, J. C. (2005). Development of an interviewer training manual for a cervical health project on the ApsÃƒÂ¡alooke reservation. Health Promotion Practice, 6(4), 414-422.
 Conrad, F., & Schober, M. F. (1999). Conversational interviewing and data quality. Proceedings on the Federal Committee on Statistical Methodological Research Conference. Retrieved February 14, 2008, from www.bls.gov/ore/abstract/st/st990250.htm.
 Davis, D. W. (1997). Nonrandom measurement error and race of interviewer effects among African Americans. Public Opinion Quarterly, 61(1), 183-207
 Doyle, J. K. (2004). Introduction to interviewing techniques. Ch. 11 in Wood. D. W. (Ed.), Handbook for IQP Advisors and Students. Prepared for the Interdisciplinary and Global Studies Division. Worcester, MA: Worcester Polytechnic Institute. Retrieved March 26, 2010, from www.wpi.edu/academics/GPP/Students/handbook.html.
 European Social Survey. (2004). Field procedures in the European Social Survey: Enhancing response rates. Retrieved January 24, 2008, from http://www.europeansocialsurvey.org/index.php?option=com_docman&task=doc_view&gid=160
 Federal Committee on Statistical Methodology. (1998). Interviewer training. Ch. 5 in Training for the future: Addressing tomorrow's survey tasks (pp. 50-61). (Report number NTIS PB99-102576). Washington, DC: U.S. Government Printing Office. Retrieved March 26, 2010, from http://www.fcsm.gov/working-papers/1998_intro-appD.html
 Fowler, F.J. Jr. & Mangione, T.W. (1985). The value of interviewer training and supervision. Final report to the National Center for Health services Research. Boston, MA: Center for Survey Research.
 Fowler, F. J. & Managione, T. W. (1990). Standardized survey interviewing: Minimizing Interviewer-related error. Beverly Hills, CA: Sage Publications
 Galbraith, M. (2003). Adult learning methods (3rd ed.). Malabar, FL: Kreiger Publishing Co
 Glewwe, P. (2005). Chapter IV: Overview of the implementation of household surveys. In United Nations Statistical Division, United Nations Department of Economic and Social Affairs (Eds.), Household surveys in developing and transition countries. New York. NY: United Nations. Retrieved March 26, 2010, from http://unstats.un.org/unsd/HHsurveys/pdf/Household_surveys.pdf
 Groves, R. (1989). Survey errors and survey costs. Hoboken, NJ: John Wiley & Sons
 Groves, R. M., Fowler, F. J. Jr., Couper, M. P. , Lepkowski, J. M. Singer, E. & Tourangeau R. (2009). Survey methodology (2nd ed.). Hoboken, NJ: John Wiley & Sons
 Groves, R. M., & McGonagle, K. A. (2001). A theory-guided interviewer training protocol regarding survey participation. Journal of Official Statistics, 17(2), 249-265.
 Hansen, M. H., Hurwitz, W. N., Marks, E. S., & Mauldin, W. P. (1951). Response errors in surveys. Journal of the American Statistical Association, 46(254), 147-190
 Hatchett, S., & Schuman, H. (1975). White respondents and race-of-interviewer effects. Public Opinion Quarterly, 39(4), 523-528.
 Hursh-César, R. P. (1976). Third world surveys: Survey research in developing nations. Delhi: Macmillan Company of India.
 Jansen, H. A. F. M., Watts, C., Ellsberg, C., Heise, L., & García-Moreno, C. (2004). Interviewer training in the WHO multi-country study on women's health and domestic violence. Violence against Women, 10(7), 831-849.
 Kane, E. W., & Macaulay, L. J. (1993). Interviewer gender and gender attitudes. Public Opinion Quarterly, 57(1), 1-28
 Kessler, R. C., Ustun, T. B., & World Health Organization. (2008). The WHO world mental health surveys: Global perspectives on the epidemiology of mental disorders. Cambridge; New York :Geneva: Cambridge University Press; Published in collaboration with the World Health Organization.
 Kish, L. (1962) Studies of interviewer variance for attitudinal variables. Journal of the American Statistical Association, 57(297), 92-115.
 Lavrakas, P. J. (1993). Telephone survey methods: Sampling, selection, and supervision (2nd ed.). Thousand Oaks, CA: Sage Publications.
 Lee, R.M. (1993). Doing research on sensitive topics. London, UK: Sage Publications.
 Liamputtong, P. (2010). Performing qualitative cross-cultural research. Cambridge, UK: Cambridge University Press.
 Living Standard Measurement Study Survey. (1996). Working paper 126: A manual for planning and implementing the Living Standard Measurement Study Survey. Retrieved July 10, 2010, from http://www-wds.worldbank.org/external
 Martin, E. (1996). Household attachment and survey coverage. Proceedings of the section on survey research methods (pp. 526-531). Alexandria, VA: American Statistical Association.
 Nyandieka, L. N., Bowden, A., Wanjau, J., & Fox-Rushby, J. A. (2002). Managing a household survey: A practical example from the KENQOL survey. Health Policy and Planning, 17(2), 207-212
 O'Brien, E. M., Mayer, T. S., Groves, R. M., & O'Neill, G. E. (2002). Interviewer training to increase survey participation. In Proceedings of the American Statistical Association: Survey Research Methods Section. Alexandria, VA: American Statistical Association
 Office of Management and Budget. (2006). Standards and guidelines for statistical surveys. Washington, DC: Office of Information and Regulatory Affairs, OMB. Retrieved January 10, 2010, from http://www.whitehouse.gov/omb/inforeg/statpolicy/standards_stat_surveys.pdf
 O'Muircheartaigh, C. & Marckward, (1980). An assessment of the reliability of World Fertility Study Data. In Proceedings of the World Fertility Survey conference, Vol. 3, 305-379. The Hague, The Netherlands: International Statistical Institute
 Papanek, H. (1979). Research on women by women: Interviewer selection and training in Indonesia. Studies in Family Planning, 10(11/12), 412-415.
 Pennell, B-E., Harkness, J., & Mohler, P. (2006). Day 8: Data collection, interviewers, and interviewer training. SRC Summer Institute Presentation. University of Michigan, Ann Arbor, MI.
 Pennell, B-E., Harkness, J., Levenstein, R., & Quaglia, M. (2010). Challenges in cross-national data collection. In J. Harkness, B. Edwards, M. Braun, T. Johnson, L. Lyberg, P. Mohler, B. E. Pennell, & T. Smith. (Eds.), Survey methods in multicultural, multinational, and multiregional contexts. New York: John Wiley & Sons.
 Pennell, B-E., Mneimneh, Z., Bowers, A., Chardoul, S., Wells, J. E., Viana, M. C., et al. (2009). Implementation of the World Mental Health survey initiative. In R. C. Kessler & T. B. Üstün (Eds.), Volume 1: Patterns of mental illness in the WMH surveys. Cambridge, MA: Cambridge University Press.
 Quaglia, M. (2006). Interviewers and interpreters: Training for a biographical survey in Sub-Saharan Africa. Paper presented at Proceedings of Q2006 European Conference on Quality in Survey Statistics. Retrieved April 30, 2008, from http://www.statistics.gov.uk/events/q2006/agenda.asp
 Schaeffer, N. C. (1980). Evaluating race-of-interviewer effects in a national survey. Sociological Methods & Research, 8(4), 400-419.
 Schnell, R. & Kreuter, F. (2005). Separating interviewer and sampling-point effects. Journal of Official Statistics, 21(3), 389—410.>
 Schuman, H., & Converse, J. M. (1971). The effects of black and white interviewers on black responses in 1968. Public Opinion Quarterly, 35(1), 44-68
 Stycos, J. M. (1952). Interviewer training in another culture. Public Opinion Quarterly, 16(2), 236-246.
 Suchman, S. & Jordan, B. (1990). Interactional troubles in face to face survey interviews. Journal of the American Statistical Association, 85, 232-241.
 Sudman, S. (1966). New approaches to control of interviewing costs. Journal of Marketing Research, 3(1), 55-61
 Ustun, T. B., Chatterji, S., Mechbal, A., & Murray, C. J. L. (2005). Chap X: Quality assurance in surveys: Standards, guidelines, and procedures. In United Nations Statistical Division, United Nations Department of Economic and Social Affairs (Eds.), Household surveys in developing and transition countries. New York, NY: United Nations. Retrieved March 26, 2010, from http://unstats.un.org/unsd/HHsurveys/pdf/Household_surveys.pdf
 Vaessen, M., Thiam, M., & Le, T. (2005). Chapter XXII: The demographic and health surveys. In United Nations Statistical Division, United Nations Department of Economic and Social Affairs (Eds.), Household surveys in developing and transition countries. New York: United Nations. Retrieved March 26, 2010, from http://unstats.un.org/unsd/HHsurveys/pdf/Household_surveys.pdf
 Weisberg, H. (2005). The total survey error approach. Chicago, IL: University of Chicago Press.
 West, B. & Olson, K. (2009) How much of interviewer variance is really nonresponse error variance? Michigan Program in Survey Methodology Brownbag Presentation. University of Michigan, Ann Arbor, MI.
 Afrobarometer Survey. Retrieved July 1, 2010, from http://www.afrobarometer.org/questionnaires/R4surveymanual_4feb08_FINAL.pdf
 Asian Barometer. Retrieved July 1, 2010, from http://www.asianbarometer.org/newenglish/introduction/default.htm
 European Social Survey. Retrieved July 1, 2010, from http://www.europeansocialsurvey.org/index.php?option=com_content&view=article&id=23&Itemid=318
 Survey of Health, Ageing, and Retirement in Europe. Retrieved July 1, 2020, from http://www.share-project.org
© 2008 The authors of the Guidelines hold the copyright. Please contact us if you wish to\n publish any of this material in any form.